diff --git a/README.md b/README.md
index f3f08822..3ca5f90c 100644
--- a/README.md
+++ b/README.md
@@ -226,9 +226,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
Automations act
on that stream, on a schedule, at a time or when their own webhook is called, alone or with their triggers joined (all
of them, a number of times, one that doesn't come in time, or a batch): they test conditions (text, patterns,
- numbers, lists, times, days, whether a value changed and whether something like it happened before in their
- namespaces) and let a decision model pick the branch, then tell you in
- your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a
+ numbers, lists, times, days, whether a value changed, the entities an asset mentions (any person, or names like
+ Acme*) and whether something like it happened before in their namespaces) and let a decision model pick the branch,
+ then tell you in your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a
workflow, pipeline or routine, or change assets and notes. The Schedules page is Lens's crontab: every routine,
automation schedule and the backup in one list with when each runs next and last, to switch, re-time or run now, and
agents do the same over the API and MCP. A Classify node in a workflow sorts an asset into labels, content types or
diff --git a/changelog.d/automation-mentions.md b/changelog.d/automation-mentions.md
new file mode 100644
index 00000000..ada1f13f
--- /dev/null
+++ b/changelog.d/automation-mentions.md
@@ -0,0 +1,6 @@
+- **Automations fire on any entity of a type, or names matching a pattern.** A trigger's where (and an If, and a
+ condition node's tree) can test what an activity mentions: `{"op": "mentions", "value": {"type": "PERSON"}}` fires
+ on any person, `{"name": "Acme*"}` on names like Acme, with `*` and `?` as wildcards, and `{"id": "lens:entity/9"}`
+ on one entity. A processed asset is now tagged with its fifty most mentioned entities, not ten, and every entity tag
+ carries its `entityType`. Conditions saved before work as they did
+ ([Operators](docs/automations.md#operators)).
diff --git a/docs/automations.md b/docs/automations.md
index b3861310..124f8381 100644
--- a/docs/automations.md
+++ b/docs/automations.md
@@ -159,7 +159,8 @@ that triggers subscribe to. `events.check_activity(a)` checks it, naming the fir
- `object`: what it is about, with the kind's own fields. Ids are `lens:
/` references, so actions can act
on them; `events.ref_id(ref, "recording")` reads the id back.
- `target`: where to, when the verb has one (Add to a collection), else null. `tag`: up to 50 objects such as
- `{"type": "Mention", "id": "lens:entity/9", "name": "St Mary's"}`.
+ `{"type": "Mention", "id": "lens:entity/9", "name": "St Mary's", "entityType": "ORG"}`: an entity's type is its
+ code, so a trigger can fire on any person mentioned ([`mentions`](#operators)).
- `summary`: one line of at most 300 characters, made by fixed rules, never by a model.
- `space`: the namespace id, or null for activities of no namespace.
- `lens`: Lens's own fields. `source` and `pos` (the source's position, `[time, id]`) are for the stream machinery.
@@ -228,8 +229,8 @@ imported), else `asset.added`. A file an email or archive held, made a resource
email is what arrived. A trigger on `asset.added` matches every one of them.
A run of an asset's pipeline that finishes is `asset.processed` (it succeeded) or `asset.failed`, one activity per
-run, batch runs included. Its actor is whoever queued it, and its tags are the asset's entities (the ten most
-mentioned), which exist only once it has been processed: `asset.processed` is the kind for "when an email about St
+run, batch runs included. Its actor is whoever queued it, and its tags are the asset's entities (the fifty most
+mentioned, as many as an activity holds, each with its `entityType`), which exist only once it has been processed: `asset.processed` is the kind for "when an email about St
Mary's is understood". A run tried again finishes again, as another activity. A deletion is logged as
`asset.deleted` where the asset is deleted (once per asset), and a move to another namespace as `asset.moved`, a
`Move` by whoever moved it, seen in the namespace it is in now (a trigger there on `asset.moved` is "when something
@@ -459,6 +460,7 @@ its items does (as `contains` does), and `not_in` passes when none does. A value
| `weekday` | days: `["sat", "sun"]`, `["monday"]` or ISO numbers (1 is Monday) | is a time on one of those days |
| `time_between` | two times of day: `["22:00", "06:00"]` | is a time of day from the first up to the second, across midnight when the second is earlier |
| `changed` | none, or a path to keep values apart by: `actor.id` | differs from the value this test saw last time (the first one seen is no change) |
+| `mentions` | the entity: `{"type": "PERSON"}`, `{"name": "Acme*"}`, both, or `{"id": "lens:entity/9"}` | is an activity (its `tag`), or a list of tags, naming an entity of that type (any of a list), with a name like that, or with that id |
- **Times** are ISO 8601 times or dates. A time written without a zone (in the value or in what is tested) is in the
automation's time zone, and so are `weekday` and `time_between`. Durations are a whole number and `s`, `m`, `h` or
@@ -468,6 +470,20 @@ its items does (as `contains` does), and `not_in` passes when none does. A value
may start alike (`(a|aa)+`) or refers back to a group (`\1`) is refused when it is saved, a pattern reads at most
the first 20,000 characters of a value, and a match that takes over 50 ms counts as no match. Compiled patterns are
cached.
+- **Mentions** are the entities an activity is tagged with (`asset.processed`, `entity.changed`). A type is its code
+ or its label in any case (`PERSON`, `person`, `Organisation`; `ORG`, `PRODUCT`, `PLACE`, `EVENT`, `WORK`, `TERM`,
+ `DATE`, `NUMBER`). In a name, `*` stands for any run of characters and `?` for one, in any case, and every other
+ character is itself; a name with neither is the whole name (`Acme` doesn't find “Acme Robotics”, `Acme*` does). A
+ value that is only text is a name. With no path the test reads the activity's `tag`; in the If give the path
+ (`event.tag`). The name is matched in time linear in its length, however many stars it has, so a crafted pattern
+ can't tie up the worker. When St Mary's or any person is mentioned in a processed asset:
+
+ ```json
+ {"as": "said", "on": "event", "kinds": ["asset.processed"],
+ "where": {"any": [{"op": "mentions", "value": {"name": "St Mary*"}},
+ {"op": "mentions", "value": {"type": "PERSON"}}]}}
+ ```
+
- **`env`**: `match.test(tree, value, env)` reads time and state from `env`, never from globals, so tests and dry
runs can fix them: `now` (the clock when not given), `timezone` (UTC when not given) and `state`.
- **`changed`** compares with the last value the same test saw for the same automation and trigger, kept in
@@ -1468,6 +1484,9 @@ built yet is in [Refine later](#refine-later); the [Timeline](timeline.md) has i
- Variables inside match values (`{"var": "vars.threshold"}`).
- Statistical conditions, such as a reading above its 7-day average.
+- `mentions` on an asset's top fifty entities: an asset that mentions more names only its most mentioned; a picker for
+ the entity and its type in the canvas's condition node (today it is written in a where or over the API and MCP);
+ an `entity.mentioned` kind, one per entity, if one per processed asset proves too coarse.
- Sequences longer than two steps ("A, then B, then C").
- The canvas shows and edits match trees on condition, filter and switch nodes (today it edits one test; a tree
saved through the API is kept).
diff --git a/docs/requirements.md b/docs/requirements.md
index 23b71a53..6c903c8a 100644
--- a/docs/requirements.md
+++ b/docs/requirements.md
@@ -278,6 +278,7 @@ request, to another check that proves the same thing; loosening a requirement or
| They test conditions, then tell you, ask the model or the agent, use a tool, call a web address with sealed credentials, run Lens work, or change assets and notes | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`; [test_actions_messages.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_messages.py) `test_notify_tells_me_in_my_chat_app`; [test_actions_ai.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_ai.py) `test_ask_model_answers_in_text_or_as_the_fields_a_schema_names`, `test_ask_agent_works_on_what_happened_and_saves_the_conversation`, `test_call_tool_reads_as_the_person_and_changes_only_by_asking_them`; [test_actions_web.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_web.py) `test_credentials_in_headers_are_sealed_to_the_automation_and_the_host_and_never_shown`; [test_actions_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_run.py) `test_run_workflow_queues_it_on_the_asset_as_the_automation`, `test_run_pipeline_processes_the_asset_again`; [test_actions_lens.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_lens.py) `test_set_field_is_kept_in_the_assets_history_by_the_automation`, `test_write_note_writes_a_page_adds_to_a_things_page_or_notes_an_asset` |
| What reaches outside Lens, or changes more than what set it off, asks first: on the Approvals page or with a yes in chat; agents decline but never approve | [test_automation_spec.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_spec.py) `test_risk_follows_the_effect`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_approve_carries_the_action_out_and_the_run_records_it`, `test_agents_over_mcp_list_and_decline_but_never_approve`; [test_approvals_chat.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals_chat.py) `test_a_yes_in_my_chat_decides_an_action_inside_lens`; [approvals-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/approvals-page.test.tsx) `groups what waits, puts the first in focus, and Enter approves it` |
| Conditions test text, patterns, numbers, lists, times, days and whether a value changed, checked as they're saved and read in words | [test_match_ops.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_ops.py) `test_each_test_passes_what_it_should`, `test_checks_refuse_values_a_test_cant_use`, `test_a_pattern_reads_a_long_value_only_so_far_and_gives_up_when_slow`, `test_changed_compares_with_the_last_value_seen`, `test_the_added_tests_read_in_words` |
+| Automations fire when an entity is mentioned: any entity of a type, a name with * and ? wildcards, or an id, among up to fifty entities a processed asset is tagged with; one that names neither doesn't fire, and conditions saved before pass as they did | [test_automation_mentions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_mentions.py) `test_a_wildcard_a_type_and_an_id_find_a_mention`, `test_a_glob_is_linear_and_checks_refuse_what_it_cant_use`, `test_trees_saved_before_still_pass_as_they_did`, `test_an_automation_fires_on_any_entity_of_a_type_or_a_name_pattern_and_not_on_others`, `test_a_processed_asset_is_tagged_with_up_to_fifty_entities` |
| Conditions look back over what happened in the automation's namespaces, as its person may see it, and compare each activity with the last one; a dry run keeps nothing | [test_match_seen.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_seen.py) `test_seen_finds_what_happened_in_the_window`, `test_an_if_looks_back_and_reads_the_context`, `test_seen_is_only_for_an_automations_if`; [test_automation_conditions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_conditions.py) `test_the_if_looks_back_in_the_automations_namespaces`, `test_changed_in_a_where_compares_with_the_last_activity`, `test_the_if_and_a_condition_node_agree_and_dry_runs_keep_nothing` |
| Joins wait for an automation's triggers together: all of them, a number of times, one that doesn't come in time, or a batch, on timers that outlast a restart | [test_match_joins.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_joins.py) `test_all_waits_for_every_trigger_within_the_window_on_the_same_key`, `test_count_fires_once_per_crossing_then_counts_again`, `test_absent_waits_with_a_timer_and_the_missing_one_cancels_it`, `test_batch_collects_then_runs_once`, `test_two_writers_at_once_lose_nothing`; [test_automation_conditions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_conditions.py) `test_an_absent_join_waits_on_a_timer_and_switching_off_drops_it` |
| Each automation has limits and budgets | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`, `test_a_run_over_budget_is_held_for_the_persons_pick` |
diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py
index 9e29b310..ab945a87 100644
--- a/fastapi_backend/app/api/mcp_tools.py
+++ b/fastapi_backend/app/api/mcp_tools.py
@@ -4007,7 +4007,8 @@ def set_mail_routing(ctx: Context, account: int, rules: list[str], watch: int |
"string",
'a match tree on the activity as JSON, like {"all": [{"path": "object.name", "op": "matches", "value": "trip|outing"}]}; '
"op is exists, empty, equals, not_equals, contains, gt, lt, in, not_in, starts_with, ends_with, matches, between, "
- "before, after, within, weekday or time_between (docs/automations.md, Operators)",
+ "before, after, within, weekday, time_between or mentions (its entity tags: "
+ '{"op": "mentions", "value": {"type": "PERSON", "name": "Acme*"}}) (docs/automations.md, Operators)',
max_length=20000,
),
Arg("unreviewed", "boolean", "only those you haven't marked reviewed"),
@@ -4526,7 +4527,8 @@ def preview_template(
# ---------- automations (docs/automations.md) ----------
AUTOMATION_HOW = (
"triggers: [{as: name, on: event, kinds: [asset.added], where?: match tree}] or [{as, on: manual}] (event_kinds lists "
- "the kinds); condition (the If): a match tree over the run context, like {all: [{path: event.object.name, op: "
+ "the kinds; on an entity mentioned: kinds [asset.processed], where {op: mentions, value: {type: PERSON, name: "
+ "Acme*}}); condition (the If): a match tree over the run context, like {all: [{path: event.object.name, op: "
"contains, value: trip}]}; graph: a workflow of scope event, {nodes: [{id: i, type: input}, {id: a, type: action, "
"config: {action: log_line, params: {text: 'New: {{ event.object.name }}'}}}], edges: [{source: i, target: a}]} "
"(automation_actions lists the actions)"
diff --git a/fastapi_backend/app/domain/events.py b/fastapi_backend/app/domain/events.py
index 53a08c50..1409eb0a 100644
--- a/fastapi_backend/app/domain/events.py
+++ b/fastapi_backend/app/domain/events.py
@@ -399,7 +399,18 @@ def schema(name):
"published": {"type": "string", "format": "date-time", "title": "When"},
"summary": {"type": "string", "title": "Summary"},
"space": {"type": ["integer", "null"], "title": "Namespace"},
- "tag": {"type": "array", "title": "Tags", "items": {"type": "object"}},
+ "tag": {
+ "type": "array",
+ "title": "Tags",
+ "items": {
+ "type": "object",
+ "properties": {
+ "id": {"type": "string", "title": "Entity"},
+ "name": {"type": "string", "title": "Entity name"},
+ "entityType": {"type": "string", "title": "Entity type"},
+ },
+ },
+ },
**parts,
},
}
diff --git a/fastapi_backend/app/domain/events_content.py b/fastapi_backend/app/domain/events_content.py
index 261ae23f..8acc66e6 100644
--- a/fastapi_backend/app/domain/events_content.py
+++ b/fastapi_backend/app/domain/events_content.py
@@ -774,7 +774,12 @@ def _child(name, label, props, untrusted, sample_actor, sample_obj, summary, as_
ASSET_FIELDS = "record::id(id) AS id, title, source, media, path, content_type, duration_ms, form"
JOB_KINDS = {"succeeded": "asset.processed", "failed": "asset.failed"}
JOB_ID_RX = re.compile(r"^(processed|failed)-job-(\d+)-(\d+)$")
-TAGS = 10 # entities a processed asset is tagged with, the most mentioned first
+TAGS = events.MAX_TAGS # entities a processed asset is tagged with, the most mentioned first: as many as an activity holds
+
+
+def _mention(e, name, typ):
+ """An entity as a tag: {type: Mention, id, name, entityType} (its type's code, PERSON, for a trigger on any person)."""
+ return store.clean({"type": "Mention", "id": f"lens:entity/{e}", "name": name, "entityType": typ or None})
def entity_tags(db, rids, most=TAGS):
@@ -787,13 +792,9 @@ def entity_tags(db, rids, most=TAGS):
if _num(m.get("entity")) is not None:
counts.setdefault(m["recording"], []).append((-int(m.get("n") or 0), m["entity"]))
top = {rid: [e for _, e in sorted(c)[:most]] for rid, c in counts.items()}
- names = _lookup(db, "entity", {e for es in top.values() for e in es})
+ names = _lookup(db, "entity", {e for es in top.values() for e in es}, "name, type")
return {
- rid: [
- {"type": "Mention", "id": f"lens:entity/{e}", "name": str(names[e].get("name"))[:200]}
- for e in es
- if names.get(e, {}).get("name")
- ]
+ rid: [_mention(e, str(names[e]["name"])[:200], names[e].get("type")) for e in es if names.get(e, {}).get("name")]
for rid, es in top.items()
}
@@ -1664,6 +1665,10 @@ def name(e, named=named):
n = named.get(e)
return (n.get("a") or n.get("b")) if n else (looked.get(e) or {}).get("name")
+ def typ(e, named=named):
+ n = named.get(e)
+ return n.get("ty") if n else (looked.get(e) or {}).get("type")
+
op = _graph_op(r.get("op"), named)
alive = [e for e in ents if not (e in named and named[e].get("a") is None)]
first = (alive or ents or [None])[0]
@@ -1694,7 +1699,7 @@ def name(e, named=named):
source="graph",
pos=[r["at"], r["id"]],
published=published,
- tag=[{"type": "Mention", "id": f"lens:entity/{e}", "name": _text(name(e), 200)} for e in ents[:GRAPH_TAGS] if name(e)],
+ tag=[_mention(e, _text(name(e), 200), typ(e)) for e in ents[:GRAPH_TAGS] if name(e)],
spaces=spaces if len(spaces) > 1 else None,
audience="admins" if len(spaces) > 1 else None,
)
diff --git a/fastapi_backend/app/domain/match.py b/fastapi_backend/app/domain/match.py
index 13c9c54a..2512d497 100644
--- a/fastapi_backend/app/domain/match.py
+++ b/fastapi_backend/app/domain/match.py
@@ -7,8 +7,8 @@
itself) with one of OPS: the workflow engine's tests (flow.OPS: exists, empty, equals, not_equals, contains, gt, lt),
which work as a condition node's do, and the ones added here: `in` and `not_in` (one of a list), `starts_with`,
`ends_with` and `matches` (text), `between` (numbers or times), `before`, `after` and `within` (times, against now),
-`weekday` and `time_between` (a time's day and time of day), and `changed` (against the value this test saw last
-time). A list passes one of the added tests when one of its items does, as `contains` does; `not_in` passes when none
+`weekday` and `time_between` (a time's day and time of day), `changed` (against the value this test saw last
+time), and `mentions` (an activity's entity tags: any entity of a type, a name with `*` and `?` wildcards, or an id). A list passes one of the added tests when one of its items does, as `contains` does; `not_in` passes when none
does. `all` holds when every tree in it does (an empty one always does), `any` when one does, `not` when its tree
doesn't. Values are literals: templates belong to actions, which keeps matching cheap and safe. A tree nests at most
MAX_DEPTH deep and holds at most MAX_LEAVES tests.
@@ -52,7 +52,7 @@
LEAF = {"path", "op", "value"}
ADDED = (
"in", "not_in", "starts_with", "ends_with", "matches", "between", "before", "after", "within", "weekday", "time_between",
- "changed",
+ "changed", "mentions",
) # fmt: skip
OPS = flow.OPS + ADDED
DAYS = ("mon", "tue", "wed", "thu", "fri", "sat", "sun")
@@ -78,7 +78,9 @@
"weekday": ("falls on a", "doesn't fall on a"),
"time_between": ("is in the day between", "is not in the day between"),
"changed": ("changed", "didn't change"),
+ "mentions": ("mentions", "doesn't mention"),
}
+MENTION_KEYS = ("type", "name", "id")
_swept = {"at": 0.0}
@@ -338,6 +340,77 @@ def _changed_value(v):
return v
+def _entity_type(t):
+ """An entity type's code (PERSON) from its code or its label in any case (person, Organisation, organization)."""
+ from . import entities
+
+ s = str(t).strip().casefold() if isinstance(t, str) else ""
+ for code, label in entities.TYPES.items():
+ if s in (code.casefold(), label.casefold()):
+ return code
+ if s == "organization":
+ return "ORG"
+ raise ValueError(f"an entity's type is one of {', '.join(entities.TYPES)}")
+
+
+def _mentions_value(v):
+ if isinstance(v, str):
+ v = {"name": v}
+ if not isinstance(v, dict) or not v or set(v) - set(MENTION_KEYS):
+ raise ValueError('give the entity to look for: {"type": "PERSON"}, {"name": "Acme*"} or both (or an id)')
+ out: dict = {}
+ if v.get("type") is not None:
+ ts = v["type"] if isinstance(v["type"], list) else [v["type"]]
+ if not 1 <= len(ts) <= 20:
+ raise ValueError("give one entity type, or a list of them")
+ out["type"] = list(dict.fromkeys(_entity_type(t) for t in ts))
+ if v.get("name") is not None:
+ n = v["name"]
+ if not isinstance(n, str) or not n.strip() or len(n) > MAX_PATTERN:
+ raise ValueError(f"give the name to look for, up to {MAX_PATTERN} characters; * stands for any run of characters, ? for one")
+ out["name"] = n.strip()
+ if v.get("id") is not None:
+ i = v["id"]
+ if isinstance(i, int) and not isinstance(i, bool):
+ i = f"lens:entity/{i}"
+ if not isinstance(i, str) or not re.match(r"^lens:entity/\d{1,20}$", i):
+ raise ValueError("give the entity's id as lens:entity/9")
+ out["id"] = i
+ if not out:
+ raise ValueError('give the entity to look for: {"type": "PERSON"}, {"name": "Acme*"} or both (or an id)')
+ return out
+
+
+def glob(pattern, text):
+ """Whether `text` is `pattern` in any case, where * stands for any run of characters and ? for one (every other
+ character is itself). Linear in the two lengths, however many stars, so a crafted pattern can't tie up the worker."""
+ p, t = pattern.casefold(), text.casefold()
+ i = j = 0
+ star, mark = -1, 0
+ while j < len(t):
+ if i < len(p) and (p[i] == "?" or (p[i] != "*" and p[i] == t[j])):
+ i, j = i + 1, j + 1
+ elif i < len(p) and p[i] == "*":
+ star, mark, i = i, j, i + 1
+ elif star >= 0:
+ mark += 1
+ i, j = star + 1, mark
+ else:
+ return False
+ return all(c == "*" for c in p[i:])
+
+
+def _mentioned(want, x):
+ """Whether one tag (a Mention: {id, name, entityType}) is the entity `want` names."""
+ if not isinstance(x, dict):
+ return False
+ if "type" in want and str(x.get("entityType") or "").upper() not in want["type"]:
+ return False
+ if "name" in want and not (isinstance(x.get("name"), str) and glob(want["name"], x["name"])):
+ return False
+ return not ("id" in want and x.get("id") != want["id"])
+
+
_VALUES = {
"in": _list_value,
"not_in": _list_value,
@@ -351,6 +424,7 @@ def _changed_value(v):
"weekday": _weekday_value,
"time_between": _time_between_value,
"changed": _changed_value,
+ "mentions": _mentions_value,
}
@@ -384,6 +458,10 @@ def test_leaf(m, value, env=None):
return st.changed(key, v)
if op in ("in", "not_in"):
return _some(v, lambda x: _one_of(x, want)) == (op == "in")
+ if op == "mentions":
+ if isinstance(v, dict) and isinstance(v.get("tag"), list): # an activity: its tags
+ v = v["tag"]
+ return _some(v, lambda x: _mentioned(want, x))
if v is None:
return False
if op in ("starts_with", "ends_with"):
@@ -1333,9 +1411,25 @@ def _value_words(op, v, kind):
return f"{v[0]} and {v[1]}"
if op == "changed":
return f"for each {_label(v, kind)}" if v else ""
+ if op == "mentions":
+ return _mention_words(v)
return _value(v)
+def _mention_words(v):
+ from . import entities
+
+ what = _either([entities.TYPES[t].lower() for t in v["type"]]) if v.get("type") else "an entity"
+ if v.get("type"):
+ what = f"{'an' if what[0] in 'aeiou' else 'a'} {what}"
+ if v.get("name"):
+ said = "named like" if any(c in v["name"] for c in "*?") else "named"
+ what += f" {said} “{v['name']}”"
+ if v.get("id"):
+ what += f" ({v['id']})"
+ return what
+
+
def describe(tree, kind=None):
"""The tree in plain words, as the builder and the agent show it: "the sender contains “@stmarys.org” and the
subject matches “trip” or “outing”". `kind` is the kind of activity the tree tests (events.py), for naming its
@@ -1363,7 +1457,8 @@ def _describe(m, kind, top):
def _leaf(m, kind, negated):
said = WORDS.get(m["op"], (m["op"], f"not {m['op']}"))[1 if negated else 0]
- words = f"{_label(m.get('path'), kind)} {said}"
+ who = "it" if m["op"] == "mentions" and m.get("path") in (None, "tag") else _label(m.get("path"), kind)
+ words = f"{who} {said}"
if m["op"] in ("exists", "empty") or (m["op"] == "changed" and not m.get("value")):
return words
return f"{words} {_value_words(m['op'], m.get('value'), kind)}"
diff --git a/fastapi_backend/tests/domain/test_automation_mentions.py b/fastapi_backend/tests/domain/test_automation_mentions.py
new file mode 100644
index 00000000..279142d6
--- /dev/null
+++ b/fastapi_backend/tests/domain/test_automation_mentions.py
@@ -0,0 +1,118 @@
+"""Automations that fire when an entity is mentioned (docs/automations.md, Operators: mentions): a processed asset is
+tagged with up to fifty of its entities, each with its type, and a trigger's where finds any entity of a type, a name
+with * and ? wildcards, or an id; one that names neither doesn't fire, and trees saved before work as they did."""
+
+from __future__ import annotations
+
+import time
+
+import pytest
+
+from app.domain import automations, events, events_content, ingest, jobs, match, store
+from tests.helpers import PODS1, drain, make_user
+
+PODS = "\n".join(f"[00:{i:02d}] {line.split('|')[0]}: {line.split('|')[2]}" for i, line in enumerate(PODS1.splitlines()))
+TAGS = [
+ {"type": "Mention", "id": "lens:entity/9", "name": "Acme Robotics", "entityType": "ORG"},
+ {"type": "Mention", "id": "lens:entity/12", "name": "Sam Lee", "entityType": "PERSON"},
+]
+SAY = {
+ "nodes": [
+ {"id": "i", "type": "input", "config": {}},
+ {"id": "a", "type": "action", "config": {"action": "log_line", "params": {"text": "Mentioned in {{ event.object.name }}"}}},
+ ],
+ "edges": [{"source": "i", "target": "a"}],
+}
+
+
+def fires(value, activity=None):
+ tree = match.check({"op": "mentions", "value": value})
+ return match.test(tree, activity or {"tag": TAGS})
+
+
+def test_a_wildcard_a_type_and_an_id_find_a_mention():
+ assert fires({"name": "Acme*"}) and fires("acme*") and fires({"name": "*lee"}) and fires({"name": "Sam L?e"})
+ assert fires({"type": "person"}) and fires({"type": "Organisation"}) and fires({"type": ["PLACE", "ORG"]})
+ assert fires({"type": "ORG", "name": "Acme*"}) and fires({"id": "lens:entity/12"}) and fires({"id": 9})
+ assert fires({"name": "acme robotics"}) # no wildcard: the whole name, in any case
+ # what doesn't fit doesn't fire: another name, a part of a name without a star, a type on another entity
+ assert not fires({"name": "Globex*"}) and not fires({"name": "Acme"}) and not fires({"type": "PLACE"})
+ assert not fires({"type": "PERSON", "name": "Acme*"}) and not fires({"id": "lens:entity/4"})
+ assert not fires({"type": "PERSON"}, {"tag": []}) and not fires({"name": "*"}, {"kind": "asset.processed"})
+ # with a path it reads the tags there (the If's event), and under not it passes when none fits
+ tree = match.check({"not": {"path": "event.tag", "op": "mentions", "value": {"name": "Globex*"}}})
+ assert match.test(tree, {"event": {"tag": TAGS}})
+
+
+def test_a_glob_is_linear_and_checks_refuse_what_it_cant_use():
+ assert match.glob("*a*a*a*a*a*a*a*a*a*b", "a" * 5000) is False # many stars, no blow-up
+ assert match.glob("[x]*", "[x] y") and not match.glob("[x]*", "x y") # only * and ? are wildcards
+ for bad in ({}, {"name": ""}, {"type": "Planet"}, {"id": "entity 9"}, {"name": "x" * 201}, {"who": "Sam"}, 4):
+ with pytest.raises(ValueError):
+ match.check({"op": "mentions", "value": bad})
+ assert match.check({"op": "mentions", "value": {"type": "organization"}})["value"] == {"type": ["ORG"]}
+ assert match.describe(match.check({"op": "mentions", "value": {"type": "person", "name": "Sam*"}})) == (
+ "it mentions a person named like “Sam*”"
+ )
+
+
+def test_trees_saved_before_still_pass_as_they_did():
+ old = {
+ "all": [
+ {"path": "tag.0.name", "op": "equals", "value": "Acme Robotics"},
+ {"path": "object.name", "op": "contains", "value": "call"},
+ ]
+ }
+ tree = match.check(old)
+ assert tree == old and match.test(tree, {"tag": TAGS, "object": {"name": "Sales call"}})
+ assert not match.test(tree, {"tag": TAGS[1:], "object": {"name": "Sales call"}})
+
+
+@pytest.fixture
+def root(db):
+ events._seen.clear()
+ events._claimed.clear()
+ events_content.forget_names()
+ automations._swept["at"] = 0.0
+ make_user(db, "root@x.io", "root password 1", admin=True)
+ from app.domain import auth
+
+ yield {**auth.find_account(db, "root@x.io"), "admin": True}
+ events_content.forget_names()
+
+
+def test_an_automation_fires_on_any_entity_of_a_type_or_a_name_pattern_and_not_on_others(db, cfg, root):
+ def on(name, value):
+ trigger = {"as": "said", "on": "event", "kinds": ["asset.processed"], "where": {"op": "mentions", "value": value}}
+ return automations.create(db, cfg, {"name": name, "triggers": [trigger], "graph": SAY, "enabled": True}, root)["id"]
+
+ by_name, by_type, other = on("Dyno", {"name": "dyno*"}), on("Any org", {"type": "ORG"}), on("Globex", {"name": "Globex*"})
+ assert automations.dispatch(db, cfg) == 0 # the first look marks where the follower starts
+ start = store.now()
+ while store.now() == start:
+ time.sleep(0.05)
+ rid = ingest.import_text(db, cfg, "pods", PODS, title="Capsid episode")
+ jobs.enqueue(db, rid, ["analyze"], by="root@x.io")
+ drain(db, cfg)
+ [a] = [x for x in events.page(db, cfg, events.viewer(1, None, admin=True), flt={"kinds": ["asset.processed"]})["items"]]
+ dyno = next(t for t in a["tag"] if t["name"] == "Dyno Therapeutics")
+ assert dyno["entityType"] == "ORG" and dyno["id"].startswith("lens:entity/")
+ automations.dispatch(db, cfg)
+ ran = {r["automation"] for r in db.rows("SELECT automation, event FROM automation_run")}
+ assert ran == {by_name, by_type} and other not in ran
+ [r] = automations.runs(db, by_name, limit=1)
+ assert automations.get_run(db, r["id"])["intents"][0]["result"]["line"] == "Mentioned in Capsid episode"
+
+
+def test_a_processed_asset_is_tagged_with_up_to_fifty_entities(db, cfg, monkeypatch):
+ rid = ingest.import_text(db, cfg, "pods", "Sam: hello.", title="Many names")
+ counts = []
+ for n in range(60):
+ e = db.next_id("entity")
+ db.q("CREATE $r CONTENT $d", r=store.R("entity", e), d={"name": f"Person {n}", "type": "PERSON", "key": f"person {n}"})
+ counts.append({"recording": rid, "entity": e, "n": 60 - n}) # the first is the most mentioned
+ rows = db.rows
+ monkeypatch.setattr(db, "rows", lambda sql, **v: counts if "FROM mentions" in sql else rows(sql, **v))
+ tags = events_content.entity_tags(db, [rid])[rid]
+ assert len(tags) == events.MAX_TAGS == 50 and tags[0]["name"] == "Person 0" and tags[-1]["name"] == "Person 49"
+ assert all(t["entityType"] == "PERSON" for t in tags)