Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,9 +226,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
Automations act
on that stream, on a schedule, at a time or when their own webhook is called, alone or with their triggers joined (all
of them, a number of times, one that doesn't come in time, or a batch): they test conditions (text, patterns,
numbers, lists, times, days, whether a value changed and whether something like it happened before in their
namespaces) and let a decision model pick the branch, then tell you in
your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a
numbers, lists, times, days, whether a value changed, the entities an asset mentions (any person, or names like
Acme*) and whether something like it happened before in their namespaces) and let a decision model pick the branch,
then tell you in your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a
workflow, pipeline or routine, or change assets and notes. The Schedules page is Lens's crontab: every routine,
automation schedule and the backup in one list with when each runs next and last, to switch, re-time or run now, and
agents do the same over the API and MCP. A Classify node in a workflow sorts an asset into labels, content types or
Expand Down
6 changes: 6 additions & 0 deletions changelog.d/automation-mentions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- **Automations fire on any entity of a type, or names matching a pattern.** A trigger's where (and an If, and a
condition node's tree) can test what an activity mentions: `{"op": "mentions", "value": {"type": "PERSON"}}` fires
on any person, `{"name": "Acme*"}` on names like Acme, with `*` and `?` as wildcards, and `{"id": "lens:entity/9"}`
on one entity. A processed asset is now tagged with its fifty most mentioned entities, not ten, and every entity tag
carries its `entityType`. Conditions saved before work as they did
([Operators](docs/automations.md#operators)).
25 changes: 22 additions & 3 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,8 @@ that triggers subscribe to. `events.check_activity(a)` checks it, naming the fir
- `object`: what it is about, with the kind's own fields. Ids are `lens:<table>/<id>` references, so actions can act
on them; `events.ref_id(ref, "recording")` reads the id back.
- `target`: where to, when the verb has one (Add to a collection), else null. `tag`: up to 50 objects such as
`{"type": "Mention", "id": "lens:entity/9", "name": "St Mary's"}`.
`{"type": "Mention", "id": "lens:entity/9", "name": "St Mary's", "entityType": "ORG"}`: an entity's type is its
code, so a trigger can fire on any person mentioned ([`mentions`](#operators)).
- `summary`: one line of at most 300 characters, made by fixed rules, never by a model.
- `space`: the namespace id, or null for activities of no namespace.
- `lens`: Lens's own fields. `source` and `pos` (the source's position, `[time, id]`) are for the stream machinery.
Expand Down Expand Up @@ -228,8 +229,8 @@ imported), else `asset.added`. A file an email or archive held, made a resource
email is what arrived. A trigger on `asset.added` matches every one of them.

A run of an asset's pipeline that finishes is `asset.processed` (it succeeded) or `asset.failed`, one activity per
run, batch runs included. Its actor is whoever queued it, and its tags are the asset's entities (the ten most
mentioned), which exist only once it has been processed: `asset.processed` is the kind for "when an email about St
run, batch runs included. Its actor is whoever queued it, and its tags are the asset's entities (the fifty most
mentioned, as many as an activity holds, each with its `entityType`), which exist only once it has been processed: `asset.processed` is the kind for "when an email about St
Mary's is understood". A run tried again finishes again, as another activity. A deletion is logged as
`asset.deleted` where the asset is deleted (once per asset), and a move to another namespace as `asset.moved`, a
`Move` by whoever moved it, seen in the namespace it is in now (a trigger there on `asset.moved` is "when something
Expand Down Expand Up @@ -459,6 +460,7 @@ its items does (as `contains` does), and `not_in` passes when none does. A value
| `weekday` | days: `["sat", "sun"]`, `["monday"]` or ISO numbers (1 is Monday) | is a time on one of those days |
| `time_between` | two times of day: `["22:00", "06:00"]` | is a time of day from the first up to the second, across midnight when the second is earlier |
| `changed` | none, or a path to keep values apart by: `actor.id` | differs from the value this test saw last time (the first one seen is no change) |
| `mentions` | the entity: `{"type": "PERSON"}`, `{"name": "Acme*"}`, both, or `{"id": "lens:entity/9"}` | is an activity (its `tag`), or a list of tags, naming an entity of that type (any of a list), with a name like that, or with that id |

- **Times** are ISO 8601 times or dates. A time written without a zone (in the value or in what is tested) is in the
automation's time zone, and so are `weekday` and `time_between`. Durations are a whole number and `s`, `m`, `h` or
Expand All @@ -468,6 +470,20 @@ its items does (as `contains` does), and `not_in` passes when none does. A value
may start alike (`(a|aa)+`) or refers back to a group (`\1`) is refused when it is saved, a pattern reads at most
the first 20,000 characters of a value, and a match that takes over 50 ms counts as no match. Compiled patterns are
cached.
- **Mentions** are the entities an activity is tagged with (`asset.processed`, `entity.changed`). A type is its code
or its label in any case (`PERSON`, `person`, `Organisation`; `ORG`, `PRODUCT`, `PLACE`, `EVENT`, `WORK`, `TERM`,
`DATE`, `NUMBER`). In a name, `*` stands for any run of characters and `?` for one, in any case, and every other
character is itself; a name with neither is the whole name (`Acme` doesn't find “Acme Robotics”, `Acme*` does). A
value that is only text is a name. With no path the test reads the activity's `tag`; in the If give the path
(`event.tag`). The name is matched in time linear in its length, however many stars it has, so a crafted pattern
can't tie up the worker. When St Mary's or any person is mentioned in a processed asset:

```json
{"as": "said", "on": "event", "kinds": ["asset.processed"],
"where": {"any": [{"op": "mentions", "value": {"name": "St Mary*"}},
{"op": "mentions", "value": {"type": "PERSON"}}]}}
```

- **`env`**: `match.test(tree, value, env)` reads time and state from `env`, never from globals, so tests and dry
runs can fix them: `now` (the clock when not given), `timezone` (UTC when not given) and `state`.
- **`changed`** compares with the last value the same test saw for the same automation and trigger, kept in
Expand Down Expand Up @@ -1468,6 +1484,9 @@ built yet is in [Refine later](#refine-later); the [Timeline](timeline.md) has i

- Variables inside match values (`{"var": "vars.threshold"}`).
- Statistical conditions, such as a reading above its 7-day average.
- `mentions` on an asset's top fifty entities: an asset that mentions more names only its most mentioned; a picker for
the entity and its type in the canvas's condition node (today it is written in a where or over the API and MCP);
an `entity.mentioned` kind, one per entity, if one per processed asset proves too coarse.
- Sequences longer than two steps ("A, then B, then C").
- The canvas shows and edits match trees on condition, filter and switch nodes (today it edits one test; a tree
saved through the API is kept).
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,7 @@ request, to another check that proves the same thing; loosening a requirement or
| They test conditions, then tell you, ask the model or the agent, use a tool, call a web address with sealed credentials, run Lens work, or change assets and notes | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`; [test_actions_messages.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_messages.py) `test_notify_tells_me_in_my_chat_app`; [test_actions_ai.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_ai.py) `test_ask_model_answers_in_text_or_as_the_fields_a_schema_names`, `test_ask_agent_works_on_what_happened_and_saves_the_conversation`, `test_call_tool_reads_as_the_person_and_changes_only_by_asking_them`; [test_actions_web.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_web.py) `test_credentials_in_headers_are_sealed_to_the_automation_and_the_host_and_never_shown`; [test_actions_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_run.py) `test_run_workflow_queues_it_on_the_asset_as_the_automation`, `test_run_pipeline_processes_the_asset_again`; [test_actions_lens.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_lens.py) `test_set_field_is_kept_in_the_assets_history_by_the_automation`, `test_write_note_writes_a_page_adds_to_a_things_page_or_notes_an_asset` |
| What reaches outside Lens, or changes more than what set it off, asks first: on the Approvals page or with a yes in chat; agents decline but never approve | [test_automation_spec.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_spec.py) `test_risk_follows_the_effect`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_approve_carries_the_action_out_and_the_run_records_it`, `test_agents_over_mcp_list_and_decline_but_never_approve`; [test_approvals_chat.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals_chat.py) `test_a_yes_in_my_chat_decides_an_action_inside_lens`; [approvals-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/approvals-page.test.tsx) `groups what waits, puts the first in focus, and Enter approves it` |
| Conditions test text, patterns, numbers, lists, times, days and whether a value changed, checked as they're saved and read in words | [test_match_ops.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_ops.py) `test_each_test_passes_what_it_should`, `test_checks_refuse_values_a_test_cant_use`, `test_a_pattern_reads_a_long_value_only_so_far_and_gives_up_when_slow`, `test_changed_compares_with_the_last_value_seen`, `test_the_added_tests_read_in_words` |
| Automations fire when an entity is mentioned: any entity of a type, a name with * and ? wildcards, or an id, among up to fifty entities a processed asset is tagged with; one that names neither doesn't fire, and conditions saved before pass as they did | [test_automation_mentions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_mentions.py) `test_a_wildcard_a_type_and_an_id_find_a_mention`, `test_a_glob_is_linear_and_checks_refuse_what_it_cant_use`, `test_trees_saved_before_still_pass_as_they_did`, `test_an_automation_fires_on_any_entity_of_a_type_or_a_name_pattern_and_not_on_others`, `test_a_processed_asset_is_tagged_with_up_to_fifty_entities` |
| Conditions look back over what happened in the automation's namespaces, as its person may see it, and compare each activity with the last one; a dry run keeps nothing | [test_match_seen.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_seen.py) `test_seen_finds_what_happened_in_the_window`, `test_an_if_looks_back_and_reads_the_context`, `test_seen_is_only_for_an_automations_if`; [test_automation_conditions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_conditions.py) `test_the_if_looks_back_in_the_automations_namespaces`, `test_changed_in_a_where_compares_with_the_last_activity`, `test_the_if_and_a_condition_node_agree_and_dry_runs_keep_nothing` |
| Joins wait for an automation's triggers together: all of them, a number of times, one that doesn't come in time, or a batch, on timers that outlast a restart | [test_match_joins.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_joins.py) `test_all_waits_for_every_trigger_within_the_window_on_the_same_key`, `test_count_fires_once_per_crossing_then_counts_again`, `test_absent_waits_with_a_timer_and_the_missing_one_cancels_it`, `test_batch_collects_then_runs_once`, `test_two_writers_at_once_lose_nothing`; [test_automation_conditions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_conditions.py) `test_an_absent_join_waits_on_a_timer_and_switching_off_drops_it` |
| Each automation has limits and budgets | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`, `test_a_run_over_budget_is_held_for_the_persons_pick` |
Expand Down
6 changes: 4 additions & 2 deletions fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -4007,7 +4007,8 @@ def set_mail_routing(ctx: Context, account: int, rules: list[str], watch: int |
"string",
'a match tree on the activity as JSON, like {"all": [{"path": "object.name", "op": "matches", "value": "trip|outing"}]}; '
"op is exists, empty, equals, not_equals, contains, gt, lt, in, not_in, starts_with, ends_with, matches, between, "
"before, after, within, weekday or time_between (docs/automations.md, Operators)",
"before, after, within, weekday, time_between or mentions (its entity tags: "
'{"op": "mentions", "value": {"type": "PERSON", "name": "Acme*"}}) (docs/automations.md, Operators)',
max_length=20000,
),
Arg("unreviewed", "boolean", "only those you haven't marked reviewed"),
Expand Down Expand Up @@ -4526,7 +4527,8 @@ def preview_template(
# ---------- automations (docs/automations.md) ----------
AUTOMATION_HOW = (
"triggers: [{as: name, on: event, kinds: [asset.added], where?: match tree}] or [{as, on: manual}] (event_kinds lists "
"the kinds); condition (the If): a match tree over the run context, like {all: [{path: event.object.name, op: "
"the kinds; on an entity mentioned: kinds [asset.processed], where {op: mentions, value: {type: PERSON, name: "
"Acme*}}); condition (the If): a match tree over the run context, like {all: [{path: event.object.name, op: "
"contains, value: trip}]}; graph: a workflow of scope event, {nodes: [{id: i, type: input}, {id: a, type: action, "
"config: {action: log_line, params: {text: 'New: {{ event.object.name }}'}}}], edges: [{source: i, target: a}]} "
"(automation_actions lists the actions)"
Expand Down
13 changes: 12 additions & 1 deletion fastapi_backend/app/domain/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,18 @@ def schema(name):
"published": {"type": "string", "format": "date-time", "title": "When"},
"summary": {"type": "string", "title": "Summary"},
"space": {"type": ["integer", "null"], "title": "Namespace"},
"tag": {"type": "array", "title": "Tags", "items": {"type": "object"}},
"tag": {
"type": "array",
"title": "Tags",
"items": {
"type": "object",
"properties": {
"id": {"type": "string", "title": "Entity"},
"name": {"type": "string", "title": "Entity name"},
"entityType": {"type": "string", "title": "Entity type"},
},
},
},
**parts,
},
}
Expand Down
21 changes: 13 additions & 8 deletions fastapi_backend/app/domain/events_content.py
Original file line number Diff line number Diff line change
Expand Up @@ -774,7 +774,12 @@ def _child(name, label, props, untrusted, sample_actor, sample_obj, summary, as_
ASSET_FIELDS = "record::id(id) AS id, title, source, media, path, content_type, duration_ms, form"
JOB_KINDS = {"succeeded": "asset.processed", "failed": "asset.failed"}
JOB_ID_RX = re.compile(r"^(processed|failed)-job-(\d+)-(\d+)$")
TAGS = 10 # entities a processed asset is tagged with, the most mentioned first
TAGS = events.MAX_TAGS # entities a processed asset is tagged with, the most mentioned first: as many as an activity holds


def _mention(e, name, typ):
"""An entity as a tag: {type: Mention, id, name, entityType} (its type's code, PERSON, for a trigger on any person)."""
return store.clean({"type": "Mention", "id": f"lens:entity/{e}", "name": name, "entityType": typ or None})


def entity_tags(db, rids, most=TAGS):
Expand All @@ -787,13 +792,9 @@ def entity_tags(db, rids, most=TAGS):
if _num(m.get("entity")) is not None:
counts.setdefault(m["recording"], []).append((-int(m.get("n") or 0), m["entity"]))
top = {rid: [e for _, e in sorted(c)[:most]] for rid, c in counts.items()}
names = _lookup(db, "entity", {e for es in top.values() for e in es})
names = _lookup(db, "entity", {e for es in top.values() for e in es}, "name, type")
return {
rid: [
{"type": "Mention", "id": f"lens:entity/{e}", "name": str(names[e].get("name"))[:200]}
for e in es
if names.get(e, {}).get("name")
]
rid: [_mention(e, str(names[e]["name"])[:200], names[e].get("type")) for e in es if names.get(e, {}).get("name")]
for rid, es in top.items()
}

Expand Down Expand Up @@ -1664,6 +1665,10 @@ def name(e, named=named):
n = named.get(e)
return (n.get("a") or n.get("b")) if n else (looked.get(e) or {}).get("name")

def typ(e, named=named):
n = named.get(e)
return n.get("ty") if n else (looked.get(e) or {}).get("type")

op = _graph_op(r.get("op"), named)
alive = [e for e in ents if not (e in named and named[e].get("a") is None)]
first = (alive or ents or [None])[0]
Expand Down Expand Up @@ -1694,7 +1699,7 @@ def name(e, named=named):
source="graph",
pos=[r["at"], r["id"]],
published=published,
tag=[{"type": "Mention", "id": f"lens:entity/{e}", "name": _text(name(e), 200)} for e in ents[:GRAPH_TAGS] if name(e)],
tag=[_mention(e, _text(name(e), 200), typ(e)) for e in ents[:GRAPH_TAGS] if name(e)],
spaces=spaces if len(spaces) > 1 else None,
audience="admins" if len(spaces) > 1 else None,
)
Expand Down
Loading
Loading