diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a402b114..61c70794c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,10 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version only in the automation's namespaces and only as its person may see them. `changed` in a trigger's where or an If compares each activity with the last one, and trying an automation keeps nothing ([Looking back](docs/automations.md#looking-back)). +- **Automations wait for their triggers together.** An automation's join runs it when all of its triggers came within + a time (for the same sender, say), when one came a number of times, when one didn't come in time after another (no + reply within two days), or once for a batch collected over a time. What it waits on outlasts a restart, and switching + the automation off drops it ([Joins](docs/automations.md#joins)). - **Specs from transcripts.** Turn a voice session or a meeting into a technical spec, saved as a note linked to it: the model writes the spec, then lists every named decision, every "no" and "not that, this", every constraint and every gap the speaker left to decide later, each with its quote and a link to the moment it was said; a quote the diff --git a/README.md b/README.md index 8bbcc7c9f..31d94ee7b 100644 --- a/README.md +++ b/README.md @@ -147,7 +147,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p calendar events, files, feed posts, what people say to Lens, sensor readings, notes, graph changes, runs, budget warnings) is kept as a stream of activities in W3C Activity Streams form, which agents read through the API, MCP and the assistant, and people review on the Timeline page (and Latest on Home), marking what they've seen. Automations act - on that stream, on a schedule, at a time or when their own webhook is called: they test conditions (text, patterns, + on that stream, on a schedule, at a time or when their own webhook is called, alone or with their triggers joined (all + of them, a number of times, one that doesn't come in time, or a batch): they test conditions (text, patterns, numbers, lists, times, days, whether a value changed and whether something like it happened before in their namespaces), then tell you in your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a @@ -230,7 +231,7 @@ These are open pull requests, not yet part of Lens: Next, and not in a pull request yet; each links to where it is tracked: -- **The automations studio:** drawing automations in the web app, joins, and the decide node, where a decision model +- **The automations studio:** drawing automations in the web app, and the decide node, where a decision model picks the branch ([automations](docs/automations.md)). - **Timeline:** what each item caused, and Approve and Decline on waiting approvals ([timeline](docs/timeline.md)). - **Notes:** nesting a note under its project or area without a click, pages in the graph, and wiki addresses for diff --git a/docs/automations.md b/docs/automations.md index 020868621..83c8cc69f 100644 --- a/docs/automations.md +++ b/docs/automations.md @@ -12,6 +12,7 @@ merged is called built. | Match trees: one condition language for triggers, an automation's If and workflow nodes | built | | Operators for times, lists, text patterns and changes (`in`, `matches`, `within`, `changed`...) | built | | Looking back over the stream in an automation's If (`seen`) | built | +| Joins: `all`, `count`, `absent` and `batch`, completed as the stream brings each trigger's activities and as their timers fire ([Joins](#joins)) | built | | Activities, the kinds catalog, the event log, following the stream, timeline reads (`app/domain/events.py`) | built | | New assets as `asset.added` (the `recordings` source) | built | | New assets by kind: emails received, calendar events, files from storage, uploads, pages saved from the web, imports | built | @@ -49,7 +50,7 @@ merged is called built. | Running automations: following the stream, limits, budgets, runs as `automation.ran`, trying and running now, the API (`app/domain/automations.py`) | built | | Automations for agents: the MCP tools and the assistant's, switching on and running now by approval | built | | Describing an automation in words: a draft from a sentence (`POST /api/v1/automations/draft`) | built | -| Joins, the studio | planned | +| The studio | planned | ## Contract @@ -415,7 +416,8 @@ tree, `path` (when given) is the part of the value the tree tests. Graphs saved ### Automations: triggers and joins Built as shapes with checks (`app/domain/automation_spec.py`), saved ([Saving automations](#saving-automations)) and -run ([Running automations](#running-automations)) for triggers on `event` and `manual` and the join `any`. Each check gives a cleaned copy, or a ValueError naming the first problem. +run ([Running automations](#running-automations)), joins included. Each check gives a cleaned copy, or a ValueError +naming the first problem. ```json {"name": "School emails", "enabled": true, "spaces": [3], "run_as": 1, @@ -441,6 +443,35 @@ run ([Running automations](#running-automations)) for triggers on `event` and `m - `check_policy(p, nodes, allow_auto=False)`: `{action node id: ask or auto}`. Only a person in a signed-in session lets a node run on its own; everywhere else (agents, drafts, the workflow API) `auto` is refused. +#### Joins + +Built (`match.join_step`). Running automations hands each activity a trigger matched (after its guards and its +claim) to `match.join_step(db, cfg, automation, alias, activity)`, which gives a list of completions, each +`{"as": alias, "events": {alias: activity or [activities]}, "event": the last}`, and each completion fires the +automation: its If and graph see every trigger's activity in `events` (`{{ events.mail.object.subject }}`), and +`event` is the last. Triggers a join doesn't name fire on their own, as with `any`. Windows are measured between the +activities' `published` times. + +| Mode | Example | Completes | +| --- | --- | --- | +| `any` | `{"mode": "any"}` | at once, for each activity | +| `all` | `{"mode": "all", "within": "1d", "key": "object.from.0"}`: an email from the school and a calendar event from the same sender within a day | when every trigger has an activity within `within` of the others (on the same `key`); each trigger's latest one goes in `events`, then it starts again | +| `count` | `{"mode": "count", "of": "login", "at_least": 3, "within": "5m", "key": "actor.id"}`: three failed logins from one device in five minutes | when `of` has `at_least` activities within `within` (per `key`), with the list of them; then it counts again | +| `absent` | `{"mode": "absent", "after": "mail", "missing": "reply", "within": "2d", "key": "object.thread"}`: no reply within two days | when `within` passes after `after` with no `missing` on the same key published after it, with the first `after` activity; a `missing` one in time cancels it | +| `batch` | `{"mode": "batch", "of": "alert", "within": "1h"}`: the hour's alerts, acted on once | `within` after the first activity, with every one collected (up to 500), by trigger | + +- What a join waits on is kept in `automation_state` (kind `join`) with small copies of the activities (long text + cut), written so two processes at once lose nothing (a version on each row). A row expires `within` after its + newest activity, or a day after an `absent` or `batch` timer's time. +- `absent` and `batch` set [timers](#timers) for their automation (`absent--`, + `batch-`), so they outlast a restart and fire once; their handlers, `join.absent` and `join.batch` + (`match.TIMER_HANDLERS`), fire the automation. A `missing` activity in time calls its timer off. A timer that ends + without firing (the automation is switched off, or Lens was off past `automations.catch_up`) drops what it waited + on, so the next activity starts afresh; one that can't be set (the automation has 1000 timers waiting) drops it + too, with a line in the log. +- `match.describe_join(join)` reads a join in words: `all of the triggers within 10 minutes, for the same + object.from`; `3 times login within 5 minutes`; `mail, then no reply within 2 days`. + ### Webhook trigger Built (`app/domain/automation_hooks.py`, `POST /api/v1/automations/hooks/{token}`). A trigger `{"on": "webhook"}` @@ -1024,9 +1055,9 @@ person who made it (`run_as`), with their permissions, in namespaces they can ed value, and saving that again keeps it. - **Checks** when it is saved: the triggers, the join and the If as in [the contract](#automations-triggers-and-joins), the graph, the namespaces, `limits` (`cooldown`, like `10m`, default `0s`; `max_per_hour`, 1 to 3600, default 60) - and the policy. Joins other than `any` are refused as coming soon until joins are built. Saving sets the timers of - its `schedule` and `at` triggers ([Schedule, at and relative triggers](#schedule-at-and-relative-triggers)), and - switching it off or deleting it calls them off. + and the policy. Saving sets the timers of its `schedule` and `at` triggers ([Schedule, at and relative + triggers](#schedule-at-and-relative-triggers)), and switching it off or deleting it calls them off, with any its + joins set. - **Webhook triggers** get their address when the automation is saved ([Webhook trigger](#webhook-trigger)): the answer's `hooks` gives each new one's `url` and signing `secret`, shown this once, and the trigger keeps the hook's id. Saving it again keeps its address; a webhook trigger dropped (or changed to another kind) loses it at once. @@ -1060,8 +1091,8 @@ is `{draft, dropped, unmapped}`: - `dropped` says what in the model's draft didn't fit and why: a trigger, its `where`, the If, an action that doesn't exist or params that don't fit its schema. With no trigger left it runs when you run it; with no action left it has no Then yet. -- `unmapped` lists what the sentence asked for that it couldn't express, such as "when both arrive" (joins are - planned). +- `unmapped` lists what the sentence asked for that it couldn't express, such as "when both arrive" (a draft + doesn't make joins yet; add one before saving). Nothing is saved: try the draft (`POST /automations/test`), change it, then save it. The call counts in the [activity ledger](activity.md) for you and its namespaces, and isn't made once a budget they count for, or everything's, is @@ -1099,8 +1130,9 @@ Each new activity is checked against each trigger that listens for its kind (a w Then it is claimed (for the automation, trigger and activity together), so it runs each automation once, even with the API and a worker both following. **`automations.fire(db, cfg, automation_id, alias, activity, events=None, dry=False)`** -is the one way every trigger starts a run (the stream with its webhook calls, run now, and later the clock, timers and -joins); it gives the run's id, or none when nothing ran. A run goes: +is the one way every trigger starts a run (the stream with its webhook calls, run now, the clock, timers and joins); +with a join other than `any`, the claimed activity goes to the join first ([Joins](#joins)) and the automation fires +when it completes. It gives the run's id, or none when nothing ran. A run goes: 1. **Its person**: if the person it acts for lost their account, their admin role (for one that works everywhere) or the right to edit one of its namespaces, the run fails saying so, and the automation is switched off. @@ -1143,8 +1175,13 @@ held run until you pick. ## Refine later +- A draft from a sentence that makes joins ("when both arrive", "if no reply within 2 days"). +- Switching an automation off and on keeps what an `all` or `count` join was waiting on, until its window passes; a + changed join keeps the old one's rows until they expire. + - Variables inside match values (`{"var": "vars.threshold"}`), once saved variables are built. - Statistical conditions, such as a reading above its 7-day average. +- Sequences longer than two steps ("A, then B, then C"). - The canvas shows and edits match trees on condition, filter and switch nodes (today it edits one test; a tree saved through the API is kept). - A `?format=as2` JSON-LD view and per-actor outboxes for ActivityPub; SurrealDB `DEFINE EVENT` for Update verbs that diff --git a/docs/requirements.md b/docs/requirements.md index b737234a2..8c56be837 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -186,6 +186,7 @@ request, to another check that proves the same thing; loosening a requirement or | What reaches outside Lens, or changes more than what set it off, asks first: on the Approvals page or with a yes in chat; agents decline but never approve | [test_automation_spec.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_spec.py) `test_risk_follows_the_effect`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_approve_carries_the_action_out_and_the_run_records_it`, `test_agents_over_mcp_list_and_decline_but_never_approve`; [test_approvals_chat.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals_chat.py) `test_a_yes_in_my_chat_decides_an_action_inside_lens`; [approvals-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/approvals-page.test.tsx) `groups what waits, puts the first in focus, and Enter approves it` | | Conditions test text, patterns, numbers, lists, times, days and whether a value changed, checked as they're saved and read in words | [test_match_ops.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_ops.py) `test_each_test_passes_what_it_should`, `test_checks_refuse_values_a_test_cant_use`, `test_a_pattern_reads_a_long_value_only_so_far_and_gives_up_when_slow`, `test_changed_compares_with_the_last_value_seen`, `test_the_added_tests_read_in_words` | | Conditions look back over what happened in the automation's namespaces, as its person may see it, and compare each activity with the last one; a dry run keeps nothing | [test_match_seen.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_seen.py) `test_seen_finds_what_happened_in_the_window`, `test_an_if_looks_back_and_reads_the_context`, `test_seen_is_only_for_an_automations_if`; [test_automation_conditions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_conditions.py) `test_the_if_looks_back_in_the_automations_namespaces`, `test_changed_in_a_where_compares_with_the_last_activity`, `test_the_if_and_a_condition_node_agree_and_dry_runs_keep_nothing` | +| Joins wait for an automation's triggers together: all of them, a number of times, one that doesn't come in time, or a batch, on timers that outlast a restart | [test_match_joins.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_match_joins.py) `test_all_waits_for_every_trigger_within_the_window_on_the_same_key`, `test_count_fires_once_per_crossing_then_counts_again`, `test_absent_waits_with_a_timer_and_the_missing_one_cancels_it`, `test_batch_collects_then_runs_once`, `test_two_writers_at_once_lose_nothing`; [test_automation_conditions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_conditions.py) `test_an_absent_join_waits_on_a_timer_and_switching_off_drops_it` | | Each automation has limits and budgets | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`, `test_a_run_over_budget_is_held_for_the_persons_pick` | | Agents make, try and run automations over MCP and in the assistant, or draft one from a sentence | [test_automation_tools.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_automation_tools.py) `test_agents_save_try_and_run_automations_over_mcp`, `test_the_assistant_saves_at_once_and_asks_to_switch_on_or_run`; [test_automations.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_automations.py) `test_an_automation_is_drafted_from_a_sentence` | diff --git a/fastapi_backend/app/domain/automations.py b/fastapi_backend/app/domain/automations.py index 19440b89d..ded6d5c73 100644 --- a/fastapi_backend/app/domain/automations.py +++ b/fastapi_backend/app/domain/automations.py @@ -13,9 +13,9 @@ - What its runs left: `last_run_at`, `last_status`, `counters` ({runs, skipped, limited, missed}). - `removed_at`: deleted (switched off and hidden; restore brings it back). -Parts of the studio built in their own steps are called once they are there (automation_nodes.provided): schedules -and times (timers), webhooks (automation_hooks) and joins other than any (match.join_step). Until then a trigger or -join that needs one is refused as coming soon. +Parts of the studio built in their own steps are called through automation_nodes.provided: schedules and times +(timers), webhooks (automation_hooks) and joins other than any (match.join_step, whose absent and batch modes wait +on timers). A trigger or join whose part isn't there is refused as coming soon. """ from __future__ import annotations @@ -981,7 +981,7 @@ def _route(db, cfg, a, t, act, who, sec): if a["join"]["mode"] != "any": step = provided("match", "join_step") done = step(db, cfg, a, t["as"], act) if step is not None else [] - return sum(1 for c in done if fire(db, cfg, a["id"], t["as"], c.get("event") or act, c.get("events")) is not None) + return sum(1 for c in done if fire(db, cfg, a["id"], c.get("as") or t["as"], c.get("event") or act, c.get("events")) is not None) return 1 if fire(db, cfg, a["id"], t["as"], act, {t["as"]: act}) is not None else 0 diff --git a/fastapi_backend/app/domain/match.py b/fastapi_backend/app/domain/match.py index d9fb3ecaf..2f6a57de5 100644 --- a/fastapi_backend/app/domain/match.py +++ b/fastapi_backend/app/domain/match.py @@ -725,6 +725,308 @@ def _seen_words(s, negated): return words +# ---------- joins ---------- +MAX_BATCH = 500 # activities a batch collects at most +SLACK = 86400 # a join row waiting on a timer is kept this long past its time, for a timer that fires late +CUT, CUT_MANY = 2000, 300 # characters of text kept in a copy of one activity, and of each in a list + + +def _small(v, cut=CUT, depth=0): + """A small copy of an activity to keep while a join waits: text cut, lists and nesting kept short.""" + if isinstance(v, str): + return v if len(v) <= cut else v[: cut - 1] + "…" + if isinstance(v, list): + return [_small(x, cut, depth + 1) for x in v[:20]] if depth < 6 else [] + if isinstance(v, dict): + return {k: _small(x, cut, depth + 1) for k, x in v.items()} if depth < 6 else {} + return v + + +def _published(a): + return _time((a or {}).get("published"), dt.timezone.utc) or dt.datetime.now(dt.timezone.utc) + + +def _iso(t): + return t.astimezone(dt.timezone.utc).isoformat(timespec="seconds") + + +def _cas(db, ref, automation, change): + """Read a state row, work out its next state (`change(row or None)` gives (new row or None to delete it, result)) + and write it only if nobody wrote it meanwhile (its `v`), else read again. Returns the result.""" + for _ in range(8): + row = db.one("SELECT * FROM $r", r=ref) + new, out = change(row) + if new is row: # nothing to change + return out + if new is not None: + new = {k: x for k, x in new.items() if k != "id"} + if row is None: + if new is None: + return out + make_room(db, automation) + try: + db.run(["CREATE $r CONTENT $d"], r=ref, d={**new, "automation": automation, "v": 1, "at": _stamp()}) + return out + except Exception as e: # noqa: BLE001 - only a row someone else made first means read again + if "already exists" not in str(e): + raise + continue + v = row.get("v") or 0 + if new is None: + done = db.values("DELETE $r WHERE v = $v RETURN BEFORE", r=ref, v=v) + else: + done = db.values( + "UPDATE $r CONTENT $d WHERE v = $v", r=ref, d={**new, "automation": automation, "v": v + 1, "at": _stamp()}, v=v + ) + if done: + return out + raise RuntimeError("a join's state kept changing while it was written; the activity is left out") + + +def _timers(): + """The timers module (docs/automations.md, Timers), imported when used: it imports this module's checks.""" + from . import timers + + return timers + + +def _arm(db, ref, aid, key, due, kind, payload): + """Set a waiting join's timer; when it can't be set (the automation has too many waiting), drop the row so the + next activity starts afresh, and say why in the log.""" + try: + _timers().set(db, key, _iso(due), kind, payload, automation=aid) + except ValueError as e: + _take(db, {"automation": aid, "state": ref.id}) + log.warning("automation %s: its join's timer wasn't set: %s", aid, e) + + +def _aliases(automation): + return [t["as"] for t in automation.get("triggers") or [] if isinstance(t, dict) and t.get("as")] + + +def _named(j): + """The triggers a join waits on; activities of the others fire on their own, as with mode any.""" + if j["mode"] == "count": + return {j["of"]} + if j["mode"] == "absent": + return {j["after"], j["missing"]} + if j["mode"] == "batch" and j.get("of"): + return {j["of"]} + return None # all of them + + +def join_step(db, cfg, automation, alias, activity): + """What one activity of trigger `alias` completes in the automation's join: a list of completions, each + {"as", "events": {alias: activity | [activities]}, "event": the last}, for automations.fire. The caller claims + each activity once per automation and trigger first. Mode any, and triggers the join doesn't name, complete at + once; all, count and absent keep what they wait on in automation_state; absent and batch set timers + (join.absent, join.batch) that complete later.""" + from . import automation_spec + + aliases = _aliases(automation) + if not aliases: + raise ValueError("a join needs the automation's triggers") + j = automation_spec.check_join(automation.get("join"), aliases) + one = [{"as": alias, "events": {alias: activity}, "event": activity}] + named = _named(j) + if j["mode"] == "any" or (named is not None and alias not in named): + return one + aid, within = automation["id"], automation_spec.check_duration(j["within"]) + key = "" + if j.get("key"): + k = flow.dig(activity, j["key"]) + if k is None: + return [] # nothing to pair it with + key = _canon(k) + t = _published(activity) + if j["mode"] == "all": + return _all(db, aid, aliases, alias, activity, key, t, within) + if j["mode"] == "count": + return _count(db, aid, j, alias, activity, key, t, within) + if j["mode"] == "absent": + return _absent(db, aid, j, alias, activity, key, t, within) + return _batch(db, aid, alias, activity, t, within) + + +def _close(entries, t, within): + return {k: e for k, e in entries.items() if abs((_published(e) - t).total_seconds()) <= within} + + +def _all(db, aid, aliases, alias, activity, key, t, within): + ref = state_ref(aid, "join", "all", key) + + def change(row): + got = _close(dict((row or {}).get("events") or {}), t, within) + got[alias] = _small(activity) + if all(a in got for a in aliases): + return None, [{"as": alias, "events": {a: got[a] for a in aliases}, "event": activity}] + last = max(_published(e) for e in got.values()) + return { + "kind": "join", + "alias": "all", + "key": key[:300], + "events": got, + "expires_at": _iso(last + dt.timedelta(seconds=within)), + }, [] + + return _cas(db, ref, aid, change) + + +def _count(db, aid, j, alias, activity, key, t, within): + ref = state_ref(aid, "join", "count", key) + + def change(row): + got = [e for e in (row or {}).get("events") or [] if abs((_published(e) - t).total_seconds()) <= within] + got.append(_small(activity, CUT_MANY)) + if len(got) >= j["at_least"]: # crossed: one run, and it starts counting again + return None, [{"as": alias, "events": {alias: got[:-1] + [activity]}, "event": activity}] + last = max(_published(e) for e in got) + return { + "kind": "join", + "alias": "count", + "key": key[:300], + "events": got, + "expires_at": _iso(last + dt.timedelta(seconds=within)), + }, [] + + return _cas(db, ref, aid, change) + + +def _timer_key(kind, aid, key=""): + return f"{kind}-{aid}" + (f"-{hashlib.sha1(key.encode()).hexdigest()[:20]}" if key else "") + + +def _absent(db, aid, j, alias, activity, key, t, within): + ref = state_ref(aid, "join", "absent", key) + if alias == j["after"]: + due = t + dt.timedelta(seconds=within) + + def wait(row): + if row is not None: # already waiting: it completes with the first + return row, False + ev = {j["after"]: _small(activity)} + return { + "kind": "join", + "alias": "absent", + "key": key[:300], + "events": ev, + "due": _iso(due), + "expires_at": _iso(due + dt.timedelta(seconds=SLACK)), + }, True + + if _cas(db, ref, aid, wait): + _arm(db, ref, aid, _timer_key("absent", aid, key), due, "join.absent", {"automation": aid, "state": ref.id, "as": j["after"]}) + return [] + + def came(row): # the missing one came: nothing is absent, if it came after what it answers + first = ((row or {}).get("events") or {}).get(j["after"]) + if first is None or t < _published(first): + return row, False + return None, True + + if _cas(db, ref, aid, came): + _timers().cancel(db, _timer_key("absent", aid, key), "the missing trigger came") + return [] + + +def _batch(db, aid, alias, activity, t, within): + ref = state_ref(aid, "join", "batch", "") + due = t + dt.timedelta(seconds=within) + + def add(row): + if row is None: + ev = {alias: [_small(activity, CUT_MANY)]} + return { + "kind": "join", + "alias": "batch", + "key": "", + "events": ev, + "n": 1, + "last": _small(activity), + "due": _iso(due), + "expires_at": _iso(due + dt.timedelta(seconds=SLACK)), + }, True + row = dict(row) + ev = {k: list(v) for k, v in (row.get("events") or {}).items()} + if (row.get("n") or 0) >= MAX_BATCH: + row["over"] = (row.get("over") or 0) + 1 + return row, False + ev.setdefault(alias, []).append(_small(activity, CUT_MANY)) + row.update(events=ev, n=(row.get("n") or 0) + 1, last=_small(activity)) + return row, False + + if _cas(db, ref, aid, add): + _arm(db, ref, aid, _timer_key("batch", aid), due, "join.batch", {"automation": aid, "state": ref.id, "as": alias}) + return [] + + +def _take(db, payload): + """A waiting join's row, taken (deleted) by the one process whose timer fired; None when it went meanwhile.""" + ref = R("automation_state", str(payload.get("state") or "")) + aid = payload.get("automation") + return _cas(db, ref, aid, lambda row: (None, row)) + + +def _fire(db, cfg, aid, completion): + from . import automations + + return automations.fire(db, cfg, aid, completion["as"], completion["event"], events=completion["events"]) + + +def absent_fired(db, cfg, payload): + """Timer join.absent: the missing trigger didn't come in time, so the automation runs with the first event.""" + row = _take(db, payload) + if not row: + return None + first = (row.get("events") or {}).get(payload["as"]) + return _fire(db, cfg, payload["automation"], {"as": payload["as"], "events": {payload["as"]: first}, "event": first}) + + +def batch_fired(db, cfg, payload): + """Timer join.batch: the batch's time is up, so the automation runs once with what it collected.""" + row = _take(db, payload) + if not row: + return None + if row.get("over"): + log.warning("automation %s: its batch kept %s activities and left out %s", payload["automation"], MAX_BATCH, row["over"]) + return _fire(db, cfg, payload["automation"], {"as": payload["as"], "events": row.get("events") or {}, "event": row.get("last")}) + + +def join_ended(db, payload, status, why): + """A join's timer ended without firing (its automation was switched off, or Lens was off past the catch-up + window): what it waited on goes too, so the next activity starts afresh.""" + _take(db, payload) + + +TIMER_HANDLERS = {"join.absent": absent_fired, "join.batch": batch_fired} + + +def register_timers(): + """Give the timers module the join handlers (timers.py calls this as it finishes loading).""" + timers = _timers() + for kind, fn in TIMER_HANDLERS.items(): + timers.handler(kind, ended=join_ended)(fn) + + +def describe_join(join, kind=None): + """A join in plain words: "all of the triggers within 10 minutes, for the same sender".""" + from . import automation_spec + + j = automation_spec.check_join(join) + if j["mode"] == "any": + return "each trigger on its own" + within = duration_words(j["within"]) + same = f", for the same {_label(j['key'], kind)}" if j.get("key") else "" + if j["mode"] == "all": + return f"all of the triggers within {within}{same}" + if j["mode"] == "count": + return f"{j['at_least']} times {j['of']} within {within}{same}" + if j["mode"] == "absent": + return f"{j['after']}, then no {j['missing']} within {within}{same}" + who = f"{j['of']} brings" if j.get("of") else "the triggers bring" + return f"what {who} in {within}, together in one run" + + # ---------- reading trees ---------- def leaves(tree): """The tests in a tree, in order (not the lookbacks: `lookbacks`).""" diff --git a/fastapi_backend/app/domain/timers.py b/fastapi_backend/app/domain/timers.py index aca3b0cdb..7c96f74bb 100644 --- a/fastapi_backend/app/domain/timers.py +++ b/fastapi_backend/app/domain/timers.py @@ -818,3 +818,7 @@ def clock_state(db, automation, now=None): } events.kind("clock.tick", **CLOCK_KIND) # first without its sample, which is built with the kind events.kind("clock.tick", **CLOCK_KIND, sample=tick(4, "morning", "2026-10-16T06:00:00Z", "Europe/London", "0 7 * * 1-5", [3])) + +from . import match # noqa: E402 - joins that wait (join.absent, join.batch) register their handlers here + +match.register_timers() diff --git a/fastapi_backend/tests/domain/test_automation_conditions.py b/fastapi_backend/tests/domain/test_automation_conditions.py index 7a7a26075..2fd60161a 100644 --- a/fastapi_backend/tests/domain/test_automation_conditions.py +++ b/fastapi_backend/tests/domain/test_automation_conditions.py @@ -1,13 +1,16 @@ -"""Conditions in running automations (docs/automations.md, Running automations): an automation's If looks back over -the stream (`seen`) in its own namespaces, as its person may see; `changed` in a trigger's where compares each -activity with the last one; the If and a condition node testing the same thing agree within a run; a dry run keeps -nothing.""" +"""Conditions and joins in running automations (docs/automations.md, Running automations): an automation's If looks +back over the stream (`seen`) in its own namespaces, as its person may see; `changed` in a trigger's where compares +each activity with the last one; the If and a condition node testing the same thing agree within a run; a dry run +keeps nothing; an absent join waits on a real timer, and switching the automation off drops what it waited on.""" from __future__ import annotations +import datetime as dt +import time + import pytest -from app.domain import auth, automations, events, store +from app.domain import auth, automations, events, store, timers from tests.helpers import make_user @@ -26,6 +29,7 @@ def node(nid, t, **config): def env(db, cfg): before = dict(events._KINDS) events.kind("test_mail.came", label="Mail came", source="log", as_type="Create", object_type="Note") + events.kind("test_mail.sent", label="Mail sent", source="log", as_type="Create", object_type="Note") events._seen.clear() events._claimed.clear() pods, calls = store.ns_id(db, "pods"), store.ns_id(db, "calls") @@ -35,10 +39,15 @@ def env(db, cfg): events._KINDS.update(before) -def mail(db, cfg, sender, space, name="Trip"): - aid = events.emit( - db, "test_mail.came", actor=events.LENS, obj={"type": "Note", "name": name, "from": [sender]}, space=space, summary=name - ) +def tick(): + """Wait for the next second, as store.now() counts them (the follower starts after its first look's second).""" + start = store.now() + while store.now() == start: + time.sleep(0.05) + + +def mail(db, cfg, sender, space, name="Trip", kind="test_mail.came"): + aid = events.emit(db, kind, actor=events.LENS, obj={"type": "Note", "name": name, "from": [sender]}, space=space, summary=name) return events.get(db, cfg, aid, events.viewer(1, None, admin=True)) @@ -81,3 +90,29 @@ def test_the_if_and_a_condition_node_agree_and_dry_runs_keep_nothing(db, cfg, en run = automations.fire(db, cfg, aid, "mail", mail(db, cfg, "door", env["pods"], "Closed")) assert run and automations.get_run(db, run)["trace"]["a"]["status"] == "done" # the node saw the If's change assert automations.fire(db, cfg, aid, "mail", mail(db, cfg, "door", env["pods"], "Closed")) is None + + +def test_an_absent_join_waits_on_a_timer_and_switching_off_drops_it(db, cfg, env): + triggers = [*MAIL, {"as": "reply", "on": "event", "kinds": ["test_mail.sent"]}] + join = {"mode": "absent", "after": "mail", "missing": "reply", "within": "1h", "key": "object.name"} + aid = make(db, cfg, env["ed"], space=env["pods"], triggers=triggers, join=join) + assert automations.dispatch(db, cfg) == 0 # the first look marks where the follower starts + tick() + mail(db, cfg, "office@stmarys.org", env["pods"], "Trip") + mail(db, cfg, "club@example.com", env["pods"], "Club") + assert automations.dispatch(db, cfg) == 0 + mail(db, cfg, "me@home", env["pods"], "Club", kind="test_mail.sent") # Club was answered in time + assert automations.dispatch(db, cfg) == 0 + waiting = db.rows("SELECT key, status FROM timer WHERE automation = $a ORDER BY status", a=aid) + assert [r["status"] for r in waiting] == ["cancelled", "waiting"] + soon = dt.datetime.now(dt.UTC) + dt.timedelta(minutes=61) + assert timers.due(db, cfg, now=soon) == 1 + [r] = automations.runs(db, aid, limit=5) + run = automations.get_run(db, r["id"]) + assert run["trigger"] == "mail" and run["intents"][0]["result"]["line"] == "Mail from office@stmarys.org" + assert not db.rows("SELECT id FROM automation_state WHERE kind = 'join'") + # switched off while one waits: its timer is called off and what it waited on goes with it + mail(db, cfg, "office@stmarys.org", env["pods"], "Outing") + assert automations.dispatch(db, cfg) == 0 and db.rows("SELECT id FROM automation_state WHERE kind = 'join'") + automations.enable(db, cfg, aid, on=False) + assert not db.rows("SELECT id FROM automation_state WHERE kind = 'join'") diff --git a/fastapi_backend/tests/domain/test_match_joins.py b/fastapi_backend/tests/domain/test_match_joins.py new file mode 100644 index 000000000..9b3fc1614 --- /dev/null +++ b/fastapi_backend/tests/domain/test_match_joins.py @@ -0,0 +1,180 @@ +"""Joins (docs/automations.md, Joins): how several triggers make one run. all waits for every trigger within a window +(on the same key), count for n of one trigger, absent for one trigger not followed by another in time (with a fake +timer), batch collects for a while; the state they keep is capped, expires, survives two writers at once, and each +join reads in words.""" + +from __future__ import annotations + +import types + +import pytest + +from app.domain import automations, match + +T0 = "2026-10-15T16:00:00Z" + + +def at(minutes): + """A published time, `minutes` after T0.""" + h, m = divmod(16 * 60 + minutes, 60) + return f"2026-10-15T{h:02d}:{m:02d}:00Z" + + +def act(aid, minutes, sender="office@stmarys.org", **obj): + return {"id": aid, "kind": "x.y", "published": at(minutes), "object": {"from": [sender], **obj}, "summary": aid} + + +def auto(join, *aliases, aid=4): + return {"id": aid, "join": join, "triggers": [{"as": a} for a in aliases or ("mail", "cal")]} + + +@pytest.fixture +def clock(monkeypatch): + """Fake timers and a fake automations.fire, recording what they were asked.""" + timers = types.SimpleNamespace(set_=[], cancelled=[]) + timers.set = lambda db, key, when, kind, payload, automation=None: timers.set_.append((key, when, kind, payload)) + timers.cancel = lambda db, key, why="": timers.cancelled.append(key) or True + fired = [] + monkeypatch.setattr(match, "_timers", lambda: timers) + monkeypatch.setattr( + automations, + "fire", + lambda db, cfg, aid, alias, activity, events=None, dry=False: fired.append((aid, alias, activity, events)) or 51, + ) + timers.fired = fired + return timers + + +def step(db, automation, alias, a): + return match.join_step(db, {}, automation, alias, a) + + +def test_any_and_triggers_a_join_doesnt_name_fire_at_once(db): + a = act("m1", 0) + assert step(db, auto({"mode": "any"}), "mail", a) == [{"as": "mail", "events": {"mail": a}, "event": a}] + count = auto({"mode": "count", "of": "mail", "at_least": 2, "within": "5m"}) + assert step(db, count, "cal", a) == [{"as": "cal", "events": {"cal": a}, "event": a}] + with pytest.raises(ValueError, match="the join's of is the name of one of the triggers"): + step(db, auto({"mode": "count", "of": "nope", "at_least": 2, "within": "5m"}), "mail", a) + with pytest.raises(ValueError, match="a join needs the automation's triggers"): + step(db, {"id": 4, "join": {"mode": "any"}}, "mail", a) + + +def test_all_waits_for_every_trigger_within_the_window_on_the_same_key(db): + j = auto({"mode": "all", "within": "1d", "key": "object.from.0"}) + m1, c_other, c1 = act("m1", 0), act("c0", 10, sender="club@example.com"), act("c1", 30) + assert step(db, j, "mail", m1) == [] + assert step(db, j, "cal", c_other) == [] # another sender: its own key + done = step(db, j, "cal", c1) + assert done == [{"as": "cal", "events": {"mail": match._small(m1), "cal": c1}, "event": c1}] + assert step(db, j, "mail", act("m2", 40)) == [] # it starts again + # too far apart: the older one is dropped + late = auto({"mode": "all", "within": "10m"}, aid=5) + assert step(db, late, "mail", act("m3", 0)) == [] + assert step(db, late, "cal", act("c3", 20)) == [] + assert step(db, late, "mail", act("m4", 25))[0]["events"]["cal"]["id"] == "c3" + assert step(db, j, "mail", {"id": "nokey", "published": T0}) == [] # nothing at the key: nothing to pair with + + +def test_count_fires_once_per_crossing_then_counts_again(db): + j = auto({"mode": "count", "of": "login", "at_least": 3, "within": "5m", "key": "object.from.0"}, "login") + assert step(db, j, "login", act("l1", 0)) == [] + assert step(db, j, "login", act("l2", 1)) == [] + assert step(db, j, "login", act("x1", 1, sender="other")) == [] + done = step(db, j, "login", act("l3", 2)) + assert len(done) == 1 and [a["id"] for a in done[0]["events"]["login"]] == ["l1", "l2", "l3"] + assert done[0]["event"]["id"] == "l3" + assert step(db, j, "login", act("l4", 3)) == [] # reset: counting starts over + assert step(db, j, "login", act("l5", 4)) == [] + assert step(db, j, "login", act("l6", 20)) == [] # l4 and l5 are past the window + assert step(db, j, "login", act("l7", 21)) == [] + assert [a["id"] for a in step(db, j, "login", act("l8", 22))[0]["events"]["login"]] == ["l6", "l7", "l8"] + + +def test_absent_waits_with_a_timer_and_the_missing_one_cancels_it(db, cfg, clock): + j = auto({"mode": "absent", "after": "mail", "missing": "reply", "within": "2d", "key": "object.thread"}, "mail", "reply") + first = act("m1", 0, thread="t1") + assert step(db, j, "mail", first) == [] + assert step(db, j, "mail", act("m2", 5, thread="t1")) == [] # already waiting: it keeps the first + assert len(clock.set_) == 1 + key, when, kind, payload = clock.set_[0] + assert key.startswith("absent-4-") and when == "2026-10-17T16:00:00+00:00" and kind == "join.absent" + # a reply on another thread changes nothing; one on this thread, after the mail, cancels the timer + assert step(db, j, "reply", act("r0", 10, thread="t2")) == [] and clock.cancelled == [] + assert step(db, j, "reply", act("r1", 60, thread="t1")) == [] + assert clock.cancelled == [key] + assert match.absent_fired(db, cfg, payload) is None and clock.fired == [] # nothing left to complete + # no reply in time: the timer runs the automation with the first mail + assert step(db, j, "mail", act("m3", 70, thread="t3")) == [] + assert step(db, j, "reply", act("r_before", 65, thread="t3")) == [] # a reply from before the mail doesn't count + payload = clock.set_[-1][3] + assert match.absent_fired(db, cfg, payload) == 51 + assert clock.fired == [(4, "mail", match._small(act("m3", 70, thread="t3")), {"mail": match._small(act("m3", 70, thread="t3"))})] + assert match.absent_fired(db, cfg, payload) is None # fired once + + +def test_batch_collects_then_runs_once(db, cfg, clock, monkeypatch): + monkeypatch.setattr(match, "MAX_BATCH", 3) + j = auto({"mode": "batch", "within": "1h", "of": "alert"}, "alert", "other") + assert step(db, j, "other", act("o1", 0))[0]["event"]["id"] == "o1" # not batched: of names alert + for n in range(5): + assert step(db, j, "alert", act(f"a{n}", n)) == [] + assert len(clock.set_) == 1 and clock.set_[0][0] == "batch-4" and clock.set_[0][2] == "join.batch" + assert clock.set_[0][1] == "2026-10-15T17:00:00+00:00" + assert match.batch_fired(db, cfg, clock.set_[0][3]) == 51 + aid, alias, last, events = clock.fired[0] + assert aid == 4 and alias == "alert" and [a["id"] for a in events["alert"]] == ["a0", "a1", "a2"] and last["id"] == "a2" + assert step(db, j, "alert", act("a9", 9)) == [] and len(clock.set_) == 2 # a new batch + + +def test_join_state_is_capped_and_expires(db, monkeypatch): + monkeypatch.setattr(match, "MAX_OPEN", 2) + j = auto({"mode": "count", "of": "mail", "at_least": 5, "within": "1h", "key": "object.from.0"}) + for n in range(4): + step(db, j, "mail", act(f"m{n}", n, sender=f"s{n}")) + assert len(db.rows("SELECT id FROM automation_state WHERE automation = 4")) == 2 + assert match.dropped(db, 4) == 2 + row = db.one("SELECT * FROM automation_state WHERE automation = 4 ORDER BY at DESC LIMIT 1") + assert row["kind"] == "join" and row["expires_at"] == "2026-10-15T17:03:00+00:00" + match.tidy(db, now="2026-10-15T17:02:30+00:00") + assert len(db.rows("SELECT id FROM automation_state WHERE automation = 4")) == 1 + match.tidy(db, now="2026-10-15T18:00:00+00:00") + assert db.rows("SELECT id FROM automation_state WHERE automation = 4") == [] + + +def test_two_writers_at_once_lose_nothing(db, monkeypatch): + j = auto({"mode": "count", "of": "mail", "at_least": 3, "within": "1h"}) + step(db, j, "mail", act("m1", 0)) + real, once = match._small, {"done": False} + + def meanwhile(v, cut=match.CUT, depth=0): + if not once["done"] and isinstance(v, dict) and v.get("id") == "m3": + once["done"] = True + step(db, j, "mail", act("m2", 1)) # another process adds one between this one's read and write + return real(v, cut, depth) + + monkeypatch.setattr(match, "_small", meanwhile) + done = step(db, j, "mail", act("m3", 2)) + assert [a["id"] for a in done[0]["events"]["mail"]] == ["m1", "m2", "m3"] + + +def test_joins_read_in_words(): + assert match.describe_join({"mode": "any"}) == "each trigger on its own" + assert match.describe_join(None) == "each trigger on its own" + assert ( + match.describe_join({"mode": "all", "within": "10m", "key": "object.from"}) + == "all of the triggers within 10 minutes, for the same object.from" + ) + assert match.describe_join({"mode": "count", "of": "login", "at_least": 3, "within": "5m"}) == "3 times login within 5 minutes" + assert ( + match.describe_join({"mode": "absent", "after": "mail", "missing": "reply", "within": "2d"}) == "mail, then no reply within 2 days" + ) + assert match.describe_join({"mode": "batch", "within": "1h"}) == "what the triggers bring in 1 hour, together in one run" + assert match.describe_join({"mode": "batch", "within": "1h", "of": "alert"}) == "what alert brings in 1 hour, together in one run" + + +def test_join_handlers_are_registered_with_timers(): + from app.domain import timers + + for kind, fn in match.TIMER_HANDLERS.items(): + assert timers._HANDLERS[kind].fn is fn and timers._HANDLERS[kind].ended is match.join_ended