From bfaae8682c4ade56ae08f54a02d603283cf380fe Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 23:10:01 +0000 Subject: [PATCH] feat(automations): draft an automation from a sentence POST /api/v1/automations/draft asks the model once to turn a sentence into a draft automation, with the kinds of activity and their fields, the match operators and the actions as its vocabulary. The reply is checked piece by piece: a trigger, its where, the If or an action that doesn't fit is dropped and said, and what the sentence asked for that can't be expressed yet (schedules) is listed. Nothing is saved. The call counts in the activity ledger and isn't made over budget. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01VCAZ9NEmBJNovqdzxvK3qi --- CHANGELOG.md | 3 + docs/api.md | 5 +- docs/automations.md | 23 ++ .../app/api/v1/routes/automations.py | 13 +- .../app/domain/automation_draft.py | 226 ++++++++++++++++++ fastapi_backend/app/schemas/automations.py | 13 + fastapi_backend/tests/api/test_automations.py | 19 ++ .../tests/domain/test_automation_draft.py | 140 +++++++++++ nextjs-frontend/app/openapi-client/index.ts | 7 + nextjs-frontend/app/openapi-client/sdk.gen.ts | 23 ++ .../app/openapi-client/types.gen.ts | 70 ++++++ nextjs-frontend/openapi.json | 105 ++++++++ threat-model.json | 2 +- threat-model.md | 2 +- threat-model.yaml | 4 +- 15 files changed, 649 insertions(+), 6 deletions(-) create mode 100644 fastapi_backend/app/domain/automation_draft.py create mode 100644 fastapi_backend/tests/domain/test_automation_draft.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 44dffa475..08bbfb731 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,9 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version `automation_actions`, `save_automation`, `test_automation` and `run_automation`, with problems in plain words. The assistant saves them switched off and asks before switching one on or running it ([Agents](docs/automations.md#agents)). +- **Describe an automation in words.** `POST /api/v1/automations/draft` asks the model once to turn a sentence into + a draft automation to try and save, and says what didn't fit and what it couldn't express + ([Describe it in words](docs/automations.md#describe-it-in-words)). ## 0.13.0 October 16, 2026 {id="0.13.0"} diff --git a/docs/api.md b/docs/api.md index c61293e9c..6e8f3bec8 100644 --- a/docs/api.md +++ b/docs/api.md @@ -186,6 +186,7 @@ resources (admins). GET /api/v1/automations POST /api/v1/automations GET /api/v1/automations/actions +POST /api/v1/automations/draft GET /api/v1/automations/{id} PATCH /api/v1/automations/{id} DELETE /api/v1/automations/{id} @@ -206,7 +207,9 @@ Automations act for the person who made them in namespaces they can edit ([Savin automations](automations.md#saving-automations)). Admins see and change all of them; anyone else the ones that work only in namespaces they can edit (`?removed=true` lists deleted ones, to restore). A save answers `{id, hooks}`. `/policy` lets an action node run without asking (`auto`) only from a signed-in session in the web app. -`/actions` lists what action nodes can do. The list adds each one's actions waiting (`waiting`) and cost this month +`/actions` lists what action nodes can do. `/draft {"text": ...}` asks the model to turn a sentence into a draft to +try and save, saying what didn't fit (`dropped`) and what it couldn't express (`unmapped`) ([Describe it in +words](automations.md#describe-it-in-words)). The list adds each one's actions waiting (`waiting`) and cost this month (`cost_month`). `/test` tries a saved automation, and `POST /automations/test {"draft": ...}` an unsaved one, on an activity (`activity`) or a kind's sample (`kind`) without keeping or sending anything; `/run` runs it now (on `activity`, else the newest one its triggers listen for). `/runs` pages its runs (`before`, `limit`; `next` for the diff --git a/docs/automations.md b/docs/automations.md index 5df5635ba..40a810eb9 100644 --- a/docs/automations.md +++ b/docs/automations.md @@ -46,6 +46,7 @@ merged is called built. | Saving automations: checks, namespaces, switching on and off, delete and restore, the policy, the API (`app/domain/automations.py`) | built | | Running automations: following the stream, limits, budgets, runs as `automation.ran`, trying and running now, the API (`app/domain/automations.py`) | built | | Automations for agents: the MCP tools and the assistant's, switching on and running now by approval | built | +| Describing an automation in words: a draft from a sentence (`POST /api/v1/automations/draft`) | built | | Joins, the studio | planned | ## Contract @@ -979,6 +980,27 @@ person who made it (`run_as`), with their permissions, in namespaces they can ed `GET /api/v1/automations/actions` lists what an action node can do: each action's params (a JSON Schema), its effect and what it gives back. +### Describe it in words + +Built (`app/domain/automation_draft.py`). `POST /api/v1/automations/draft {"text": "when a recording is added to pods, +log its title", "spaces": [3]}` asks the model once to turn the sentence into a draft, with the kinds of activity and +their fields, the [match operators](#match-trees) and the [actions](#actions-and-intents) as its vocabulary. The answer +is `{draft, dropped, unmapped}`: + +- `draft` is what `POST /automations` takes, switched off: its name, triggers (on activities, a webhook, whose + address comes when it is saved, by hand, or in time: a schedule, a time, or a time in an activity such as a + meeting's start, in UTC unless the sentence names a zone), If and a graph with its actions in order (an action that may reach + outside Lens hangs off the one before it, as [the leaf rule](#the-event-workflow-scope) has it). +- `dropped` says what in the model's draft didn't fit and why: a trigger, its `where`, the If, an action that doesn't + exist or params that don't fit its schema. With no trigger left it runs when you run it; with no action left it has + no Then yet. +- `unmapped` lists what the sentence asked for that it couldn't express, such as "when both arrive" (joins are + planned). + +Nothing is saved: try the draft (`POST /automations/test`), change it, then save it. The call counts in the [activity +ledger](activity.md) for you and its namespaces, and isn't made once a budget they count for, or everything's, is +used up. + ### Agents Built. Over [MCP](mcp.md) an agent uses `automations` (list them, read one with its latest runs, or one run), @@ -1128,3 +1150,4 @@ held run until you pick. last status and the stream). - Automations that listen for every kind (or hundreds of automations): an index of triggers by kind and `where`, so a look reads each activity once whatever the number of automations. +- Drafts from a sentence: the person's own time zone when the sentence names none (today UTC). diff --git a/fastapi_backend/app/api/v1/routes/automations.py b/fastapi_backend/app/api/v1/routes/automations.py index 58059b03b..afd88c7d2 100644 --- a/fastapi_backend/app/api/v1/routes/automations.py +++ b/fastapi_backend/app/api/v1/routes/automations.py @@ -9,12 +9,14 @@ from fastapi import APIRouter, HTTPException, Query from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer, domain_errors -from app.domain import actions, auth, automations, events, store +from app.domain import actions, auth, automation_draft, automations, events, store from app.schemas.automations import ( ActionInfo, Automation, AutomationCatalog, AutomationCreate, + AutomationDrafted, + AutomationDraftRequest, AutomationDraftTry, AutomationPolicy, AutomationRun, @@ -210,6 +212,15 @@ def decide_held_run(run: int, body: HeldRunDecision, user: Writer, acl: Acl, db: return AutomationStarted(run=got, status=None if got is None else automations.get_run(db, got)["status"]) +@router.post("/draft") +def draft_automation(body: AutomationDraftRequest, user: Writer, db: Db, cfg: Cfg) -> AutomationDrafted: + """Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the + sentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What + doesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it.""" + with domain_errors(): + return AutomationDrafted(**automation_draft.draft(db, cfg, body.text, _who(user), body.spaces)) + + @router.post("/test") def test_draft(body: AutomationDraftTry, user: CurrentUser, acl: Acl, db: Db, cfg: Cfg) -> AutomationTried: """Try an automation before you save it: a dry run on an activity, or on a kind's sample (default: its first diff --git a/fastapi_backend/app/domain/automation_draft.py b/fastapi_backend/app/domain/automation_draft.py new file mode 100644 index 000000000..9d622c6ba --- /dev/null +++ b/fastapi_backend/app/domain/automation_draft.py @@ -0,0 +1,226 @@ +"""Describe an automation in words (docs/automations.md, Describe it in words): one model call turns a sentence such as +"when a new recording is added to pods, log its title" into a draft automation, with the kinds of activity, the +match operators and the actions as its vocabulary. The draft is checked piece by piece: what doesn't fit is dropped +and said, and what the model couldn't map is listed. Nothing is saved; the person (or agent) reads it, changes it and +saves it. + +The call counts in the activity ledger for the person and the namespaces it would work in, and isn't made once a +budget they count for (or everything's) is used up. +""" + +from __future__ import annotations + +import json + +from . import actions, activity, automation_spec, automations, budgets, events, llm, match, store + +TEXT_MAX = 2000 +SCHEMA = { + "type": "object", + "properties": { + "name": {"type": "string"}, + "description": {"type": "string"}, + "triggers": {"type": "array", "items": {"type": "object"}}, + "if": {"type": "object"}, + "actions": { + "type": "array", + "items": { + "type": "object", + "properties": {"action": {"type": "string"}, "params": {"type": "object"}}, + "required": ["action"], + }, + }, + "unmapped": {"type": "array", "items": {"type": "string"}}, + }, + "required": ["name", "triggers", "actions"], +} +SYSTEM = """You turn a person's sentence into an automation for Lens, an archive of their documents, recordings and +messages. An automation has triggers (when), an optional If (a test) and actions (then). Reply with JSON only: +{"name": short name, "description": one sentence, "triggers": [...], "if": a match tree or omit it, + "actions": [{"action": name, "params": {...}}], "unmapped": [each part of the sentence you couldn't express]}. + +Triggers: {"as": a short lowercase name, "on": "event", "kinds": [kinds below], "where": a match tree on the +activity (paths like object.name)}, {"as": a short lowercase name, "on": "webhook"} (an address another service or +device calls; Lens gives it when the automation is saved), or {"as": "manual", "on": "manual"}. TIMES + +A match tree is {"all": [...]}, {"any": [...]} or {"not": tree} around tests {"path": ..., "op": ..., "value": ...}; +op is one of OPS. The If's paths start with the run context's keys (event, events, trigger, automation, namespace, +user, now), like event.object.name. + +An action's params may hold templates over the run context, like "New: {{ event.object.name }}". Use only the kinds, +actions and params listed; anything else goes in unmapped. Never invent values the sentence doesn't give.""" + + +TIMES = { + "schedule": '{"as": a short lowercase name, "on": "schedule", "cron": five-field cron like "0 7 * * 1-5", "timezone": an ' + 'IANA zone like "Europe/London"}', + "at": '{"as": a short lowercase name, "on": "at", "at": a local time like "2026-10-20T09:00", "timezone": an IANA zone}', + "relative": '{"as": a short lowercase name, "on": "relative", "kinds": [kinds below], "where": a match tree, "field": ' + 'the time in the activity it counts from, like object.startTime, "offset": like "-30m" (before) or "2h" (after)}', +} + + +def _times(): + """What the model is told about triggers in time: the ones built, and the rest put in unmapped.""" + have = [on for on in TIMES if not _soon(on)] + if not have: + return "Schedules and times aren't available yet: put those parts in unmapped." + said = "Triggers in time: " + ", ".join(TIMES[on] for on in have) + ". With no time zone in the sentence, use UTC." + if len(have) < len(TIMES): + said += f" Triggers on {' or '.join(on for on in TIMES if on not in have)} aren't available yet: put those in unmapped." + return said + + +def _fields(schema, prefix="", depth=0): + """The dotted paths of an activity's fields with their titles, to name in tests.""" + out = [] + for k, v in ((schema or {}).get("properties") or {}).items(): + path = f"{prefix}{k}" + if v.get("type") == "object" and depth < 1: + out += _fields(v, path + ".", depth + 1) + elif v.get("title"): + out.append(f"{path} ({v['title']})") + return out + + +def vocabulary(): + """What the model may use: the kinds of activity with their fields, the operators and the actions.""" + kinds = [ + {"kind": k["name"], "label": k["label"], "fields": _fields(k.get("schema") or {})} for k in events.kinds() if not k.get("noisy") + ] + acts = [ + { + "action": a["name"], + "label": a["label"], + "description": a.get("description") or "", + "params": a["params"], + "required": a.get("required") or [], + } + for a in actions.catalog() + ] + return {"kinds": kinds, "ops": list(match.WORDS), "actions": acts} + + +def _soon(on): + """Whether a trigger needs a part of the studio that isn't here yet (automations.SOON).""" + need = automations.SOON.get(on) if isinstance(on, str) else None + return need is not None and automations.provided(need[0], need[1]) is None + + +def _triggers(raw, dropped, unmapped): + out, seen = [], set() + for n, t in enumerate(raw if isinstance(raw, list) else [], 1): + if isinstance(t, dict) and _soon(t.get("on")): + unmapped.append(f"a trigger on {t['on']} (coming soon)") + continue + try: + got = automation_spec.check_trigger(t, n) + except ValueError as e: + where = isinstance(t, dict) and t.pop("where", None) + try: + got = automation_spec.check_trigger(t, n) if where else None + except ValueError: + got = None + if got is None: + dropped.append(str(e)) + continue + dropped.append(f"{e} (its where was left out)") + if got["as"] in seen: + continue + seen.add(got["as"]) + out.append(got) + if not out: + out = [{"as": "manual", "on": "manual"}] + dropped.append("no trigger fit, so it runs when you run it") + return out[: automation_spec.MAX_TRIGGERS] + + +def _actions(raw, dropped): + out = [] + for n, a in enumerate(raw if isinstance(raw, list) else [], 1): + name = a.get("action") if isinstance(a, dict) else None + try: + spec = actions.get(str(name)) + except KeyError: + dropped.append(f"action {n}: there's no action {name}") + continue + try: + params = actions.check_params(spec, a.get("params") or {}) + except ValueError as e: + dropped.append(f"action {n}: {e}") + continue + out.append((spec, params)) + return out + + +def _graph(acts): + """The actions in order after the input node; one that reaches outside Lens (or may) is a leaf off the one + before it, as the leaf rule has it, and the rest go on from there.""" + nodes, edges, last = [{"id": "input", "type": "input", "config": {}}], [], "input" + for n, (spec, params) in enumerate(acts, 1): + nid = f"a{n}" + nodes.append({"id": nid, "type": "action", "config": {"action": spec.name, "params": params}}) + edges.append({"source": last, "target": nid}) + if isinstance(spec.effect, str) and spec.effect != "out": # one worked out from its params may reach out + last = nid + return {"nodes": nodes, "edges": edges} + + +def draft(db, cfg, text, user, spaces=None): + """{draft, dropped, unmapped}: `draft` is what POST /automations takes (switched off), `dropped` the parts of the + model's reply that didn't fit and why, `unmapped` the parts of the sentence it couldn't express.""" + text = " ".join(str(text or "").split()) + if not text: + raise ValueError("say what the automation should do") + if len(text) > TEXT_MAX: + raise ValueError(f"say it in at most {TEXT_MAX} characters") + account = automations._account(db, (user or {}).get("id")) + if not account: + raise ValueError("an automation acts for a person; sign in") + can = automations.editable(db, account) + if not can: + raise ValueError(f"{account.get('name') or account['email']} can't edit any namespace, so an automation couldn't work anywhere") + spaces = [int(s) for s in spaces] if spaces else None + if spaces and not set(spaces) <= can: + raise ValueError("you can make automations only in namespaces you can edit") + if not llm.configured(cfg): + raise ValueError("no language model is configured (Settings → LLM provider)") + refs = [f"account:{account['id']}", *(f"space:{s}" for s in spaces or [])] + why = budgets.used_up(db, refs) + if why: + raise ValueError(f"over budget: {why}") + vocab = vocabulary() + system = SYSTEM.replace("OPS", ", ".join(vocab["ops"])).replace("TIMES", _times()) + ask = ( + f"Kinds of activity and their fields:\n{json.dumps(vocab['kinds'])}\n\nActions:\n{json.dumps(vocab['actions'])}\n\nSentence: {text}" + ) + with activity.scope(db, *refs, cfg=cfg): + try: + reply = llm.json_out(cfg, system, ask, SCHEMA) + except llm.LLMError as e: + raise ValueError(f"the model couldn't draft it: {e}") from None + dropped, unmapped = [], [str(u)[:300] for u in reply.get("unmapped") or [] if str(u).strip()][:20] + out = { + "name": " ".join(str(reply.get("name") or "").split())[:200] or "New automation", + "description": " ".join(str(reply.get("description") or "").split())[:2000] or None, + "enabled": False, + "triggers": _triggers(reply.get("triggers"), dropped, unmapped), + } + if spaces: + out["spaces"] = spaces + if reply.get("if"): + try: + out["if"] = automation_spec.check_if(reply["if"]) + except ValueError as e: + dropped.append(f"the If: {e}") + acts = _actions(reply.get("actions"), dropped) + if acts: + out["graph"] = _graph(acts) + else: + dropped.append("no action fit; add one before saving") + if acts: + try: + automations._check(db, dict(out), None, account) + except ValueError as e: + dropped.append(f"as a whole: {e}") + return {"draft": store.clean(out), "dropped": dropped, "unmapped": unmapped} diff --git a/fastapi_backend/app/schemas/automations.py b/fastapi_backend/app/schemas/automations.py index 9d0cbbed7..234bb6532 100644 --- a/fastapi_backend/app/schemas/automations.py +++ b/fastapi_backend/app/schemas/automations.py @@ -206,3 +206,16 @@ class AutomationStarted(ResponseModel): class AutomationUpcoming(ResponseModel): times: list[Any] = Field(default_factory=list, description="when it runs next, from its schedules and timers (coming soon)") + + +class AutomationDraftRequest(RequestModel): + text: str = Field( + max_length=2000, description="what it should do, in a sentence, like: when a recording is added to pods, log its title" + ) + spaces: list[int] | None = Field(None, description="namespace ids it would work in; default: each one you can edit") + + +class AutomationDrafted(ResponseModel): + draft: dict[str, Any] = Field(description="what POST /automations takes, switched off: name, triggers, if, graph") + dropped: list[str] = Field(default_factory=list, description="parts of the model's draft that didn't fit, and why") + unmapped: list[str] = Field(default_factory=list, description="parts of the sentence it couldn't express (schedules, say)") diff --git a/fastapi_backend/tests/api/test_automations.py b/fastapi_backend/tests/api/test_automations.py index 9e45a2110..30e28be8b 100644 --- a/fastapi_backend/tests/api/test_automations.py +++ b/fastapi_backend/tests/api/test_automations.py @@ -339,3 +339,22 @@ def test_a_run_held_over_budget_waits_for_the_persons_pick(client, db, cfg, env) assert client.get(f"{URL}/runs/{held['run']}", headers=he).json()["status"] == "released" r = client.post(f"{URL}/runs/{held['run']}/decide", headers=he, json={"run": False}) assert r.status_code == 400 and "isn't waiting on a budget" in r.json()["detail"] + + +def test_an_automation_is_drafted_from_a_sentence(client, db, env, monkeypatch): + from app.domain import automation_draft + + reply = {"name": "Say new", "triggers": NEW, "actions": [{"action": "log_line", "params": {"text": "New: {{ event.object.name }}"}}]} + monkeypatch.setattr(automation_draft.llm, "configured", lambda cfg: True) + monkeypatch.setattr(automation_draft.llm, "json_out", lambda cfg, system, user, schema, model=None: reply) + r = client.post(f"{URL}/draft", headers=env["he"], json={"text": "when something new comes in, log its name"}) + assert r.status_code == 200, r.text + got = r.json() + assert got["draft"]["enabled"] is False and got["draft"]["triggers"] == [{**NEW[0], "from_automations": False}] + assert got["dropped"] == [] and got["unmapped"] == [] + # it tries and saves as it is; drafting saved nothing + assert db.rows("SELECT id FROM automation") == [] + assert client.post(f"{URL}/test", headers=env["he"], json={"draft": got["draft"]}).json()["status"] == "done" + assert client.post(URL, headers=env["he"], json=got["draft"]).status_code == 200 + assert "can't edit any namespace" in problem(client, env["hv"], {"text": "log it"}, f"{URL}/draft") + assert client.post(f"{URL}/draft", headers=env["he"], json={"text": "x" * 2001}).status_code == 422 diff --git a/fastapi_backend/tests/domain/test_automation_draft.py b/fastapi_backend/tests/domain/test_automation_draft.py new file mode 100644 index 000000000..4ad1186c6 --- /dev/null +++ b/fastapi_backend/tests/domain/test_automation_draft.py @@ -0,0 +1,140 @@ +"""Describing an automation in words (docs/automations.md, Describe it in words): one model call, with the kinds, the +operators and the actions as its vocabulary; what doesn't fit is dropped and said; nothing is saved; refused over +budget; and the call counts in the activity ledger.""" + +from __future__ import annotations + +import pytest + +from app.domain import activity, auth, automation_draft, automations, budgets, store +from tests.helpers import make_user, without + + +@pytest.fixture +def env(db, cfg, monkeypatch): + make_user(db, "ed@x.io", "editor password 1", roles={"pods": "editor"}) + make_user(db, "vi@x.io", "viewer password 1", roles={"pods": "viewer"}) + cfg = {**cfg, "llm": {**cfg["llm"], "base_url": "http://model.invalid/v1", "model": "m"}} + asked = [] + + def reply(answer): + def fake(cfg, system, user, schema, model=None): + asked.append({"system": system, "user": user}) + activity.record("out", "model.chat", [], cfg, db, model="m", cost_usd=0.01) + return answer + + monkeypatch.setattr(automation_draft.llm, "json_out", fake) + + return { + "cfg": cfg, + "ed": {"id": auth.find_account(db, "ed@x.io")["id"]}, + "vi": {"id": auth.find_account(db, "vi@x.io")["id"]}, + "pods": store.ns_id(db, "pods"), + "asked": asked, + "reply": reply, + } + + +def test_a_sentence_becomes_a_draft_that_saves(db, env): + cfg = env["cfg"] + env["reply"]( + { + "name": "Log new pods", + "description": "Logs each new recording's title", + "triggers": [ + { + "as": "new", + "on": "event", + "kinds": ["asset.added"], + "where": {"all": [{"path": "object.media", "op": "equals", "value": "audio"}]}, + } + ], + "if": {"all": [{"path": "event.object.name", "op": "exists"}]}, + "actions": [ + {"action": "notify", "params": {"to": "me", "text": "New: {{ event.object.name }}"}}, + {"action": "log_line", "params": {"text": "Logged {{ event.object.name }}"}}, + ], + "unmapped": ["only on weekdays"], + } + ) + got = automation_draft.draft(db, cfg, "When an audio recording is added, tell me and log it, on weekdays", env["ed"]) + d = got["draft"] + assert got["dropped"] == [] and got["unmapped"] == ["only on weekdays"] + assert d["name"] == "Log new pods" and d["enabled"] is False and d["triggers"][0]["where"]["all"][0]["value"] == "audio" + # an action that may reach outside Lens is a leaf off the one before it; the rest go on from there + assert d["graph"]["edges"] == [{"source": "input", "target": "a1"}, {"source": "input", "target": "a2"}] + # the model was given the vocabulary + said = env["asked"][0] + assert "asset.added" in said["user"] and "log_line" in said["user"] and "contains" in said["system"] + assert "object.name (Title)" in said["user"] + # nothing was saved, and the call counted for the person + assert db.rows("SELECT id FROM automation") == [] + assert activity.costs(db, [f"account:{env['ed']['id']}"])[f"account:{env['ed']['id']}"]["calls"] == 1 + # it saves as it is + aid = automations.create(db, cfg, d, {**env["ed"]})["id"] + assert automations.get(db, aid)["enabled"] is False + + +def test_triggers_in_time_are_drafted_once_the_timers_are_there(db, env, monkeypatch): + cfg = env["cfg"] + morning = {"as": "morning", "on": "schedule", "cron": "0 7 * * 1-5", "timezone": "Europe/London"} + env["reply"]({"name": "Mornings", "triggers": [morning], "actions": [{"action": "log_line", "params": {"text": "Morning"}}]}) + got = automation_draft.draft(db, cfg, "every weekday at 7 log morning", env["ed"]) + assert got["draft"]["triggers"] == [morning] and got["dropped"] == [] and got["unmapped"] == [] + assert '"on": "schedule"' in env["asked"][0]["system"] and '"on": "relative"' in env["asked"][0]["system"] + # without them, the model is told to leave times out + without(monkeypatch, "timers") + automation_draft.draft(db, cfg, "every weekday at 7 log morning", env["ed"]) + assert '"on": "schedule"' not in env["asked"][1]["system"] and "aren't available yet" in env["asked"][1]["system"] + + +def test_what_doesnt_fit_is_dropped_and_said(db, env, monkeypatch): + without(monkeypatch, "timers") # schedules come with their own module + cfg = env["cfg"] + env["reply"]( + { + "name": "Mixed", + "triggers": [ + {"as": "morning", "on": "schedule", "cron": "0 7 * * *"}, + {"as": "door", "on": "webhook"}, + {"as": "new", "on": "event", "kinds": ["asset.added"], "where": {"all": [{"path": "x", "op": "matches"}]}}, + {"as": "odd", "on": "event", "kinds": ["no.such"]}, + ], + "if": {"all": [{"path": "nowhere.at.all", "op": "exists"}]}, + "actions": [ + {"action": "launch_rockets"}, + {"action": "log_line", "params": {"text": 5}}, + {"action": "log_line", "params": {"text": "ok"}}, + ], + } + ) + got = automation_draft.draft(db, cfg, "every morning and when something comes in, launch rockets", env["ed"]) + d = got["draft"] + assert got["unmapped"] == ["a trigger on schedule (coming soon)"] + assert [t["as"] for t in d["triggers"]] == ["door", "new"] and "where" not in d["triggers"][1] + assert any("its where was left out" in x for x in got["dropped"]) and any("no.such" in x for x in got["dropped"]) + assert any(x.startswith("the If:") for x in got["dropped"]) and "if" not in d + assert any("no action launch_rockets" in x for x in got["dropped"]) and any(x.startswith("action 2: ") for x in got["dropped"]) + assert [n["config"]["action"] for n in d["graph"]["nodes"][1:]] == ["log_line"] + # with nothing left: run by hand, and no Then + env["reply"]({"name": "", "triggers": [], "actions": []}) + got = automation_draft.draft(db, cfg, "do something", env["ed"]) + assert got["draft"]["triggers"] == [{"as": "manual", "on": "manual"}] and "graph" not in got["draft"] + assert got["draft"]["name"] == "New automation" and "no action fit; add one before saving" in got["dropped"] + + +def test_drafting_is_refused_when_it_cant_or_mustnt_ask(db, env, cfg): + env["reply"]({"name": "x", "triggers": [], "actions": []}) + with pytest.raises(ValueError, match="say what"): + automation_draft.draft(db, env["cfg"], " ", env["ed"]) + with pytest.raises(ValueError, match="can't edit any namespace"): + automation_draft.draft(db, env["cfg"], "log new things", env["vi"]) + with pytest.raises(ValueError, match="only in namespaces you can edit"): + automation_draft.draft(db, env["cfg"], "log new things", env["ed"], [store.ns_id(db, "calls")]) + with pytest.raises(ValueError, match="no language model"): + automation_draft.draft(db, {**cfg, "llm": {**cfg["llm"], "base_url": None}}, "log new things", env["ed"]) + activity.record("out", "model.chat", [f"space:{env['pods']}"], db=db, model="m", cost_usd=0.6) + budgets.put(db, f"space:{env['pods']}", usd=0.5) + with pytest.raises(ValueError, match="over budget"): + automation_draft.draft(db, env["cfg"], "log new things", env["ed"], [env["pods"]]) + assert env["asked"] == [] diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index d734dd408..b5db04b3b 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -186,6 +186,8 @@ export type { Automation, AutomationCatalog, AutomationCreate, + AutomationDrafted, + AutomationDraftRequest, AutomationDraftTry, AutomationPolicy, AutomationRun, @@ -892,6 +894,11 @@ export type { DownloadFileErrors, DownloadFileResponse, DownloadFileResponses, + DraftAutomationData, + DraftAutomationError, + DraftAutomationErrors, + DraftAutomationResponse, + DraftAutomationResponses, DropPasskeysData, DropPasskeysError, DropPasskeysErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index b93cd1e5b..552691e6a 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -421,6 +421,9 @@ import type { DownloadFileData, DownloadFileErrors, DownloadFileResponses, + DraftAutomationData, + DraftAutomationErrors, + DraftAutomationResponses, DropPasskeysData, DropPasskeysErrors, DropPasskeysResponses, @@ -4184,6 +4187,26 @@ export class Automations { }); } + /** + * Draft Automation + * + * Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the + * sentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What + * doesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it. + */ + public static draftAutomation( + options: Options, + ): RequestResult { + return (options.client ?? client).post({ + url: "/api/v1/automations/draft", + ...options, + headers: { + "Content-Type": "application/json", + ...options.headers, + }, + }); + } + /** * Test Draft * diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 9f85dbefc..06ed72de5 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -1678,6 +1678,24 @@ export type AutomationCreate = { } | null; }; +/** + * AutomationDraftRequest + */ +export type AutomationDraftRequest = { + /** + * Text + * + * what it should do, in a sentence, like: when a recording is added to pods, log its title + */ + text: string; + /** + * Spaces + * + * namespace ids it would work in; default: each one you can edit + */ + spaces?: Array | null; +}; + /** * AutomationDraftTry */ @@ -1706,6 +1724,33 @@ export type AutomationDraftTry = { draft: AutomationCreate; }; +/** + * AutomationDrafted + */ +export type AutomationDrafted = { + /** + * Draft + * + * what POST /automations takes, switched off: name, triggers, if, graph + */ + draft: { + [key: string]: unknown; + }; + /** + * Dropped + * + * parts of the model's draft that didn't fit, and why + */ + dropped?: Array; + /** + * Unmapped + * + * parts of the sentence it couldn't express (schedules, say) + */ + unmapped?: Array; + [key: string]: unknown; +}; + /** * AutomationPolicy */ @@ -26808,6 +26853,31 @@ export type DecideHeldRunResponses = { export type DecideHeldRunResponse = DecideHeldRunResponses[keyof DecideHeldRunResponses]; +export type DraftAutomationData = { + body: AutomationDraftRequest; + path?: never; + query?: never; + url: "/api/v1/automations/draft"; +}; + +export type DraftAutomationErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type DraftAutomationError = DraftAutomationErrors[keyof DraftAutomationErrors]; + +export type DraftAutomationResponses = { + /** + * Successful Response + */ + 200: AutomationDrafted; +}; + +export type DraftAutomationResponse = DraftAutomationResponses[keyof DraftAutomationResponses]; + export type TestDraftData = { body: AutomationDraftTry; path?: never; diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 7eacbd785..285f91af5 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -6103,6 +6103,48 @@ } } }, + "/api/v1/automations/draft": { + "post": { + "tags": [ + "automations" + ], + "summary": "Draft Automation", + "description": "Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the\nsentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What\ndoesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it.", + "operationId": "draft_automation", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AutomationDraftRequest" + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AutomationDrafted" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/automations/test": { "post": { "tags": [ @@ -32919,6 +32961,37 @@ ], "title": "AutomationCreate" }, + "AutomationDraftRequest": { + "properties": { + "text": { + "type": "string", + "maxLength": 2000, + "title": "Text", + "description": "what it should do, in a sentence, like: when a recording is added to pods, log its title" + }, + "spaces": { + "anyOf": [ + { + "items": { + "type": "integer" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Spaces", + "description": "namespace ids it would work in; default: each one you can edit" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "text" + ], + "title": "AutomationDraftRequest" + }, "AutomationDraftTry": { "properties": { "activity": { @@ -32969,6 +33042,38 @@ ], "title": "AutomationDraftTry" }, + "AutomationDrafted": { + "properties": { + "draft": { + "additionalProperties": true, + "type": "object", + "title": "Draft", + "description": "what POST /automations takes, switched off: name, triggers, if, graph" + }, + "dropped": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Dropped", + "description": "parts of the model's draft that didn't fit, and why" + }, + "unmapped": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Unmapped", + "description": "parts of the sentence it couldn't express (schedules, say)" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "draft" + ], + "title": "AutomationDrafted" + }, "AutomationPolicy": { "properties": { "node": { diff --git a/threat-model.json b/threat-model.json index d930130dc..6ec3f6194 100644 --- a/threat-model.json +++ b/threat-model.json @@ -582,7 +582,7 @@ }, { "entry_point": "automations", - "parameter": "/api/v1/automations (save, try, run now, decide), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on", + "parameter": "/api/v1/automations (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on", "attacker_controllable": "yes", "component": "assistant", "caller_must_enforce": "none — it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR); actions reaching outside Lens ask first; outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS); chains and bursts are bounded (P-AUTOMATION-LOOP)", diff --git a/threat-model.md b/threat-model.md index 7539acaef..11822785b 100644 --- a/threat-model.md +++ b/threat-model.md @@ -305,7 +305,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `.lens-control/inbox/upgrade` (host upgrade agent), `/metrics`, `.lens-control/prometheus-token` | `ref` (release tag or this install's branch; empty: the version running), with `https=`, `public=` (`0`/`1`) and `target=` (`full`/`lean`); `action=services` with `monitoring=on`/`off`; the scrape token | the inbox is writable by the containers (and local users); `/metrics` by anyone reaching the web app | x-env-config, credential | the agent installs only a release tag or the install's branch, changes only the HTTPS ports, the web app's network reach and the edition (any other value is dropped) and only toggles the `monitoring` profile, nothing else from the file, so a writer of the inbox can at most restart Lens with those choices or open it to the network; `/metrics` answers only the scrape token (constant-time compare); the token file is world-readable on the host and grants metrics only, which carry no personal content | *(documented, docs/updates.md, docs/prometheus.md "Privacy")* | | `PUT /api/v1/admin/install`, `POST /api/v1/settings/https/ports`, MCP `set_install_settings` | `public`, `target`, `open` | admin only (MCP: with the write scope) | x-env-config | admin is trusted; audited (`lens.install`, `https.ports`); the change reaches the machine only as an inbox request, which the agent checks again | *(documented, docs/updates.md "Setting the machine up again")* | | MCP `edit_entity`, `edit_topic`, `merge_topics`, `delete_topic`, `undo_topic_change`, `undo_graph_change`; assistant `propose_topic_change` | entity and topic ids, names, labels, definitions | yes, an agent with an editor's write-scope token; indirectly through content the model reads | data, x-prompt-injection | the curate permission in each namespace touched and a write-capable principal (the same route functions as the web app); the assistant's changes wait for the person's approval; merges and deletes are kept to undo, and audited | *(documented, docs/mcp.md "curate entities and topics as an editor does")* | -| `/api/v1/automations` (save, try, run now, decide), the MCP and assistant automation tools; the activities an automation runs on | triggers, `where` and If match trees, the event workflow graph (action params are Jinja templates), `policy`, `limits`, an activity id; the text of the activities themselves | yes: any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params | callback/code (declarative), data, x-prompt-injection | none; it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR), actions reaching outside Lens ask first, outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS), and chains and bursts are bounded (P-AUTOMATION-LOOP) | *(documented, docs/automations.md "Running automations")* | +| `/api/v1/automations` (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools; the activities an automation runs on | triggers, `where` and If match trees, the event workflow graph (action params are Jinja templates), `policy`, `limits`, an activity id; the text of the activities themselves | yes: any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params | callback/code (declarative), data, x-prompt-injection | none; it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR), actions reaching outside Lens ask first, outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS), and chains and bursts are bounded (P-AUTOMATION-LOOP) | *(documented, docs/automations.md "Running automations")* | | `ask/NAME.want` in a microsandbox's `/data/.lens/programs` (LibreOffice and Chromium on first use) | a program's name | writable by the Lens user inside the sandbox | x-env-config | the start script, as root, installs only `libreoffice` or `chromium`, each a fixed list of signed Debian packages, and answers in `got/`, which only root writes; it reads nothing from the file but its name, and only deletes any other name or link | *(documented, docs/microsandbox.md "LibreOffice and Chromium")* | **Stateful postconditions.** A refused or failed one-time secret check consumes nothing; a used one is deleted before diff --git a/threat-model.yaml b/threat-model.yaml index 075958dd1..8e3afce75 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -2,7 +2,7 @@ # Authority order: prose > yaml > json. Regenerate whenever the prose changes. schema: threat-model-sidecar/v2 project: lens -prose_version: "threat-model.md@sha256:42f18e17d459248818e8fe496acd9d5b1a96bfaf19d503aeb4a45f1449deecc9" +prose_version: "threat-model.md@sha256:d8bda21af9290dd0c018869ca7e0f3b4ce7d78cfcd77d304cb369cd716e42ec5" model_status: under-review triage_policy: strict confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8} @@ -526,7 +526,7 @@ entry_points: - id: "automations" component: assistant parameters: - - name: "/api/v1/automations (save, try, run now, decide), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on" + - name: "/api/v1/automations (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on" attacker_controllable: true x-condition: "any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params" control_kinds: [callback/code, data, x-prompt-injection]