diff --git a/CHANGELOG.md b/CHANGELOG.md index 44dffa475..08bbfb731 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,9 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version `automation_actions`, `save_automation`, `test_automation` and `run_automation`, with problems in plain words. The assistant saves them switched off and asks before switching one on or running it ([Agents](docs/automations.md#agents)). +- **Describe an automation in words.** `POST /api/v1/automations/draft` asks the model once to turn a sentence into + a draft automation to try and save, and says what didn't fit and what it couldn't express + ([Describe it in words](docs/automations.md#describe-it-in-words)). ## 0.13.0 October 16, 2026 {id="0.13.0"} diff --git a/docs/api.md b/docs/api.md index c61293e9c..6e8f3bec8 100644 --- a/docs/api.md +++ b/docs/api.md @@ -186,6 +186,7 @@ resources (admins). GET /api/v1/automations POST /api/v1/automations GET /api/v1/automations/actions +POST /api/v1/automations/draft GET /api/v1/automations/{id} PATCH /api/v1/automations/{id} DELETE /api/v1/automations/{id} @@ -206,7 +207,9 @@ Automations act for the person who made them in namespaces they can edit ([Savin automations](automations.md#saving-automations)). Admins see and change all of them; anyone else the ones that work only in namespaces they can edit (`?removed=true` lists deleted ones, to restore). A save answers `{id, hooks}`. `/policy` lets an action node run without asking (`auto`) only from a signed-in session in the web app. -`/actions` lists what action nodes can do. The list adds each one's actions waiting (`waiting`) and cost this month +`/actions` lists what action nodes can do. `/draft {"text": ...}` asks the model to turn a sentence into a draft to +try and save, saying what didn't fit (`dropped`) and what it couldn't express (`unmapped`) ([Describe it in +words](automations.md#describe-it-in-words)). The list adds each one's actions waiting (`waiting`) and cost this month (`cost_month`). `/test` tries a saved automation, and `POST /automations/test {"draft": ...}` an unsaved one, on an activity (`activity`) or a kind's sample (`kind`) without keeping or sending anything; `/run` runs it now (on `activity`, else the newest one its triggers listen for). `/runs` pages its runs (`before`, `limit`; `next` for the diff --git a/docs/automations.md b/docs/automations.md index 5df5635ba..40a810eb9 100644 --- a/docs/automations.md +++ b/docs/automations.md @@ -46,6 +46,7 @@ merged is called built. | Saving automations: checks, namespaces, switching on and off, delete and restore, the policy, the API (`app/domain/automations.py`) | built | | Running automations: following the stream, limits, budgets, runs as `automation.ran`, trying and running now, the API (`app/domain/automations.py`) | built | | Automations for agents: the MCP tools and the assistant's, switching on and running now by approval | built | +| Describing an automation in words: a draft from a sentence (`POST /api/v1/automations/draft`) | built | | Joins, the studio | planned | ## Contract @@ -979,6 +980,27 @@ person who made it (`run_as`), with their permissions, in namespaces they can ed `GET /api/v1/automations/actions` lists what an action node can do: each action's params (a JSON Schema), its effect and what it gives back. +### Describe it in words + +Built (`app/domain/automation_draft.py`). `POST /api/v1/automations/draft {"text": "when a recording is added to pods, +log its title", "spaces": [3]}` asks the model once to turn the sentence into a draft, with the kinds of activity and +their fields, the [match operators](#match-trees) and the [actions](#actions-and-intents) as its vocabulary. The answer +is `{draft, dropped, unmapped}`: + +- `draft` is what `POST /automations` takes, switched off: its name, triggers (on activities, a webhook, whose + address comes when it is saved, by hand, or in time: a schedule, a time, or a time in an activity such as a + meeting's start, in UTC unless the sentence names a zone), If and a graph with its actions in order (an action that may reach + outside Lens hangs off the one before it, as [the leaf rule](#the-event-workflow-scope) has it). +- `dropped` says what in the model's draft didn't fit and why: a trigger, its `where`, the If, an action that doesn't + exist or params that don't fit its schema. With no trigger left it runs when you run it; with no action left it has + no Then yet. +- `unmapped` lists what the sentence asked for that it couldn't express, such as "when both arrive" (joins are + planned). + +Nothing is saved: try the draft (`POST /automations/test`), change it, then save it. The call counts in the [activity +ledger](activity.md) for you and its namespaces, and isn't made once a budget they count for, or everything's, is +used up. + ### Agents Built. Over [MCP](mcp.md) an agent uses `automations` (list them, read one with its latest runs, or one run), @@ -1128,3 +1150,4 @@ held run until you pick. last status and the stream). - Automations that listen for every kind (or hundreds of automations): an index of triggers by kind and `where`, so a look reads each activity once whatever the number of automations. +- Drafts from a sentence: the person's own time zone when the sentence names none (today UTC). diff --git a/fastapi_backend/app/api/v1/routes/automations.py b/fastapi_backend/app/api/v1/routes/automations.py index 58059b03b..afd88c7d2 100644 --- a/fastapi_backend/app/api/v1/routes/automations.py +++ b/fastapi_backend/app/api/v1/routes/automations.py @@ -9,12 +9,14 @@ from fastapi import APIRouter, HTTPException, Query from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer, domain_errors -from app.domain import actions, auth, automations, events, store +from app.domain import actions, auth, automation_draft, automations, events, store from app.schemas.automations import ( ActionInfo, Automation, AutomationCatalog, AutomationCreate, + AutomationDrafted, + AutomationDraftRequest, AutomationDraftTry, AutomationPolicy, AutomationRun, @@ -210,6 +212,15 @@ def decide_held_run(run: int, body: HeldRunDecision, user: Writer, acl: Acl, db: return AutomationStarted(run=got, status=None if got is None else automations.get_run(db, got)["status"]) +@router.post("/draft") +def draft_automation(body: AutomationDraftRequest, user: Writer, db: Db, cfg: Cfg) -> AutomationDrafted: + """Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the + sentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What + doesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it.""" + with domain_errors(): + return AutomationDrafted(**automation_draft.draft(db, cfg, body.text, _who(user), body.spaces)) + + @router.post("/test") def test_draft(body: AutomationDraftTry, user: CurrentUser, acl: Acl, db: Db, cfg: Cfg) -> AutomationTried: """Try an automation before you save it: a dry run on an activity, or on a kind's sample (default: its first diff --git a/fastapi_backend/app/domain/automation_draft.py b/fastapi_backend/app/domain/automation_draft.py new file mode 100644 index 000000000..9d622c6ba --- /dev/null +++ b/fastapi_backend/app/domain/automation_draft.py @@ -0,0 +1,226 @@ +"""Describe an automation in words (docs/automations.md, Describe it in words): one model call turns a sentence such as +"when a new recording is added to pods, log its title" into a draft automation, with the kinds of activity, the +match operators and the actions as its vocabulary. The draft is checked piece by piece: what doesn't fit is dropped +and said, and what the model couldn't map is listed. Nothing is saved; the person (or agent) reads it, changes it and +saves it. + +The call counts in the activity ledger for the person and the namespaces it would work in, and isn't made once a +budget they count for (or everything's) is used up. +""" + +from __future__ import annotations + +import json + +from . import actions, activity, automation_spec, automations, budgets, events, llm, match, store + +TEXT_MAX = 2000 +SCHEMA = { + "type": "object", + "properties": { + "name": {"type": "string"}, + "description": {"type": "string"}, + "triggers": {"type": "array", "items": {"type": "object"}}, + "if": {"type": "object"}, + "actions": { + "type": "array", + "items": { + "type": "object", + "properties": {"action": {"type": "string"}, "params": {"type": "object"}}, + "required": ["action"], + }, + }, + "unmapped": {"type": "array", "items": {"type": "string"}}, + }, + "required": ["name", "triggers", "actions"], +} +SYSTEM = """You turn a person's sentence into an automation for Lens, an archive of their documents, recordings and +messages. An automation has triggers (when), an optional If (a test) and actions (then). Reply with JSON only: +{"name": short name, "description": one sentence, "triggers": [...], "if": a match tree or omit it, + "actions": [{"action": name, "params": {...}}], "unmapped": [each part of the sentence you couldn't express]}. + +Triggers: {"as": a short lowercase name, "on": "event", "kinds": [kinds below], "where": a match tree on the +activity (paths like object.name)}, {"as": a short lowercase name, "on": "webhook"} (an address another service or +device calls; Lens gives it when the automation is saved), or {"as": "manual", "on": "manual"}. TIMES + +A match tree is {"all": [...]}, {"any": [...]} or {"not": tree} around tests {"path": ..., "op": ..., "value": ...}; +op is one of OPS. The If's paths start with the run context's keys (event, events, trigger, automation, namespace, +user, now), like event.object.name. + +An action's params may hold templates over the run context, like "New: {{ event.object.name }}". Use only the kinds, +actions and params listed; anything else goes in unmapped. Never invent values the sentence doesn't give.""" + + +TIMES = { + "schedule": '{"as": a short lowercase name, "on": "schedule", "cron": five-field cron like "0 7 * * 1-5", "timezone": an ' + 'IANA zone like "Europe/London"}', + "at": '{"as": a short lowercase name, "on": "at", "at": a local time like "2026-10-20T09:00", "timezone": an IANA zone}', + "relative": '{"as": a short lowercase name, "on": "relative", "kinds": [kinds below], "where": a match tree, "field": ' + 'the time in the activity it counts from, like object.startTime, "offset": like "-30m" (before) or "2h" (after)}', +} + + +def _times(): + """What the model is told about triggers in time: the ones built, and the rest put in unmapped.""" + have = [on for on in TIMES if not _soon(on)] + if not have: + return "Schedules and times aren't available yet: put those parts in unmapped." + said = "Triggers in time: " + ", ".join(TIMES[on] for on in have) + ". With no time zone in the sentence, use UTC." + if len(have) < len(TIMES): + said += f" Triggers on {' or '.join(on for on in TIMES if on not in have)} aren't available yet: put those in unmapped." + return said + + +def _fields(schema, prefix="", depth=0): + """The dotted paths of an activity's fields with their titles, to name in tests.""" + out = [] + for k, v in ((schema or {}).get("properties") or {}).items(): + path = f"{prefix}{k}" + if v.get("type") == "object" and depth < 1: + out += _fields(v, path + ".", depth + 1) + elif v.get("title"): + out.append(f"{path} ({v['title']})") + return out + + +def vocabulary(): + """What the model may use: the kinds of activity with their fields, the operators and the actions.""" + kinds = [ + {"kind": k["name"], "label": k["label"], "fields": _fields(k.get("schema") or {})} for k in events.kinds() if not k.get("noisy") + ] + acts = [ + { + "action": a["name"], + "label": a["label"], + "description": a.get("description") or "", + "params": a["params"], + "required": a.get("required") or [], + } + for a in actions.catalog() + ] + return {"kinds": kinds, "ops": list(match.WORDS), "actions": acts} + + +def _soon(on): + """Whether a trigger needs a part of the studio that isn't here yet (automations.SOON).""" + need = automations.SOON.get(on) if isinstance(on, str) else None + return need is not None and automations.provided(need[0], need[1]) is None + + +def _triggers(raw, dropped, unmapped): + out, seen = [], set() + for n, t in enumerate(raw if isinstance(raw, list) else [], 1): + if isinstance(t, dict) and _soon(t.get("on")): + unmapped.append(f"a trigger on {t['on']} (coming soon)") + continue + try: + got = automation_spec.check_trigger(t, n) + except ValueError as e: + where = isinstance(t, dict) and t.pop("where", None) + try: + got = automation_spec.check_trigger(t, n) if where else None + except ValueError: + got = None + if got is None: + dropped.append(str(e)) + continue + dropped.append(f"{e} (its where was left out)") + if got["as"] in seen: + continue + seen.add(got["as"]) + out.append(got) + if not out: + out = [{"as": "manual", "on": "manual"}] + dropped.append("no trigger fit, so it runs when you run it") + return out[: automation_spec.MAX_TRIGGERS] + + +def _actions(raw, dropped): + out = [] + for n, a in enumerate(raw if isinstance(raw, list) else [], 1): + name = a.get("action") if isinstance(a, dict) else None + try: + spec = actions.get(str(name)) + except KeyError: + dropped.append(f"action {n}: there's no action {name}") + continue + try: + params = actions.check_params(spec, a.get("params") or {}) + except ValueError as e: + dropped.append(f"action {n}: {e}") + continue + out.append((spec, params)) + return out + + +def _graph(acts): + """The actions in order after the input node; one that reaches outside Lens (or may) is a leaf off the one + before it, as the leaf rule has it, and the rest go on from there.""" + nodes, edges, last = [{"id": "input", "type": "input", "config": {}}], [], "input" + for n, (spec, params) in enumerate(acts, 1): + nid = f"a{n}" + nodes.append({"id": nid, "type": "action", "config": {"action": spec.name, "params": params}}) + edges.append({"source": last, "target": nid}) + if isinstance(spec.effect, str) and spec.effect != "out": # one worked out from its params may reach out + last = nid + return {"nodes": nodes, "edges": edges} + + +def draft(db, cfg, text, user, spaces=None): + """{draft, dropped, unmapped}: `draft` is what POST /automations takes (switched off), `dropped` the parts of the + model's reply that didn't fit and why, `unmapped` the parts of the sentence it couldn't express.""" + text = " ".join(str(text or "").split()) + if not text: + raise ValueError("say what the automation should do") + if len(text) > TEXT_MAX: + raise ValueError(f"say it in at most {TEXT_MAX} characters") + account = automations._account(db, (user or {}).get("id")) + if not account: + raise ValueError("an automation acts for a person; sign in") + can = automations.editable(db, account) + if not can: + raise ValueError(f"{account.get('name') or account['email']} can't edit any namespace, so an automation couldn't work anywhere") + spaces = [int(s) for s in spaces] if spaces else None + if spaces and not set(spaces) <= can: + raise ValueError("you can make automations only in namespaces you can edit") + if not llm.configured(cfg): + raise ValueError("no language model is configured (Settings → LLM provider)") + refs = [f"account:{account['id']}", *(f"space:{s}" for s in spaces or [])] + why = budgets.used_up(db, refs) + if why: + raise ValueError(f"over budget: {why}") + vocab = vocabulary() + system = SYSTEM.replace("OPS", ", ".join(vocab["ops"])).replace("TIMES", _times()) + ask = ( + f"Kinds of activity and their fields:\n{json.dumps(vocab['kinds'])}\n\nActions:\n{json.dumps(vocab['actions'])}\n\nSentence: {text}" + ) + with activity.scope(db, *refs, cfg=cfg): + try: + reply = llm.json_out(cfg, system, ask, SCHEMA) + except llm.LLMError as e: + raise ValueError(f"the model couldn't draft it: {e}") from None + dropped, unmapped = [], [str(u)[:300] for u in reply.get("unmapped") or [] if str(u).strip()][:20] + out = { + "name": " ".join(str(reply.get("name") or "").split())[:200] or "New automation", + "description": " ".join(str(reply.get("description") or "").split())[:2000] or None, + "enabled": False, + "triggers": _triggers(reply.get("triggers"), dropped, unmapped), + } + if spaces: + out["spaces"] = spaces + if reply.get("if"): + try: + out["if"] = automation_spec.check_if(reply["if"]) + except ValueError as e: + dropped.append(f"the If: {e}") + acts = _actions(reply.get("actions"), dropped) + if acts: + out["graph"] = _graph(acts) + else: + dropped.append("no action fit; add one before saving") + if acts: + try: + automations._check(db, dict(out), None, account) + except ValueError as e: + dropped.append(f"as a whole: {e}") + return {"draft": store.clean(out), "dropped": dropped, "unmapped": unmapped} diff --git a/fastapi_backend/app/schemas/automations.py b/fastapi_backend/app/schemas/automations.py index 9d0cbbed7..234bb6532 100644 --- a/fastapi_backend/app/schemas/automations.py +++ b/fastapi_backend/app/schemas/automations.py @@ -206,3 +206,16 @@ class AutomationStarted(ResponseModel): class AutomationUpcoming(ResponseModel): times: list[Any] = Field(default_factory=list, description="when it runs next, from its schedules and timers (coming soon)") + + +class AutomationDraftRequest(RequestModel): + text: str = Field( + max_length=2000, description="what it should do, in a sentence, like: when a recording is added to pods, log its title" + ) + spaces: list[int] | None = Field(None, description="namespace ids it would work in; default: each one you can edit") + + +class AutomationDrafted(ResponseModel): + draft: dict[str, Any] = Field(description="what POST /automations takes, switched off: name, triggers, if, graph") + dropped: list[str] = Field(default_factory=list, description="parts of the model's draft that didn't fit, and why") + unmapped: list[str] = Field(default_factory=list, description="parts of the sentence it couldn't express (schedules, say)") diff --git a/fastapi_backend/tests/api/test_automations.py b/fastapi_backend/tests/api/test_automations.py index 9e45a2110..30e28be8b 100644 --- a/fastapi_backend/tests/api/test_automations.py +++ b/fastapi_backend/tests/api/test_automations.py @@ -339,3 +339,22 @@ def test_a_run_held_over_budget_waits_for_the_persons_pick(client, db, cfg, env) assert client.get(f"{URL}/runs/{held['run']}", headers=he).json()["status"] == "released" r = client.post(f"{URL}/runs/{held['run']}/decide", headers=he, json={"run": False}) assert r.status_code == 400 and "isn't waiting on a budget" in r.json()["detail"] + + +def test_an_automation_is_drafted_from_a_sentence(client, db, env, monkeypatch): + from app.domain import automation_draft + + reply = {"name": "Say new", "triggers": NEW, "actions": [{"action": "log_line", "params": {"text": "New: {{ event.object.name }}"}}]} + monkeypatch.setattr(automation_draft.llm, "configured", lambda cfg: True) + monkeypatch.setattr(automation_draft.llm, "json_out", lambda cfg, system, user, schema, model=None: reply) + r = client.post(f"{URL}/draft", headers=env["he"], json={"text": "when something new comes in, log its name"}) + assert r.status_code == 200, r.text + got = r.json() + assert got["draft"]["enabled"] is False and got["draft"]["triggers"] == [{**NEW[0], "from_automations": False}] + assert got["dropped"] == [] and got["unmapped"] == [] + # it tries and saves as it is; drafting saved nothing + assert db.rows("SELECT id FROM automation") == [] + assert client.post(f"{URL}/test", headers=env["he"], json={"draft": got["draft"]}).json()["status"] == "done" + assert client.post(URL, headers=env["he"], json=got["draft"]).status_code == 200 + assert "can't edit any namespace" in problem(client, env["hv"], {"text": "log it"}, f"{URL}/draft") + assert client.post(f"{URL}/draft", headers=env["he"], json={"text": "x" * 2001}).status_code == 422 diff --git a/fastapi_backend/tests/domain/test_automation_draft.py b/fastapi_backend/tests/domain/test_automation_draft.py new file mode 100644 index 000000000..4ad1186c6 --- /dev/null +++ b/fastapi_backend/tests/domain/test_automation_draft.py @@ -0,0 +1,140 @@ +"""Describing an automation in words (docs/automations.md, Describe it in words): one model call, with the kinds, the +operators and the actions as its vocabulary; what doesn't fit is dropped and said; nothing is saved; refused over +budget; and the call counts in the activity ledger.""" + +from __future__ import annotations + +import pytest + +from app.domain import activity, auth, automation_draft, automations, budgets, store +from tests.helpers import make_user, without + + +@pytest.fixture +def env(db, cfg, monkeypatch): + make_user(db, "ed@x.io", "editor password 1", roles={"pods": "editor"}) + make_user(db, "vi@x.io", "viewer password 1", roles={"pods": "viewer"}) + cfg = {**cfg, "llm": {**cfg["llm"], "base_url": "http://model.invalid/v1", "model": "m"}} + asked = [] + + def reply(answer): + def fake(cfg, system, user, schema, model=None): + asked.append({"system": system, "user": user}) + activity.record("out", "model.chat", [], cfg, db, model="m", cost_usd=0.01) + return answer + + monkeypatch.setattr(automation_draft.llm, "json_out", fake) + + return { + "cfg": cfg, + "ed": {"id": auth.find_account(db, "ed@x.io")["id"]}, + "vi": {"id": auth.find_account(db, "vi@x.io")["id"]}, + "pods": store.ns_id(db, "pods"), + "asked": asked, + "reply": reply, + } + + +def test_a_sentence_becomes_a_draft_that_saves(db, env): + cfg = env["cfg"] + env["reply"]( + { + "name": "Log new pods", + "description": "Logs each new recording's title", + "triggers": [ + { + "as": "new", + "on": "event", + "kinds": ["asset.added"], + "where": {"all": [{"path": "object.media", "op": "equals", "value": "audio"}]}, + } + ], + "if": {"all": [{"path": "event.object.name", "op": "exists"}]}, + "actions": [ + {"action": "notify", "params": {"to": "me", "text": "New: {{ event.object.name }}"}}, + {"action": "log_line", "params": {"text": "Logged {{ event.object.name }}"}}, + ], + "unmapped": ["only on weekdays"], + } + ) + got = automation_draft.draft(db, cfg, "When an audio recording is added, tell me and log it, on weekdays", env["ed"]) + d = got["draft"] + assert got["dropped"] == [] and got["unmapped"] == ["only on weekdays"] + assert d["name"] == "Log new pods" and d["enabled"] is False and d["triggers"][0]["where"]["all"][0]["value"] == "audio" + # an action that may reach outside Lens is a leaf off the one before it; the rest go on from there + assert d["graph"]["edges"] == [{"source": "input", "target": "a1"}, {"source": "input", "target": "a2"}] + # the model was given the vocabulary + said = env["asked"][0] + assert "asset.added" in said["user"] and "log_line" in said["user"] and "contains" in said["system"] + assert "object.name (Title)" in said["user"] + # nothing was saved, and the call counted for the person + assert db.rows("SELECT id FROM automation") == [] + assert activity.costs(db, [f"account:{env['ed']['id']}"])[f"account:{env['ed']['id']}"]["calls"] == 1 + # it saves as it is + aid = automations.create(db, cfg, d, {**env["ed"]})["id"] + assert automations.get(db, aid)["enabled"] is False + + +def test_triggers_in_time_are_drafted_once_the_timers_are_there(db, env, monkeypatch): + cfg = env["cfg"] + morning = {"as": "morning", "on": "schedule", "cron": "0 7 * * 1-5", "timezone": "Europe/London"} + env["reply"]({"name": "Mornings", "triggers": [morning], "actions": [{"action": "log_line", "params": {"text": "Morning"}}]}) + got = automation_draft.draft(db, cfg, "every weekday at 7 log morning", env["ed"]) + assert got["draft"]["triggers"] == [morning] and got["dropped"] == [] and got["unmapped"] == [] + assert '"on": "schedule"' in env["asked"][0]["system"] and '"on": "relative"' in env["asked"][0]["system"] + # without them, the model is told to leave times out + without(monkeypatch, "timers") + automation_draft.draft(db, cfg, "every weekday at 7 log morning", env["ed"]) + assert '"on": "schedule"' not in env["asked"][1]["system"] and "aren't available yet" in env["asked"][1]["system"] + + +def test_what_doesnt_fit_is_dropped_and_said(db, env, monkeypatch): + without(monkeypatch, "timers") # schedules come with their own module + cfg = env["cfg"] + env["reply"]( + { + "name": "Mixed", + "triggers": [ + {"as": "morning", "on": "schedule", "cron": "0 7 * * *"}, + {"as": "door", "on": "webhook"}, + {"as": "new", "on": "event", "kinds": ["asset.added"], "where": {"all": [{"path": "x", "op": "matches"}]}}, + {"as": "odd", "on": "event", "kinds": ["no.such"]}, + ], + "if": {"all": [{"path": "nowhere.at.all", "op": "exists"}]}, + "actions": [ + {"action": "launch_rockets"}, + {"action": "log_line", "params": {"text": 5}}, + {"action": "log_line", "params": {"text": "ok"}}, + ], + } + ) + got = automation_draft.draft(db, cfg, "every morning and when something comes in, launch rockets", env["ed"]) + d = got["draft"] + assert got["unmapped"] == ["a trigger on schedule (coming soon)"] + assert [t["as"] for t in d["triggers"]] == ["door", "new"] and "where" not in d["triggers"][1] + assert any("its where was left out" in x for x in got["dropped"]) and any("no.such" in x for x in got["dropped"]) + assert any(x.startswith("the If:") for x in got["dropped"]) and "if" not in d + assert any("no action launch_rockets" in x for x in got["dropped"]) and any(x.startswith("action 2: ") for x in got["dropped"]) + assert [n["config"]["action"] for n in d["graph"]["nodes"][1:]] == ["log_line"] + # with nothing left: run by hand, and no Then + env["reply"]({"name": "", "triggers": [], "actions": []}) + got = automation_draft.draft(db, cfg, "do something", env["ed"]) + assert got["draft"]["triggers"] == [{"as": "manual", "on": "manual"}] and "graph" not in got["draft"] + assert got["draft"]["name"] == "New automation" and "no action fit; add one before saving" in got["dropped"] + + +def test_drafting_is_refused_when_it_cant_or_mustnt_ask(db, env, cfg): + env["reply"]({"name": "x", "triggers": [], "actions": []}) + with pytest.raises(ValueError, match="say what"): + automation_draft.draft(db, env["cfg"], " ", env["ed"]) + with pytest.raises(ValueError, match="can't edit any namespace"): + automation_draft.draft(db, env["cfg"], "log new things", env["vi"]) + with pytest.raises(ValueError, match="only in namespaces you can edit"): + automation_draft.draft(db, env["cfg"], "log new things", env["ed"], [store.ns_id(db, "calls")]) + with pytest.raises(ValueError, match="no language model"): + automation_draft.draft(db, {**cfg, "llm": {**cfg["llm"], "base_url": None}}, "log new things", env["ed"]) + activity.record("out", "model.chat", [f"space:{env['pods']}"], db=db, model="m", cost_usd=0.6) + budgets.put(db, f"space:{env['pods']}", usd=0.5) + with pytest.raises(ValueError, match="over budget"): + automation_draft.draft(db, env["cfg"], "log new things", env["ed"], [env["pods"]]) + assert env["asked"] == [] diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index d734dd408..b5db04b3b 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -186,6 +186,8 @@ export type { Automation, AutomationCatalog, AutomationCreate, + AutomationDrafted, + AutomationDraftRequest, AutomationDraftTry, AutomationPolicy, AutomationRun, @@ -892,6 +894,11 @@ export type { DownloadFileErrors, DownloadFileResponse, DownloadFileResponses, + DraftAutomationData, + DraftAutomationError, + DraftAutomationErrors, + DraftAutomationResponse, + DraftAutomationResponses, DropPasskeysData, DropPasskeysError, DropPasskeysErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index b93cd1e5b..552691e6a 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -421,6 +421,9 @@ import type { DownloadFileData, DownloadFileErrors, DownloadFileResponses, + DraftAutomationData, + DraftAutomationErrors, + DraftAutomationResponses, DropPasskeysData, DropPasskeysErrors, DropPasskeysResponses, @@ -4184,6 +4187,26 @@ export class Automations { }); } + /** + * Draft Automation + * + * Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the + * sentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What + * doesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it. + */ + public static draftAutomation( + options: Options, + ): RequestResult { + return (options.client ?? client).post({ + url: "/api/v1/automations/draft", + ...options, + headers: { + "Content-Type": "application/json", + ...options.headers, + }, + }); + } + /** * Test Draft * diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 9f85dbefc..06ed72de5 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -1678,6 +1678,24 @@ export type AutomationCreate = { } | null; }; +/** + * AutomationDraftRequest + */ +export type AutomationDraftRequest = { + /** + * Text + * + * what it should do, in a sentence, like: when a recording is added to pods, log its title + */ + text: string; + /** + * Spaces + * + * namespace ids it would work in; default: each one you can edit + */ + spaces?: Array | null; +}; + /** * AutomationDraftTry */ @@ -1706,6 +1724,33 @@ export type AutomationDraftTry = { draft: AutomationCreate; }; +/** + * AutomationDrafted + */ +export type AutomationDrafted = { + /** + * Draft + * + * what POST /automations takes, switched off: name, triggers, if, graph + */ + draft: { + [key: string]: unknown; + }; + /** + * Dropped + * + * parts of the model's draft that didn't fit, and why + */ + dropped?: Array; + /** + * Unmapped + * + * parts of the sentence it couldn't express (schedules, say) + */ + unmapped?: Array; + [key: string]: unknown; +}; + /** * AutomationPolicy */ @@ -26808,6 +26853,31 @@ export type DecideHeldRunResponses = { export type DecideHeldRunResponse = DecideHeldRunResponses[keyof DecideHeldRunResponses]; +export type DraftAutomationData = { + body: AutomationDraftRequest; + path?: never; + query?: never; + url: "/api/v1/automations/draft"; +}; + +export type DraftAutomationErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type DraftAutomationError = DraftAutomationErrors[keyof DraftAutomationErrors]; + +export type DraftAutomationResponses = { + /** + * Successful Response + */ + 200: AutomationDrafted; +}; + +export type DraftAutomationResponse = DraftAutomationResponses[keyof DraftAutomationResponses]; + export type TestDraftData = { body: AutomationDraftTry; path?: never; diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 7eacbd785..285f91af5 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -6103,6 +6103,48 @@ } } }, + "/api/v1/automations/draft": { + "post": { + "tags": [ + "automations" + ], + "summary": "Draft Automation", + "description": "Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the\nsentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What\ndoesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it.", + "operationId": "draft_automation", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AutomationDraftRequest" + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/AutomationDrafted" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/automations/test": { "post": { "tags": [ @@ -32919,6 +32961,37 @@ ], "title": "AutomationCreate" }, + "AutomationDraftRequest": { + "properties": { + "text": { + "type": "string", + "maxLength": 2000, + "title": "Text", + "description": "what it should do, in a sentence, like: when a recording is added to pods, log its title" + }, + "spaces": { + "anyOf": [ + { + "items": { + "type": "integer" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Spaces", + "description": "namespace ids it would work in; default: each one you can edit" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "text" + ], + "title": "AutomationDraftRequest" + }, "AutomationDraftTry": { "properties": { "activity": { @@ -32969,6 +33042,38 @@ ], "title": "AutomationDraftTry" }, + "AutomationDrafted": { + "properties": { + "draft": { + "additionalProperties": true, + "type": "object", + "title": "Draft", + "description": "what POST /automations takes, switched off: name, triggers, if, graph" + }, + "dropped": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Dropped", + "description": "parts of the model's draft that didn't fit, and why" + }, + "unmapped": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Unmapped", + "description": "parts of the sentence it couldn't express (schedules, say)" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "draft" + ], + "title": "AutomationDrafted" + }, "AutomationPolicy": { "properties": { "node": { diff --git a/threat-model.json b/threat-model.json index d930130dc..6ec3f6194 100644 --- a/threat-model.json +++ b/threat-model.json @@ -582,7 +582,7 @@ }, { "entry_point": "automations", - "parameter": "/api/v1/automations (save, try, run now, decide), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on", + "parameter": "/api/v1/automations (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on", "attacker_controllable": "yes", "component": "assistant", "caller_must_enforce": "none — it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR); actions reaching outside Lens ask first; outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS); chains and bursts are bounded (P-AUTOMATION-LOOP)", diff --git a/threat-model.md b/threat-model.md index 7539acaef..11822785b 100644 --- a/threat-model.md +++ b/threat-model.md @@ -305,7 +305,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `.lens-control/inbox/upgrade` (host upgrade agent), `/metrics`, `.lens-control/prometheus-token` | `ref` (release tag or this install's branch; empty: the version running), with `https=`, `public=` (`0`/`1`) and `target=` (`full`/`lean`); `action=services` with `monitoring=on`/`off`; the scrape token | the inbox is writable by the containers (and local users); `/metrics` by anyone reaching the web app | x-env-config, credential | the agent installs only a release tag or the install's branch, changes only the HTTPS ports, the web app's network reach and the edition (any other value is dropped) and only toggles the `monitoring` profile, nothing else from the file, so a writer of the inbox can at most restart Lens with those choices or open it to the network; `/metrics` answers only the scrape token (constant-time compare); the token file is world-readable on the host and grants metrics only, which carry no personal content | *(documented, docs/updates.md, docs/prometheus.md "Privacy")* | | `PUT /api/v1/admin/install`, `POST /api/v1/settings/https/ports`, MCP `set_install_settings` | `public`, `target`, `open` | admin only (MCP: with the write scope) | x-env-config | admin is trusted; audited (`lens.install`, `https.ports`); the change reaches the machine only as an inbox request, which the agent checks again | *(documented, docs/updates.md "Setting the machine up again")* | | MCP `edit_entity`, `edit_topic`, `merge_topics`, `delete_topic`, `undo_topic_change`, `undo_graph_change`; assistant `propose_topic_change` | entity and topic ids, names, labels, definitions | yes, an agent with an editor's write-scope token; indirectly through content the model reads | data, x-prompt-injection | the curate permission in each namespace touched and a write-capable principal (the same route functions as the web app); the assistant's changes wait for the person's approval; merges and deletes are kept to undo, and audited | *(documented, docs/mcp.md "curate entities and topics as an editor does")* | -| `/api/v1/automations` (save, try, run now, decide), the MCP and assistant automation tools; the activities an automation runs on | triggers, `where` and If match trees, the event workflow graph (action params are Jinja templates), `policy`, `limits`, an activity id; the text of the activities themselves | yes: any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params | callback/code (declarative), data, x-prompt-injection | none; it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR), actions reaching outside Lens ask first, outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS), and chains and bursts are bounded (P-AUTOMATION-LOOP) | *(documented, docs/automations.md "Running automations")* | +| `/api/v1/automations` (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools; the activities an automation runs on | triggers, `where` and If match trees, the event workflow graph (action params are Jinja templates), `policy`, `limits`, an activity id; the text of the activities themselves | yes: any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params | callback/code (declarative), data, x-prompt-injection | none; it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR), actions reaching outside Lens ask first, outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS), and chains and bursts are bounded (P-AUTOMATION-LOOP) | *(documented, docs/automations.md "Running automations")* | | `ask/NAME.want` in a microsandbox's `/data/.lens/programs` (LibreOffice and Chromium on first use) | a program's name | writable by the Lens user inside the sandbox | x-env-config | the start script, as root, installs only `libreoffice` or `chromium`, each a fixed list of signed Debian packages, and answers in `got/`, which only root writes; it reads nothing from the file but its name, and only deletes any other name or link | *(documented, docs/microsandbox.md "LibreOffice and Chromium")* | **Stateful postconditions.** A refused or failed one-time secret check consumes nothing; a used one is deleted before diff --git a/threat-model.yaml b/threat-model.yaml index 075958dd1..8e3afce75 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -2,7 +2,7 @@ # Authority order: prose > yaml > json. Regenerate whenever the prose changes. schema: threat-model-sidecar/v2 project: lens -prose_version: "threat-model.md@sha256:42f18e17d459248818e8fe496acd9d5b1a96bfaf19d503aeb4a45f1449deecc9" +prose_version: "threat-model.md@sha256:d8bda21af9290dd0c018869ca7e0f3b4ce7d78cfcd77d304cb369cd716e42ec5" model_status: under-review triage_policy: strict confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8} @@ -526,7 +526,7 @@ entry_points: - id: "automations" component: assistant parameters: - - name: "/api/v1/automations (save, try, run now, decide), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on" + - name: "/api/v1/automations (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools: triggers, where and If match trees, the event workflow graph (action params are Jinja templates), policy, limits, an activity id; the text of the activities an automation runs on" attacker_controllable: true x-condition: "any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params" control_kinds: [callback/code, data, x-prompt-injection]