Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
`automation_actions`, `save_automation`, `test_automation` and `run_automation`, with problems in plain words. The
assistant saves them switched off and asks before switching one on or running it
([Agents](docs/automations.md#agents)).
- **Describe an automation in words.** `POST /api/v1/automations/draft` asks the model once to turn a sentence into
a draft automation to try and save, and says what didn't fit and what it couldn't express
([Describe it in words](docs/automations.md#describe-it-in-words)).

## 0.13.0 <small>October 16, 2026</small> {id="0.13.0"}

Expand Down
5 changes: 4 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ resources (admins).
GET /api/v1/automations
POST /api/v1/automations
GET /api/v1/automations/actions
POST /api/v1/automations/draft
GET /api/v1/automations/{id}
PATCH /api/v1/automations/{id}
DELETE /api/v1/automations/{id}
Expand All @@ -206,7 +207,9 @@ Automations act for the person who made them in namespaces they can edit ([Savin
automations](automations.md#saving-automations)). Admins see and change all of them; anyone else the ones that work
only in namespaces they can edit (`?removed=true` lists deleted ones, to restore). A save answers `{id, hooks}`.
`/policy` lets an action node run without asking (`auto`) only from a signed-in session in the web app.
`/actions` lists what action nodes can do. The list adds each one's actions waiting (`waiting`) and cost this month
`/actions` lists what action nodes can do. `/draft {"text": ...}` asks the model to turn a sentence into a draft to
try and save, saying what didn't fit (`dropped`) and what it couldn't express (`unmapped`) ([Describe it in
words](automations.md#describe-it-in-words)). The list adds each one's actions waiting (`waiting`) and cost this month
(`cost_month`). `/test` tries a saved automation, and `POST /automations/test {"draft": ...}` an unsaved one, on an
activity (`activity`) or a kind's sample (`kind`) without keeping or sending anything; `/run` runs it now (on
`activity`, else the newest one its triggers listen for). `/runs` pages its runs (`before`, `limit`; `next` for the
Expand Down
23 changes: 23 additions & 0 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ merged is called built.
| Saving automations: checks, namespaces, switching on and off, delete and restore, the policy, the API (`app/domain/automations.py`) | built |
| Running automations: following the stream, limits, budgets, runs as `automation.ran`, trying and running now, the API (`app/domain/automations.py`) | built |
| Automations for agents: the MCP tools and the assistant's, switching on and running now by approval | built |
| Describing an automation in words: a draft from a sentence (`POST /api/v1/automations/draft`) | built |
| Joins, the studio | planned |

## Contract
Expand Down Expand Up @@ -979,6 +980,27 @@ person who made it (`run_as`), with their permissions, in namespaces they can ed
`GET /api/v1/automations/actions` lists what an action node can do: each action's params (a JSON Schema), its effect
and what it gives back.

### Describe it in words

Built (`app/domain/automation_draft.py`). `POST /api/v1/automations/draft {"text": "when a recording is added to pods,
log its title", "spaces": [3]}` asks the model once to turn the sentence into a draft, with the kinds of activity and
their fields, the [match operators](#match-trees) and the [actions](#actions-and-intents) as its vocabulary. The answer
is `{draft, dropped, unmapped}`:

- `draft` is what `POST /automations` takes, switched off: its name, triggers (on activities, a webhook, whose
address comes when it is saved, by hand, or in time: a schedule, a time, or a time in an activity such as a
meeting's start, in UTC unless the sentence names a zone), If and a graph with its actions in order (an action that may reach
outside Lens hangs off the one before it, as [the leaf rule](#the-event-workflow-scope) has it).
- `dropped` says what in the model's draft didn't fit and why: a trigger, its `where`, the If, an action that doesn't
exist or params that don't fit its schema. With no trigger left it runs when you run it; with no action left it has
no Then yet.
- `unmapped` lists what the sentence asked for that it couldn't express, such as "when both arrive" (joins are
planned).

Nothing is saved: try the draft (`POST /automations/test`), change it, then save it. The call counts in the [activity
ledger](activity.md) for you and its namespaces, and isn't made once a budget they count for, or everything's, is
used up.

### Agents

Built. Over [MCP](mcp.md) an agent uses `automations` (list them, read one with its latest runs, or one run),
Expand Down Expand Up @@ -1128,3 +1150,4 @@ held run until you pick.
last status and the stream).
- Automations that listen for every kind (or hundreds of automations): an index of triggers by kind and `where`, so a
look reads each activity once whatever the number of automations.
- Drafts from a sentence: the person's own time zone when the sentence names none (today UTC).
13 changes: 12 additions & 1 deletion fastapi_backend/app/api/v1/routes/automations.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,14 @@
from fastapi import APIRouter, HTTPException, Query

from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer, domain_errors
from app.domain import actions, auth, automations, events, store
from app.domain import actions, auth, automation_draft, automations, events, store
from app.schemas.automations import (
ActionInfo,
Automation,
AutomationCatalog,
AutomationCreate,
AutomationDrafted,
AutomationDraftRequest,
AutomationDraftTry,
AutomationPolicy,
AutomationRun,
Expand Down Expand Up @@ -210,6 +212,15 @@ def decide_held_run(run: int, body: HeldRunDecision, user: Writer, acl: Acl, db:
return AutomationStarted(run=got, status=None if got is None else automations.get_run(db, got)["status"])


@router.post("/draft")
def draft_automation(body: AutomationDraftRequest, user: Writer, db: Db, cfg: Cfg) -> AutomationDrafted:
"""Describe an automation in words: the model (one call, in the activity ledger, refused over budget) turns the
sentence into a draft with the kinds of activity, the match operators and the actions as its vocabulary. What
doesn't fit is dropped and said; nothing is saved. Try it with POST /automations/test, then save it."""
with domain_errors():
return AutomationDrafted(**automation_draft.draft(db, cfg, body.text, _who(user), body.spaces))


@router.post("/test")
def test_draft(body: AutomationDraftTry, user: CurrentUser, acl: Acl, db: Db, cfg: Cfg) -> AutomationTried:
"""Try an automation before you save it: a dry run on an activity, or on a kind's sample (default: its first
Expand Down
226 changes: 226 additions & 0 deletions fastapi_backend/app/domain/automation_draft.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
"""Describe an automation in words (docs/automations.md, Describe it in words): one model call turns a sentence such as
"when a new recording is added to pods, log its title" into a draft automation, with the kinds of activity, the
match operators and the actions as its vocabulary. The draft is checked piece by piece: what doesn't fit is dropped
and said, and what the model couldn't map is listed. Nothing is saved; the person (or agent) reads it, changes it and
saves it.

The call counts in the activity ledger for the person and the namespaces it would work in, and isn't made once a
budget they count for (or everything's) is used up.
"""

from __future__ import annotations

import json

from . import actions, activity, automation_spec, automations, budgets, events, llm, match, store

TEXT_MAX = 2000
SCHEMA = {
"type": "object",
"properties": {
"name": {"type": "string"},
"description": {"type": "string"},
"triggers": {"type": "array", "items": {"type": "object"}},
"if": {"type": "object"},
"actions": {
"type": "array",
"items": {
"type": "object",
"properties": {"action": {"type": "string"}, "params": {"type": "object"}},
"required": ["action"],
},
},
"unmapped": {"type": "array", "items": {"type": "string"}},
},
"required": ["name", "triggers", "actions"],
}
SYSTEM = """You turn a person's sentence into an automation for Lens, an archive of their documents, recordings and
messages. An automation has triggers (when), an optional If (a test) and actions (then). Reply with JSON only:
{"name": short name, "description": one sentence, "triggers": [...], "if": a match tree or omit it,
"actions": [{"action": name, "params": {...}}], "unmapped": [each part of the sentence you couldn't express]}.

Triggers: {"as": a short lowercase name, "on": "event", "kinds": [kinds below], "where": a match tree on the
activity (paths like object.name)}, {"as": a short lowercase name, "on": "webhook"} (an address another service or
device calls; Lens gives it when the automation is saved), or {"as": "manual", "on": "manual"}. TIMES

A match tree is {"all": [...]}, {"any": [...]} or {"not": tree} around tests {"path": ..., "op": ..., "value": ...};
op is one of OPS. The If's paths start with the run context's keys (event, events, trigger, automation, namespace,
user, now), like event.object.name.

An action's params may hold templates over the run context, like "New: {{ event.object.name }}". Use only the kinds,
actions and params listed; anything else goes in unmapped. Never invent values the sentence doesn't give."""


TIMES = {
"schedule": '{"as": a short lowercase name, "on": "schedule", "cron": five-field cron like "0 7 * * 1-5", "timezone": an '
'IANA zone like "Europe/London"}',
"at": '{"as": a short lowercase name, "on": "at", "at": a local time like "2026-10-20T09:00", "timezone": an IANA zone}',
"relative": '{"as": a short lowercase name, "on": "relative", "kinds": [kinds below], "where": a match tree, "field": '
'the time in the activity it counts from, like object.startTime, "offset": like "-30m" (before) or "2h" (after)}',
}


def _times():
"""What the model is told about triggers in time: the ones built, and the rest put in unmapped."""
have = [on for on in TIMES if not _soon(on)]
if not have:
return "Schedules and times aren't available yet: put those parts in unmapped."
said = "Triggers in time: " + ", ".join(TIMES[on] for on in have) + ". With no time zone in the sentence, use UTC."
if len(have) < len(TIMES):
said += f" Triggers on {' or '.join(on for on in TIMES if on not in have)} aren't available yet: put those in unmapped."
return said


def _fields(schema, prefix="", depth=0):
"""The dotted paths of an activity's fields with their titles, to name in tests."""
out = []
for k, v in ((schema or {}).get("properties") or {}).items():
path = f"{prefix}{k}"
if v.get("type") == "object" and depth < 1:
out += _fields(v, path + ".", depth + 1)
elif v.get("title"):
out.append(f"{path} ({v['title']})")
return out


def vocabulary():
"""What the model may use: the kinds of activity with their fields, the operators and the actions."""
kinds = [
{"kind": k["name"], "label": k["label"], "fields": _fields(k.get("schema") or {})} for k in events.kinds() if not k.get("noisy")
]
acts = [
{
"action": a["name"],
"label": a["label"],
"description": a.get("description") or "",
"params": a["params"],
"required": a.get("required") or [],
}
for a in actions.catalog()
]
return {"kinds": kinds, "ops": list(match.WORDS), "actions": acts}


def _soon(on):
"""Whether a trigger needs a part of the studio that isn't here yet (automations.SOON)."""
need = automations.SOON.get(on) if isinstance(on, str) else None
return need is not None and automations.provided(need[0], need[1]) is None


def _triggers(raw, dropped, unmapped):
out, seen = [], set()
for n, t in enumerate(raw if isinstance(raw, list) else [], 1):
if isinstance(t, dict) and _soon(t.get("on")):
unmapped.append(f"a trigger on {t['on']} (coming soon)")
continue
try:
got = automation_spec.check_trigger(t, n)
except ValueError as e:
where = isinstance(t, dict) and t.pop("where", None)
try:
got = automation_spec.check_trigger(t, n) if where else None
except ValueError:
got = None
if got is None:
dropped.append(str(e))
continue
dropped.append(f"{e} (its where was left out)")
if got["as"] in seen:
continue
seen.add(got["as"])
out.append(got)
if not out:
out = [{"as": "manual", "on": "manual"}]
dropped.append("no trigger fit, so it runs when you run it")
return out[: automation_spec.MAX_TRIGGERS]


def _actions(raw, dropped):
out = []
for n, a in enumerate(raw if isinstance(raw, list) else [], 1):
name = a.get("action") if isinstance(a, dict) else None
try:
spec = actions.get(str(name))
except KeyError:
dropped.append(f"action {n}: there's no action {name}")
continue
try:
params = actions.check_params(spec, a.get("params") or {})
except ValueError as e:
dropped.append(f"action {n}: {e}")
continue
out.append((spec, params))
return out


def _graph(acts):
"""The actions in order after the input node; one that reaches outside Lens (or may) is a leaf off the one
before it, as the leaf rule has it, and the rest go on from there."""
nodes, edges, last = [{"id": "input", "type": "input", "config": {}}], [], "input"
for n, (spec, params) in enumerate(acts, 1):
nid = f"a{n}"
nodes.append({"id": nid, "type": "action", "config": {"action": spec.name, "params": params}})
edges.append({"source": last, "target": nid})
if isinstance(spec.effect, str) and spec.effect != "out": # one worked out from its params may reach out
last = nid
return {"nodes": nodes, "edges": edges}


def draft(db, cfg, text, user, spaces=None):
"""{draft, dropped, unmapped}: `draft` is what POST /automations takes (switched off), `dropped` the parts of the
model's reply that didn't fit and why, `unmapped` the parts of the sentence it couldn't express."""
text = " ".join(str(text or "").split())
if not text:
raise ValueError("say what the automation should do")
if len(text) > TEXT_MAX:
raise ValueError(f"say it in at most {TEXT_MAX} characters")
account = automations._account(db, (user or {}).get("id"))
if not account:
raise ValueError("an automation acts for a person; sign in")
can = automations.editable(db, account)
if not can:
raise ValueError(f"{account.get('name') or account['email']} can't edit any namespace, so an automation couldn't work anywhere")
spaces = [int(s) for s in spaces] if spaces else None
if spaces and not set(spaces) <= can:
raise ValueError("you can make automations only in namespaces you can edit")
if not llm.configured(cfg):
raise ValueError("no language model is configured (Settings → LLM provider)")
refs = [f"account:{account['id']}", *(f"space:{s}" for s in spaces or [])]
why = budgets.used_up(db, refs)
if why:
raise ValueError(f"over budget: {why}")
vocab = vocabulary()
system = SYSTEM.replace("OPS", ", ".join(vocab["ops"])).replace("TIMES", _times())
ask = (
f"Kinds of activity and their fields:\n{json.dumps(vocab['kinds'])}\n\nActions:\n{json.dumps(vocab['actions'])}\n\nSentence: {text}"
)
with activity.scope(db, *refs, cfg=cfg):
try:
reply = llm.json_out(cfg, system, ask, SCHEMA)
except llm.LLMError as e:
raise ValueError(f"the model couldn't draft it: {e}") from None
dropped, unmapped = [], [str(u)[:300] for u in reply.get("unmapped") or [] if str(u).strip()][:20]
out = {
"name": " ".join(str(reply.get("name") or "").split())[:200] or "New automation",
"description": " ".join(str(reply.get("description") or "").split())[:2000] or None,
"enabled": False,
"triggers": _triggers(reply.get("triggers"), dropped, unmapped),
}
if spaces:
out["spaces"] = spaces
if reply.get("if"):
try:
out["if"] = automation_spec.check_if(reply["if"])
except ValueError as e:
dropped.append(f"the If: {e}")
acts = _actions(reply.get("actions"), dropped)
if acts:
out["graph"] = _graph(acts)
else:
dropped.append("no action fit; add one before saving")
if acts:
try:
automations._check(db, dict(out), None, account)
except ValueError as e:
dropped.append(f"as a whole: {e}")
return {"draft": store.clean(out), "dropped": dropped, "unmapped": unmapped}
13 changes: 13 additions & 0 deletions fastapi_backend/app/schemas/automations.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,3 +206,16 @@ class AutomationStarted(ResponseModel):

class AutomationUpcoming(ResponseModel):
times: list[Any] = Field(default_factory=list, description="when it runs next, from its schedules and timers (coming soon)")


class AutomationDraftRequest(RequestModel):
text: str = Field(
max_length=2000, description="what it should do, in a sentence, like: when a recording is added to pods, log its title"
)
spaces: list[int] | None = Field(None, description="namespace ids it would work in; default: each one you can edit")


class AutomationDrafted(ResponseModel):
draft: dict[str, Any] = Field(description="what POST /automations takes, switched off: name, triggers, if, graph")
dropped: list[str] = Field(default_factory=list, description="parts of the model's draft that didn't fit, and why")
unmapped: list[str] = Field(default_factory=list, description="parts of the sentence it couldn't express (schedules, say)")
19 changes: 19 additions & 0 deletions fastapi_backend/tests/api/test_automations.py
Original file line number Diff line number Diff line change
Expand Up @@ -339,3 +339,22 @@ def test_a_run_held_over_budget_waits_for_the_persons_pick(client, db, cfg, env)
assert client.get(f"{URL}/runs/{held['run']}", headers=he).json()["status"] == "released"
r = client.post(f"{URL}/runs/{held['run']}/decide", headers=he, json={"run": False})
assert r.status_code == 400 and "isn't waiting on a budget" in r.json()["detail"]


def test_an_automation_is_drafted_from_a_sentence(client, db, env, monkeypatch):
from app.domain import automation_draft

reply = {"name": "Say new", "triggers": NEW, "actions": [{"action": "log_line", "params": {"text": "New: {{ event.object.name }}"}}]}
monkeypatch.setattr(automation_draft.llm, "configured", lambda cfg: True)
monkeypatch.setattr(automation_draft.llm, "json_out", lambda cfg, system, user, schema, model=None: reply)
r = client.post(f"{URL}/draft", headers=env["he"], json={"text": "when something new comes in, log its name"})
assert r.status_code == 200, r.text
got = r.json()
assert got["draft"]["enabled"] is False and got["draft"]["triggers"] == [{**NEW[0], "from_automations": False}]
assert got["dropped"] == [] and got["unmapped"] == []
# it tries and saves as it is; drafting saved nothing
assert db.rows("SELECT id FROM automation") == []
assert client.post(f"{URL}/test", headers=env["he"], json={"draft": got["draft"]}).json()["status"] == "done"
assert client.post(URL, headers=env["he"], json=got["draft"]).status_code == 200
assert "can't edit any namespace" in problem(client, env["hv"], {"text": "log it"}, f"{URL}/draft")
assert client.post(f"{URL}/draft", headers=env["he"], json={"text": "x" * 2001}).status_code == 422
Loading
Loading