Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Automations run.** An automation saved over the API (`/api/v1/automations`) now reacts to new assets in its
namespaces: when its If passes, it runs its workflow as the person who made it, within its limits and budgets, and
each run shows what its actions did. Try one on an activity without keeping or sending anything, or run it now;
Settings → Automations switches them all off and sets how far they may go
([Running automations](docs/automations.md#running-automations)).

## 0.13.0 <small>October 16, 2026</small> {id="0.13.0"}

- **Updating Lens in a microsandbox on a Mac gets past "Building Lens".** An update after a build that stopped
Expand Down
1 change: 1 addition & 0 deletions docs/activity.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Each row names every resource it counts for, as `table:id`:
| Resource | Example |
| --- | --- |
| A routine and one of its runs | `routine:3`, `routine_run:12` |
| An automation and one of its runs | `automation:4`, `automation_run:51` |
| A job, its asset, namespace and pipeline | `job:40`, `recording:7`, `space:1`, `pipeline:2` |
| A workflow | `workflow:4` |
| A chat | `chat:9` |
Expand Down
16 changes: 15 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,13 +192,27 @@ DELETE /api/v1/automations/{id}
POST /api/v1/automations/{id}/restore
POST /api/v1/automations/{id}/enable
POST /api/v1/automations/{id}/policy
POST /api/v1/automations/{id}/test
POST /api/v1/automations/test
POST /api/v1/automations/{id}/run
GET /api/v1/automations/{id}/runs
GET /api/v1/automations/{id}/upcoming
GET /api/v1/automations/runs
GET /api/v1/automations/runs/{run}
POST /api/v1/automations/runs/{run}/decide
```

Automations act for the person who made them in namespaces they can edit ([Saving
automations](automations.md#saving-automations)). Admins see and change all of them; anyone else the ones that work
only in namespaces they can edit (`?removed=true` lists deleted ones, to restore). A save answers `{id, hooks}`.
`/policy` lets an action node run without asking (`auto`) only from a signed-in session in the web app.
`/actions` lists what action nodes can do.
`/actions` lists what action nodes can do. The list adds each one's actions waiting (`waiting`) and cost this month
(`cost_month`). `/test` tries a saved automation, and `POST /automations/test {"draft": ...}` an unsaved one, on an
activity (`activity`) or a kind's sample (`kind`) without keeping or sending anything; `/run` runs it now (on
`activity`, else the newest one its triggers listen for). `/runs` pages its runs (`before`, `limit`; `next` for the
page after), `/runs?activity=<id>` lists the runs an activity started, `/runs/{run}` gives one with its trace, log and
intents, and `/runs/{run}/decide {"run": true}` runs a run held over budget, or skips it ([Running
automations](automations.md#running-automations)).

## activities

Expand Down
106 changes: 88 additions & 18 deletions docs/automations.md

Large diffs are not rendered by default.

27 changes: 15 additions & 12 deletions docs/budgets.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Budgets

A budget caps what a routine, pipeline, workflow or namespace may cost. Budgets are off unless an admin sets one.
A budget caps what a routine, automation, pipeline, workflow or namespace may cost. Budgets are off unless an admin sets one.
What has been spent comes from the [activity ledger](activity.md), so a budget counts everything done for that
resource: a namespace's budget counts every job in it, and a routine's counts its runs and the jobs it queued.

Expand Down Expand Up @@ -44,17 +44,20 @@ PUT /api/v1/setup/budget

## Before a run

Before a routine runs and before a job's first step, Lens looks at the budgets of everything the run counts for: the
routine, the pipelines and workflows its actions name and its namespaces; a job's pipeline, namespace and the routine
that queued it. It estimates the next run from the average of that resource's runs in the last 30 days (always an
estimate, shown with ≈). A run goes ahead unless a budget is used up, or would be once the estimated run is done
(for a `run` budget, unless the estimate alone passes the cap). Then `on_over` decides:

- `ask` (default): the run is held. A routine gets a run with status `held`, a job the status `held`, each with
`hold.why` saying where the budget stands. Someone picks: run it now, once, whatever the budget says
(`POST /api/v1/routines/runs/{run}/decide {"run": true}`, `POST /api/v1/jobs/{jid}/release {"run": true}`), or skip
it (`{"run": false}`). Doing nothing changes nothing: a held run stays held, and a routine that comes due again adds
to the one held run (`hold.missed`) rather than piling up.
Before a routine runs, an automation runs and a job's first step, Lens looks at the budgets of everything the run
counts for: the routine, the pipelines and workflows its actions name and its namespaces; the automation, its workflow
and its namespaces; a job's pipeline, namespace and the routine that queued it. It estimates the next run from the
average of that resource's runs in the last 30 days (always an estimate, shown with ≈). A run goes ahead unless a
budget is used up, or would be once the estimated run is done (for a `run` budget, unless the estimate alone passes
the cap). Then `on_over` decides:

- `ask` (default): the run is held. A routine or an automation gets a run with status `held`, a job the status
`held`, each with `hold.why` saying where the budget stands. Someone picks: run it now, once, whatever the budget
says, or skip it (`{"run": false}`):
`POST /api/v1/routines/runs/{run}/decide {"run": true}`,
`POST /api/v1/automations/runs/{run}/decide {"run": true}`,
`POST /api/v1/jobs/{jid}/release {"run": true}`. Doing nothing changes nothing: a held run stays held, and a routine
or automation that comes due again adds to the one held run (`hold.missed`) rather than piling up.
- `skip`: the run is skipped, and says why.
- `assistant`: the decision model (Settings › AI assistant, decisions) weighs it, with where the budget stands and the
estimate. It runs only when the model is sure (`decisions.act_above`); otherwise it's held as with `ask`.
Expand Down
13 changes: 9 additions & 4 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -463,15 +463,20 @@ Where namespaces may send notifications ([Notifications](notifications.md)). Set

## Automations

The stream of activities automations follow ([Automations](automations.md#contract)), their timers, and
how long their [approvals](automations.md#approvals) wait. Settings → Automations:
How automations run ([Running automations](automations.md#running-automations)), the stream of activities they
follow ([Automations](automations.md#contract)), their timers, and how long their
[approvals](automations.md#approvals) wait. Settings → Automations:

| Setting | Default | |
|---|---|---|
| `automations.enabled` | true | off: no automation runs and nothing is lost; back on, they react to what happens from then on |
| `automations.max_depth` | 3 | how long a chain of automations setting each other off may get, 1–10 |
| `automations.max_runs_per_minute` | 60 | runs a minute, all automations together, 1–6000 |
| `automations.keep_runs_days` | 30 | how long runs are kept, 1–3650 days; each automation keeps its last 20 whatever their age |
| `automations.catch_up` | `1h` | how late a timer (a schedule, an action that waits) may still fire after Lens was off, `0s` to `7d`; older ones are missed ([Time](automations.md#time)) |
| `automations.approval_days` | 7 | how long an automation's action waits for your pick before it expires and nothing runs, 1–365 days |
| `events.poll_seconds` | 5 | how often followers look for new activities when nothing wakes them, 1–3600 |
| `events.keep_days` | 30 | how long activities with no row of their own (the event log) are kept, 1–3650 days |
| `automations.approval_days` | 7 | how long an automation's action waits for your pick before it expires and nothing runs, 1–365 days |
| `automations.catch_up` | `1h` | how late a timer (a schedule, an action that waits) may still fire after Lens was off, `0s` to `7d`; older ones are missed ([Time](automations.md#time)) |

## Trusted proxies

Expand Down
2 changes: 1 addition & 1 deletion docs/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ In Docker, `localhost` is the container itself: point the endpoint at a collecto
| Trace | each call to the model server (`chat llama3.1`) | model, server host, temperature, tokens in and out, estimated cost |
| Metric | `http.server.request.duration` | method, route, status code |
| Metric | `lens.job.step.duration`, `lens.jobs` | step type, outcome |
| Metric | `lens.routine.runs`, `lens.workflow.runs` | outcome |
| Metric | `lens.routine.runs`, `lens.automation.runs`, `lens.workflow.runs` | outcome |
| Metric | `gen_ai.client.operation.duration`, `gen_ai.client.token.usage` | model, token type |
| Metric | `lens.llm.cost` (USD) | model |

Expand Down
141 changes: 138 additions & 3 deletions fastapi_backend/app/api/v1/routes/automations.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,31 @@

from typing import Any

from fastapi import APIRouter, HTTPException
from fastapi import APIRouter, HTTPException, Query

from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer, domain_errors
from app.domain import actions, auth, automations, store
from app.domain import actions, auth, automations, events, store
from app.schemas.automations import (
ActionInfo,
Automation,
AutomationCatalog,
AutomationCreate,
AutomationDraftTry,
AutomationPolicy,
AutomationRun,
AutomationRunDetail,
AutomationRunNow,
AutomationRuns,
AutomationSaved,
AutomationStarted,
AutomationSwitch,
AutomationTried,
AutomationTry,
AutomationUpcoming,
AutomationUpdate,
)
from app.schemas.common import Ok
from app.schemas.routines import HeldRunDecision

router = APIRouter(prefix="/automations", tags=["automations"])

Expand Down Expand Up @@ -49,7 +59,8 @@ def list_automations(user: CurrentUser, acl: Acl, db: Db, removed: bool = False)
"""Your automations: every one for an admin, else those that work only in namespaces you can edit. `removed`
lists the deleted ones instead, to restore."""
spaces = None if user.admin else acl.editable()
return AutomationCatalog(automations=automations.list_automations(db, spaces, removed))
rows = automations.list_automations(db, spaces, removed)
return AutomationCatalog(automations=automations.extras(db, rows))


@router.get("/actions")
Expand Down Expand Up @@ -132,3 +143,127 @@ def set_policy(aid: int, body: AutomationPolicy, user: Writer, acl: Acl, db: Db)
got = automations.set_policy(db, aid, body.node, body.value, user.email)
auth.audit(db, user.as_audit(), "automation.policy", f"automation:{aid}", {"node": body.node, "value": body.value})
return got


def _run_of(db: Any, user: Principal, acl: Any, run_id: int) -> dict[str, Any]:
"""A run of an automation this person manages (404 otherwise), with its trace, log and intents."""
try:
r = automations.get_run(db, run_id)
except KeyError:
raise HTTPException(404, "not found") from None
for removed in (False, True):
try:
a = automations.get(db, r["automation"], removed)
except KeyError:
continue
if automations.manages(a, _editable(db, user, acl), user.admin):
return r
raise HTTPException(404, "not found")


def _activity(db: Any, cfg: Any, user: Principal, acl: Any, activity_id: str | None, kind: str | None) -> dict[str, Any] | None:
"""The activity to try or run it on: one you may see, by id, or a kind's sample; None for the default."""
if activity_id:
a = events.get(db, cfg, activity_id, events.viewer(user.id, acl.readable(), user.admin))
if a is None:
raise HTTPException(404, "no such activity")
return a
if kind:
try:
a = events.kind_info(kind)["sample"]
except KeyError:
raise HTTPException(400, f"no kind {kind}; GET /api/v1/activities/kinds lists them") from None
if a is None:
raise HTTPException(400, f"{kind} has no sample to try it on; give an activity")
return a
return None


@router.get("/runs")
def runs_caused(user: CurrentUser, acl: Acl, db: Db, activity: str = Query(description="an activity's id")) -> list[AutomationRun]:
"""What an activity caused: the runs of your automations it started (it was their event, or one of a join's)."""
editable = _editable(db, user, acl)
out = []
for r in automations.caused(db, activity):
try:
a = automations.get(db, r["automation"])
except KeyError:
continue
if automations.manages(a, editable, user.admin):
out.append(AutomationRun(**r))
return out


@router.get("/runs/{run:int}")
def get_run(run: int, user: CurrentUser, acl: Acl, db: Db) -> AutomationRunDetail:
"""One run: what each node did (trace), its log, and what its actions did, asked or wait to do (intents)."""
return AutomationRunDetail(**_run_of(db, user, acl, run))


@router.post("/runs/{run:int}/decide")
def decide_held_run(run: int, body: HeldRunDecision, user: Writer, acl: Acl, db: Db, cfg: Cfg) -> AutomationStarted:
"""Pick for a run held over a budget: run it now, once, whatever its budgets say, or skip it."""
r = _run_of(db, user, acl, run)
with domain_errors():
got = automations.decide_held(db, cfg, run, body.run, user.email)
auth.audit(db, user.as_audit(), "automation.release" if body.run else "automation.skip", f"automation:{r['automation']}", {"run": run})
return AutomationStarted(run=got, status=None if got is None else automations.get_run(db, got)["status"])


@router.post("/test")
def test_draft(body: AutomationDraftTry, user: CurrentUser, acl: Acl, db: Db, cfg: Cfg) -> AutomationTried:
"""Try an automation before you save it: a dry run on an activity, or on a kind's sample (default: its first
trigger's kind). Says whether its If passes and what each action would do; keeps, sends and counts nothing."""
a = _activity(db, cfg, user, acl, body.activity, body.kind)
draft = body.draft.model_dump(exclude_unset=True, by_alias=True)
with domain_errors():
got = automations.test(db, cfg, draft=draft, activity=a, alias=body.as_, user=_who(user))
return AutomationTried(**got)


@router.post("/{aid:int}/test")
def test_automation(aid: int, user: CurrentUser, acl: Acl, db: Db, cfg: Cfg, body: AutomationTry | None = None) -> AutomationTried:
"""Try a saved automation: a dry run on an activity, or on a kind's sample (default: its first trigger's kind).
Keeps, sends and counts nothing."""
body = body or AutomationTry()
_mine(db, user, acl, aid)
a = _activity(db, cfg, user, acl, body.activity, body.kind)
with domain_errors():
got = automations.test(db, cfg, aid, activity=a, alias=body.as_)
return AutomationTried(**got)


@router.post("/{aid:int}/run")
def run_automation(aid: int, user: Writer, acl: Acl, db: Db, cfg: Cfg, body: AutomationRunNow | None = None) -> AutomationStarted:
"""Run it now: on an activity, else the newest one its triggers listen for, else as started by hand. It acts as it
would on its own: limits and budgets apply, and actions that ask first wait for your pick."""
body = body or AutomationRunNow()
_mine(db, user, acl, aid)
a = _activity(db, cfg, user, acl, body.activity, None)
with domain_errors():
got = automations.run_now(db, cfg, aid, _who(user), activity=a, alias=body.as_)
auth.audit(db, user.as_audit(), "automation.run", f"automation:{aid}", {"run": got["run"]})
return AutomationStarted(**got)


@router.get("/{aid:int}/runs")
def list_runs(
aid: int,
user: CurrentUser,
acl: Acl,
db: Db,
before: int | None = Query(None, description="a run id: the runs before it"),
limit: int = Query(20, ge=1, le=100),
) -> AutomationRuns:
"""Its runs, newest first; `next` pages on."""
_mine(db, user, acl, aid, removed=False)
rows = automations.runs(db, aid, before, limit)
return AutomationRuns(runs=[AutomationRun(**r) for r in rows], next=rows[-1]["id"] if len(rows) == limit else None)


@router.get("/{aid:int}/upcoming")
def upcoming(aid: int, user: CurrentUser, acl: Acl, db: Db, n: int = Query(10, ge=1, le=100)) -> AutomationUpcoming:
"""When it runs next, from its schedules and timers (coming soon: empty until schedules are built)."""
_mine(db, user, acl, aid)
with domain_errors():
return AutomationUpcoming(times=automations.upcoming(db, aid, n))
2 changes: 1 addition & 1 deletion fastapi_backend/app/api/v1/routes/budgets.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from app.schemas.common import Ok

router = APIRouter(tags=["budgets"])
RESOURCE = Query(description="all (everything Lens spends), routine:<id>, pipeline:<id>, workflow:<id> or space:<id>")
RESOURCE = Query(description="all (everything Lens spends), routine:<id>, automation:<id>, pipeline:<id>, workflow:<id> or space:<id>")


@router.get("/budgets")
Expand Down
2 changes: 2 additions & 0 deletions fastapi_backend/app/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,7 @@ def platform_main(argv, config):

from .domain import (
auth,
automations,
channels,
https,
jobs,
Expand Down Expand Up @@ -386,6 +387,7 @@ def password():
notify.start(db, C, stop, name=wk.name, log=say) # sends notifications too (docs/notifications.md)
if not (a.steps or a.no_schedule): # and scans watched folders and runs routines, as `lens watch` does
routines.start(db, C, stop, log=say)
automations.start(db, C, stop, log=say) # and automations, following the stream
sensors.start(db, C, stop, log=say, name=wk.name) # and the sensor hub, while sensors are on
smb_share.start(db, C, stop, name=wk.name, log=say) # and the SMB share of the file tree, while it's on
shares.start(db, C, stop, log=say, name=wk.name) # and the file shares (S3, WebDAV), while they're on
Expand Down
4 changes: 3 additions & 1 deletion fastapi_backend/app/core/runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from app.domain import (
activity,
auth,
automations,
channels,
components,
content_types,
Expand Down Expand Up @@ -85,14 +86,15 @@ def prepare(self) -> None:
)

def start_background(self) -> None:
"""Inline workers, the watched-folder poller, the notifier, the routine scheduler, the sensor hub (idle while
"""Inline workers, the watched-folder poller, the notifier, the routine scheduler, automations, the sensor hub (idle while
sensors are off) and the chat channels (each idle while it's off). Production runs these as separate processes (`lens worker`)."""
for i in range(max(0, int(self.current()["workers"]["inline"]))):
w = jobs.Worker(self.db, self.current, name=f"api-{os.getpid()}-{i}", log=log.info)
threading.Thread(target=w.loop, args=(self.stop,), daemon=True, name=f"worker-{i}").start()

notify.start(self.db, self.current, self.stop, name=f"api-{os.getpid()}", log=log.warning)
routines.start(self.db, self.current, self.stop, log=log.info)
automations.start(self.db, self.current, self.stop, log=log.info)
sensors.start(self.db, self.current, self.stop, log=log.info, name=f"api-{os.getpid()}")
channels.start_all(self.db, self.current, self.stop, lambda: self.base, name=f"api-{os.getpid()}", log_fn=log.warning)
tunnel.start(self.db, self.current, self.stop, name=f"api-{os.getpid()}", log=log.info)
Expand Down
6 changes: 5 additions & 1 deletion fastapi_backend/app/domain/activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
"recordings": "recording",
"resources": "recording",
"routines": "routine",
"automations": "automation",
"runs": "routine_run",
"pipelines": "pipeline",
"workflows": "workflow",
Expand Down Expand Up @@ -123,9 +124,12 @@ def current():


def _table(parts, i):
"""The table a key after path segment i names: /api/v1/notes/{id} are note pages, a recording's notes/{id} its notes."""
"""The table a key after path segment i names: /api/v1/notes/{id} are note pages, a recording's notes/{id} its
notes, /api/v1/automations/runs/{id} an automation's runs."""
if parts[i] == "notes" and i and parts[i - 1] == "v1":
return "note_page"
if parts[i] == "runs" and i and parts[i - 1] == "automations":
return "automation_run"
return TABLES.get(parts[i])


Expand Down
Loading
Loading