Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ merged is called built.
| A webhook address per automation trigger, and calls to it as `webhook.received` ([Webhook trigger](#webhook-trigger)) | built |
| Timers that outlast a restart and fire once across the API and the worker (`app/domain/timers.py`, [Time](#time)) | built |
| The timers behind schedule, `at` and `relative` triggers, the `clock.tick` kind and what's coming ([Time](#time)); automations set them once they are stored | built |
| Actions that wait (`delay`, `when`) ([Time](#time)) | planned |
| Actions that wait (`delay`, `when`): kept as timers, carried out at their time, asking first when they should then ([Time](#time)) | built |
| The Timeline page with filters, grouping, review keys and new items, and Home's Latest card ([Timeline](timeline.md)) | built |
| Storing and running automations, joins, approvals, variables, the studio | planned |

Expand Down Expand Up @@ -422,7 +422,8 @@ Checked when it is saved, on every path (create, a new version, a body):
with params that fit its schema, and each template in its params is valid.
- **The leaf rule:** an action that reaches outside Lens or other people (effect `out`, worked out from its params)
or that waits has nothing connected after it, as it can't pass anything on while it waits for a yes or its time.
- Waiting before an action (`delay`, `when`) is refused as coming soon until timers are built.
- Waiting before an action takes a `delay` (more than 0, like `30m`) or a `when` (a time, or a template giving one),
not both; it is kept as a timer ([Actions that wait](#actions-that-wait)).

Its automation runs it: an event workflow isn't run on an asset, as a pipeline step or as a routine's action, and the
canvas's list of workflows leaves it out (`GET /api/v1/workflows?scope=event` lists them).
Expand Down Expand Up @@ -636,6 +637,30 @@ timers.clock_state(db, automation) # each schedule and at trigger: wai
schedule with `GET /api/v1/routines/schedule`, which now reads weekdays, single days and days of the month in
words too ("every weekday at 07:00", "at 09:00 on the 20th of every month").

### Actions that wait

Built as timers; the automation's run hands them over once automations are stored. An action node with a `delay`
(`2h`) or a `when` (a time, or a template giving one) makes an intent with that time and status `scheduled`, and
`timers.later(db, intent)` keeps it as the timer `intent-<intent key>` (its run and node, like `intent-51-n7`), giving
back `{timer, at}` to keep with the intent. The run's page shows it as "scheduled for 18:22".

When its time comes, its risk is worked out again (`timers.risk_now`) from the automation as it is then, not as it
was when the run made the intent:

- `read` and `self` run on their own.
- `lens` runs on its own only if it did when the run made it (the event's own object) and its namespaces are still
the automation's.
- `out` runs on its own only while the person still has that node set to run on its own; otherwise it asks.
- The action's own check may only make it stricter. The effect is the stricter of the intent's and what its params
give now.

A routine one is carried out through `actions.perform` as the automation (its person's permissions as they are then),
inside its run's cause (what it makes carries `lens.cause`, for the loop guard) and its ledger scope
(`automation:<id>`, `automation_run:<run>`). One that should ask becomes an approval (`approvals.ask`, once approvals
are built; until then it waits, saying why). The run hears of each step through `automations.update_intent`: done
with what it gave back, failed with why, waiting, or expired when the automation was switched off or removed first,
the timer was cancelled, or Lens was off past the catch-up window.

## Refine later

- More operators: `in`, `matches` (bounded regular expressions), `between`, times against now, `changed`.
Expand Down Expand Up @@ -674,7 +699,7 @@ timers.clock_state(db, automation) # each schedule and at trigger: wai
- Calling a web address: secrets in the address itself (a Slack or Discord webhook URL) sealed like headers; PATCH and
DELETE (DELETE would count as deleting); OAuth sign-in for an API; form-encoded bodies; retries with backoff;
networks allowed for automations apart from notifications.
- Time: calendar-aware schedules ("first workday of the month", holidays); sunrise and sunset triggers; an all-day
- Time: an action that waits shown in the person's time zone in `upcoming` (today UTC); calendar-aware schedules ("first workday of the month", holidays); sunrise and sunset triggers; an all-day
event's date in the person's time zone rather than UTC; upcoming relative timers in the person's time zone; an
event that's cancelled calling off its relative timers; the timeline's kind filter leaving out `clock.tick`, which
is never stored; sub-second timers woken by `events.wake()` alone; telling the other process (the worker) at once when the
Expand Down
121 changes: 120 additions & 1 deletion fastapi_backend/app/domain/timers.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@

What sets them: `schedule_automation` (an automation's schedule and at triggers; they fire it with a `clock.tick`
activity), `relative` (a relative trigger's time for one activity, moved when the activity's object moves), and
`upcoming` lists what an automation has waiting, in words.
`later` (an action that waits: carried out at its time, asking first when it should then). `upcoming` lists what an
automation has waiting, in words.

While Lens was off: a timer due within automations.catch_up (1 hour by default) fires late, with `late: true` in its
payload; an older one is missed (counted on its automation). A repeating one fires once, for its latest time in the
Expand All @@ -23,6 +24,7 @@

from __future__ import annotations

import contextlib
import datetime as dt
import hashlib
import json
Expand Down Expand Up @@ -600,6 +602,114 @@ def relative(db, automation, alias, activity, field, offset="0s", now=None):
return {"key": key, "at": _iso(when), "set": True, "note": None}


# ---------- actions that wait ----------
PERFORM = "intent.perform" # carry out an intent whose time came (an action node with a delay or a when)
_STRICT = {e: n for n, e in enumerate(automation_spec.EFFECTS)} # read, self, lens, out: each reaches further


def later(db, intent, depth=1):
"""Keep an intent that waits for its time (an action node with a `delay` or a `when`; M6 hands it here): a timer
at its `when`, `intent-<key>`, that carries it out then, asking first when it should (`risk_now`). `depth` is
the run's (for the loop guard on what it causes). What to keep with the intent: {timer, at}."""
i = automation_spec.check_intent(intent)
if not i["when"]:
raise ValueError(f"intent {i['key']} doesn't say when it runs (when)")
key = f"intent-{i['key']}"
payload = {"intent": {**i, "status": "scheduled"}, "depth": max(1, int(depth or 1))}
row = set(db, key, i["when"], PERFORM, payload, i["automation"])
return {"timer": key, "at": row["at"]}


def _automation_now(db, aid):
"""The automation as it is stored now ({enabled, policy, spaces}); None when it isn't (removed, or automations
aren't stored yet)."""
if aid is None:
return None
try:
return db.one("SELECT enabled, policy, spaces FROM $r", r=R("automation", int(aid)))
except Exception: # noqa: BLE001 - no automation table yet
return None


def risk_now(db, intent, automation=None):
"""routine or ask, worked out again for an intent whose time came: from its effect (the stricter of the one it was
made with and what its params give now), the automation's policy now (an out action runs on its own only while
its node is set to), its namespaces now (a lens action stays routine only on the event's own object, in
namespaces the automation still has), and the action's own check, which may only be stricter. `automation` is the
stored automation ({policy, spaces}), or None when there is none to read: then an out action asks."""
from . import actions

spec = actions.get(intent["action"])
effect = max(intent["effect"], actions.effect_of(spec, intent["params"]), key=lambda e: _STRICT.get(e, 3))
policy = (automation or {}).get("policy") or {}
spaces = (automation or {}).get("spaces")
if effect in ("read", "self"):
risk = "routine"
elif effect == "lens":
still = spaces is None or {int(s) for s in intent["spaces"]} <= {int(s) for s in spaces}
risk = "routine" if intent["risk"] == "routine" and still else "ask"
else:
risk = "routine" if policy.get(intent["node"]) == "auto" else "ask"
if risk == "routine" and spec.risk is not None:
ctx = {"automation": {"id": intent["automation"], "spaces": intent["spaces"] if spaces is None else spaces}}
try:
risk = "routine" if spec.risk(intent["params"], {**ctx, "node": intent["node"]}) == "routine" else "ask"
except Exception: # noqa: BLE001 - when the action can't tell, it asks
risk = "ask"
return risk


def _intent_status(db, i, status, result=None):
"""Say on the run what became of an intent (M6's automations.update_intent owns the run's intents)."""
update = _provided("automations", "update_intent")
if update is None or i.get("run") is None:
log.info("automation %s run %s: intent %s %s", i.get("automation"), i.get("run"), i.get("key"), status)
return
update(db, i["run"], i["key"], status, result)


def _intent_ended(db, payload, status, why):
i = payload.get("intent")
if isinstance(i, dict):
_intent_status(db, i, "expired", {"why": why})


@handler(PERFORM, ended=_intent_ended)
def _perform(db, cfg, p):
from . import actions, activity # here: actions loads its action modules, which load routines, which load this

i = automation_spec.check_intent(p["intent"])
auto = _automation_now(db, i["automation"])
if i["automation"] is not None and _provided("automations", "fire") is not None and (not auto or auto.get("enabled") is False):
_intent_status(db, i, "expired", {"why": "the automation was switched off or removed before its time"})
return
try:
risk = risk_now(db, i, auto)
except KeyError:
_intent_status(db, i, "failed", {"error": f"there's no action {i['action']} any more"})
raise ValueError(f"there's no action {i['action']} any more") from None
if risk == "ask":
ask = _provided("approvals", "ask")
approval = ask(db, cfg, {**i, "risk": "ask", "status": "waiting"}) if ask is not None else None
_intent_status(db, i, "waiting", store.clean({"approval": approval, "why": "waiting for your pick"}))
return
refs = []
if i["automation"] is not None:
refs.append(f"automation:{i['automation']}")
if i["run"] is not None:
refs.append(f"automation_run:{i['run']}")
try:
with contextlib.ExitStack() as stack:
if i["automation"] is not None and i["run"] is not None:
stack.enter_context(events.caused_by({"automation": i["automation"], "run": i["run"], "depth": int(p.get("depth") or 1)}))
stack.enter_context(activity.scope(db, *refs, cfg=cfg))
out = actions.perform(db, cfg, {**i, "risk": "routine", "status": "done"}, None)
except Exception as e:
_intent_status(db, i, "failed", {"error": f"{type(e).__name__}: {e}"[:500]})
raise
_intent_status(db, i, "done", out)


# ---------- what is coming ----------
def _describe(r, now):
p = r.get("payload") or {}
Expand All @@ -616,6 +726,15 @@ def _describe(r, now):
name = (a.get("object") or {}).get("name") or a.get("summary") or a["id"]
what = f"{_offset_words(p.get('offset') or '0s')} “{str(name)[:120]}” {(title or p['field']).lower()}"
return f"{what[:1].upper()}{what[1:]}: on {local(r['at'], 'UTC', now)} (UTC)", True
if r["kind"] == PERFORM and isinstance(p.get("intent"), dict):
from . import actions

i = p["intent"]
try:
what = actions.describe(i["action"], i.get("params") or {})
except KeyError:
what = i["action"]
return f"{what}: on {local(r['at'], 'UTC', now)} (UTC)", bool(i.get("untrusted"))
alias_text = f" of trigger {alias}" if alias else ""
return f"{r['kind']}{alias_text}: on {local(r['at'], 'UTC', now)} (UTC)", False

Expand Down
9 changes: 6 additions & 3 deletions fastapi_backend/tests/domain/test_automation_nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def _problem(db, g):
return str(e.value)


def test_event_workflows_are_checked_on_save(db, made):
def test_event_workflows_are_checked_on_save(db, made, monkeypatch):
assert "needs an action node" in _problem(db, graph(node("p", "pick", path="event"), edges=[edge("in", "p")]))
assert "choose an action" in _problem(db, graph(node("a", "action", action="nope"), edges=[edge("in", "a")]))
assert "needs Line (text)" in _problem(db, graph(node("a", "action", action="log_line"), edges=[edge("in", "a")]))
Expand All @@ -101,9 +101,12 @@ def test_event_workflows_are_checked_on_save(db, made):
node("a", "action", action="tag", params={"object": "{{ event.object.id }}", "tags": "{{ event.tag }}"}), edges=[edge("in", "a")]
)
assert workflows.validate_graph(db, ok, "event")["nodes"][1]["config"]["params"]["tags"] == "{{ event.tag }}"
# waiting before an action comes with the time module; until then it is refused, as are both kinds of wait
# waiting before an action comes with the time module (timers.later); without it, it is refused
waits = graph(node("a", "action", action="log_line", params={"text": "x"}, delay="5m"), edges=[edge("in", "a")])
assert "coming soon" in _problem(db, waits)
assert workflows.validate_graph(db, waits, "event")["nodes"][1]["config"]["delay"] == "5m"
with monkeypatch.context() as m:
m.setattr(automation_nodes, "_timers", lambda: None)
assert "coming soon" in _problem(db, waits)
assert "not both" in _problem(
db,
graph(
Expand Down
Loading
Loading