From 8feffcc0ef8f01d7e9b0ff8a4fe32643cb335add Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 23:41:10 +0000 Subject: [PATCH] feat(automations): what people say to Lens as chat.message activities Messages Lens already keeps (chat_message with role user) are read by (created_at, id) over a new (role, created_at) index. A web-app message is its person's alone; a chat app's belongs to the namespace its room answers in, else to the answering account. Room chats keep where they are (chat.via), and an automation's own question to its agent is marked with its origin so it never counts as a person's. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01QewNs6jxSB2vRoag2EvdyX --- docs/automations.md | 25 ++- docs/chat-rooms.md | 4 + fastapi_backend/app/domain/actions_ai.py | 2 +- fastapi_backend/app/domain/bridge.py | 25 ++- fastapi_backend/app/domain/chat.py | 18 +- fastapi_backend/app/domain/events_streams.py | 188 ++++++++++++++++++ fastapi_backend/app/domain/store.py | 2 + .../tests/api/test_chat_activities.py | 151 ++++++++++++++ threat-model.json | 2 +- threat-model.md | 2 +- threat-model.yaml | 4 +- 11 files changed, 410 insertions(+), 13 deletions(-) create mode 100644 fastapi_backend/tests/api/test_chat_activities.py diff --git a/docs/automations.md b/docs/automations.md index 5b34b303a..ba32ea2c8 100644 --- a/docs/automations.md +++ b/docs/automations.md @@ -16,7 +16,8 @@ merged is called built. | Runs that finished or failed, deletions and calendar events that moved | built | | Comments, notes written and saved, and changes to the entity graph | built | | Sensors: readings a sensor keeps, alert and notable log patterns, devices that report for the first time ([Sensors](#sensors-as-activities)) | built | -| More kinds: chat, system events | planned | +| What people say to Lens, in the web app and chat apps, as `chat.message` ([Chat](#chat-as-activities)) | built | +| More kinds: system events | planned | | The timeline API (`GET /api/v1/activities`), the MCP tools `timeline` and `event_kinds`, the assistant's `timeline` | built | | Review marks on the timeline (reviewed, important, hidden), their filters and `mark_activities` ([Timeline](timeline.md)) | built | | The shapes of triggers, joins, an If, policy and intents, checked (`app/domain/automation_spec.py`) | built | @@ -106,6 +107,7 @@ and the paths holding outside text are in `GET /api/v1/activities/kinds` (`event | `sensor.reading` | | `sensor_readings` | `{type: Note, id: lens:sensor_reading/, name: ": ", url: /sensors/}`, `stream`, `value`, `text`, `fields` (a JSON reading's numbers), `level` (syslog), `pattern`, `label` (the line's pattern and its label); noisy | Kitchen sensor temperature: 31.2 | | `sensor.alert` | | `log` | `{type: Note, id: lens:sensor_pattern/}`, `label` (alert or notable), `pattern`, `example` (a line it fits), `count` (lines in all), `lines` (in the batch that raised it), `hour` | 192.168.1.1 logged an alert: wlan: authentication failed for | | `sensor.joined` | | `sensors` | `{type: Service, id: lens:sensor/, url: /sensors/}`, `sensor_type` (mqtt, syslog, ...), `device`, `status` | A new MQTT device reported: zigbee2mqtt/kitchen | +| `chat.message` | | `chat_messages` | `{type: Note, id: lens:chat_message/, name: the start of it, url: /chat/}`, `text`, `channel` (web, telegram, slack, bridge, mailbox), `room`, `chat`, `direct` (a private chat or the web app), `attachments` (how many files) | sam wrote in Telegram: “groceries: milk, eggs” | | `webhook.received` | | `log` | `{type: Note, hook: the webhook's id, payload: what was sent (JSON, a form's fields or text), content_type, query}`; the actor is the webhook (`Service`, `lens:automation_hook/`, "Webhook for ") | Webhook for Front door was called | A new asset is one activity, of the most specific kind its row tells, decided by its content type and where it came @@ -158,6 +160,22 @@ outside text. - `sensor.joined` is a device or host that made itself a sensor by reporting for the first time (MQTT, syslog), read from its `storage_source` row by `created_at`; a sensor added by hand isn't news. +#### Chat as activities + +Built (`app/domain/events_streams.py`). `chat.message` is what a person said to Lens: a question in the web app (Chat, +the setup conversation) or a message to Lens in a chat app (Telegram, Slack, Matterbridge, the assistant's mailbox). +It reads the messages Lens already keeps (`chat_message` with role user), so nothing new is stored, and what isn't +said to Lens in a room is never kept, so it is never an activity. Its text and names are outside text. + +- In the web app the actor is the person (`lens:account/`), and the activity is theirs alone (`lens.audience` + owner): not even admins or others in the namespace see it. Its namespace is the conversation's scope. +- In a chat app the actor is who wrote (`Person`, `lens:chat_user/:`, their name there), + since a chat user isn't a Lens account. A room given to a namespace's assistant belongs to that namespace; any + other conversation (a private chat, an open room) is the answering account's alone. The conversation keeps where + it is (`chat.via`: the channel, the room, who, a private chat or not) from its next message on. +- A question an automation's `ask_agent` asks (`origin` on the message) isn't a person's, so it is never a + `chat.message`, and an automation can't set itself off that way. + ### Sources and the event log Built. A source (`events.Source`, added with `events.register_source`) reads rows Lens already keeps as activities, @@ -190,6 +208,8 @@ A kind belongs to the source named in its `source`, and a source may list more i - `sensor_readings` (`app/domain/events_streams.py`) reads `sensor_reading` by `at` (microseconds), the id breaking ties, one sensor at a time, as `sensor.reading`; `sensors` reads the `storage_source` rows discovery made by `created_at`, as `sensor.joined`. +- `chat_messages` (`app/domain/events_streams.py`) reads `chat_message` rows with role user by `created_at` over the + index on (role, created_at), the id breaking ties, as `chat.message`. - `log` reads `event_log`: activities no row of their own records (a deletion, a webhook call), written by `events.emit(db, kind, *, actor, obj, space, summary, target=None, tag=None, key=None, published=None, spaces=None, audience=None, owner=None, untrusted=None)`, which gives the id (`log-`). Emitting a `key` again changes @@ -523,3 +543,6 @@ the agent can draft an event as a note. - Sensors: a threshold per reading in the intake (only readings over a value become activities, for sensors that keep everything); a reading's unit, which sensors don't record yet; an alert as soon as a pattern is labelled, for the lines already seen that hour. +- Chat: recording every message in a room as an activity (opt-in per channel, off by default); a chat user who is + also a Lens account as that person; a timeline page of others' private messages is read and left out, up to five + source pages, so narrow it by account if that proves slow. diff --git a/docs/chat-rooms.md b/docs/chat-rooms.md index 6f3044129..9770076d4 100644 --- a/docs/chat-rooms.md +++ b/docs/chat-rooms.md @@ -134,6 +134,10 @@ That's all. Without an account given, it answers as the admin who turned it on. "Matterbridge · room · person", listed in Chat with the account's other conversations. Follow-ups keep their context, and older messages are folded into a running summary ([Long conversations](assistant.md)). `Lens /clear` starts the conversation afresh and `Lens /compact` summarises it now. +- **Automations hear what's said to Lens.** Each message said to Lens (and only those: the room's other talk is never + kept) is a `chat.message` activity an automation can trigger on, "when someone messages Lens 'groceries' in + Telegram" ([Automations](automations.md#chat-as-activities)). A room given to a namespace belongs to it; any other + conversation is the answering account's alone. - **A namespace's own assistant.** **Rooms for a namespace's assistant** gives a room to a namespace (`team = pods`, or one channel: `team/general = pods`). Conversations there are scoped to that namespace, so when its assistant is on ([A namespace's own assistant](assistant.md#a-namespaces-own-assistant)) it answers with its diff --git a/fastapi_backend/app/domain/actions_ai.py b/fastapi_backend/app/domain/actions_ai.py index 0b43618ee..1ccc76396 100644 --- a/fastapi_backend/app/domain/actions_ai.py +++ b/fastapi_backend/app/domain/actions_ai.py @@ -257,7 +257,7 @@ def _ask_agent(db, cfg, intent, user, dry): if about: url = (about.get("object") or {}).get("url") if isinstance(about.get("object"), dict) else None page = {"url": str(url or f"/timeline?activity={about.get('id') or ''}"), "title": _short(happened, 300)} - chat.add(db, cid, "user", instructions, context=page) + chat.add(db, cid, "user", instructions, context=page, origin=box.origin) # reading the activity counts as reading outside text for this conversation from now on (ai_tools.read_before) steps = ( [{"tool": "automation_event", "args": {"id": about.get("id")}, "summary": f"Read what happened: {_short(happened, 200)}"}] diff --git a/fastapi_backend/app/domain/bridge.py b/fastapi_backend/app/domain/bridge.py index 0c7cdb550..974afd687 100644 --- a/fastapi_backend/app/domain/bridge.py +++ b/fastapi_backend/app/domain/bridge.py @@ -176,15 +176,30 @@ def _account(db, cfg): return {"id": u["id"], "email": u["email"], "admin": bool(u.get("admin"))} -def conversation(db, account, m, namespace=None, label="Matterbridge"): +def conversation(db, account, m, namespace=None, label="Matterbridge", channel="bridge"): """The conversation for this person in this room, started on their first message; scoped to `namespace` (the room's namespace assistant), which follows bridge.rooms when it changes. In a room no namespace is given to, the - scope is routing's (route). `label` names the channel in its title.""" + scope is routing's (route). `label` names the channel in its title; the chat keeps where it is (`via`: the + channel's key and label, the room, who is talking and whether it's a private chat) for its messages as activities + (events_streams.py, chat.message).""" who = m["thread"] if m.get("thread") else [m.get("gateway"), m.get("channel"), m.get("account"), m.get("username")] key = hashlib.sha1(json.dumps(who).encode()).hexdigest() scope = {"namespaces": [namespace]} if namespace else {} - cid = db.one("SELECT VALUE record::id(id) FROM chat WHERE bridge = $k AND account = $a LIMIT 1", k=key, a=account["id"]) + via = store.clean( + { + "channel": channel, + "label": label, + "room": str(m.get("channel") or "")[:200] or None, + "user": (m.get("username") or "").strip("<> ")[:200] or None, + "userid": str(m.get("userid") or "")[:200] or None, + "private": bool(m.get("private")), + } + ) + row = db.one("SELECT record::id(id) AS id, via FROM chat WHERE bridge = $k AND account = $a LIMIT 1", k=key, a=account["id"]) + cid = row["id"] if row else None if cid: + if row.get("via") != via: # a chat from before chats kept it, or someone renamed + db.q("UPDATE $r SET via = $v", r=R("chat", cid), v=via) if namespace: db.q("UPDATE $r SET scope = $s, room = 'given' WHERE (scope ?? {}) != $s", r=R("chat", cid), s=scope) else: # the room was given to a namespace and no longer is: routing starts again @@ -192,7 +207,7 @@ def conversation(db, account, m, namespace=None, label="Matterbridge"): return cid where = " · ".join(x for x in [m.get("channel"), (m.get("username") or "").strip("<> ")] if x) cid = chat.create(db, account["id"], title=f"{label} · {where}" if where else label, scope=scope, kind="bridge") - db.q("UPDATE $r SET bridge = $k, room = $g", r=R("chat", cid), k=key, g="given" if namespace else None) + db.q("UPDATE $r SET bridge = $k, room = $g, via = $v", r=R("chat", cid), k=key, g="given" if namespace else None, v=via) return cid @@ -513,7 +528,7 @@ def handle(db, cfg, base, m, ch=None): if not q: return False account = _account(db, cfg) - cid = conversation(db, account, m, ns if sid is not None else None, ch.label) + cid = conversation(db, account, m, ns if sid is not None else None, ch.label, ch.key) try: # a private chat with someone the channel vouches for is where the account's person is told things channels.remember(db, cfg, ch, account, m, cid) except Exception: # noqa: BLE001 - answering comes first diff --git a/fastapi_backend/app/domain/chat.py b/fastapi_backend/app/domain/chat.py index 4dc4252bb..a3b3180e4 100644 --- a/fastapi_backend/app/domain/chat.py +++ b/fastapi_backend/app/domain/chat.py @@ -351,10 +351,23 @@ def history(db, cid): def add( - db, cid, role, content, passages=None, stopped=False, steps=None, notice=None, error=None, model=None, attachments=None, context=None + db, + cid, + role, + content, + passages=None, + stopped=False, + steps=None, + notice=None, + error=None, + model=None, + attachments=None, + context=None, + origin=None, ): """Save a message; an answer keeps the tool steps it took, any notice (e.g. the model can't use tools) and error; a - question asked from a page keeps what it shared of it (shared_context).""" + question asked from a page keeps what it shared of it (shared_context). A question an automation asked keeps its + `origin` ({automation, run, node}), so it isn't taken for a person's message (`chat.message`).""" mid = db.next_id("chat_message") db.q( "CREATE $r CONTENT $d", @@ -373,6 +386,7 @@ def add( "error": error, "model": model, "context": context, + "origin": origin or None, } ), ) diff --git a/fastapi_backend/app/domain/events_streams.py b/fastapi_backend/app/domain/events_streams.py index 26b311859..e9b215682 100644 --- a/fastapi_backend/app/domain/events_streams.py +++ b/fastapi_backend/app/domain/events_streams.py @@ -10,6 +10,11 @@ once per pattern per hour, whatever the sensor keeps of its lines. - `sensor.joined` (source `sensors`): a device or host that made itself a sensor by reporting (`storage_source` rows discovery made), by `created_at`. +- `chat.message` (source `chat_messages`): what a person said to Lens, in the web app or a chat app (Telegram, Slack, + Matterbridge, the mailbox): `chat_message` rows with role user, read by (created_at, id) over the index on + (role, created_at). Only messages addressed to Lens are stored, so a room's other talk never is; a question an + automation asked its agent (`origin` on the row) isn't a person's. A web-app chat is its person's alone (`lens.audience` + owner); a chat app's belongs to the namespace its room answers in, else to the account the channel answers as. - `webhook.received`: a call to an automation's webhook address (automation_hooks.py), emitted by the route that takes it. Its payload and query are outside text. @@ -27,6 +32,8 @@ SENSOR_RX = re.compile(r"^lens:sensor/(\d+)$") READING_RX = re.compile(r"^reading-([a-z0-9]{1,100})$") JOINED_RX = re.compile(r"^joined-sensor-(\d+)$") +SAID_RX = re.compile(r"^said-(\d+)$") +REF_UNSAFE = re.compile(r"[^A-Za-z0-9_.:@+-]") TEXT_MOST = 1000 # characters of a reading's text an activity carries @@ -385,6 +392,144 @@ def alert(db, pid, pt, sensor, count): ) +# ---------- chat ---------- +SAID_FIELDS = "id, record::id(id) AS n, chat, content, created_at, attachments" +SAID_WHERE = "role = 'user' AND origin = NONE" + + +def _short(text, most): + text = " ".join(str(text or "").split()) + return text if len(text) <= most else text[: most - 1].rstrip() + "…" + + +def _said(db, rows): + """chat_message rows (role user) as chat.message activities; a message whose chat is gone, or whose time isn't one, + is skipped.""" + rows = [r for r in rows if r and isinstance(r.get("n"), int)] + if not rows: + return [] + chats = events_content._lookup(db, "chat", {r.get("chat") for r in rows}, "account, kind, scope, via, title") + ids = {name: sid for sid, name in store.space_names(db).items()} + people = events_content.actors(db, [(c.get("account"), None) for c in chats.values() if (c.get("kind") or "chat") != "bridge"]) + out = [] + for r in rows: + c = chats.get(r.get("chat")) + if not c: + continue + try: + published = events.stamp(r["created_at"]) + except (TypeError, ValueError, KeyError): + continue + spaces = [ids[n] for n in ((c.get("scope") or {}).get("namespaces") or []) if n in ids] + text = str(r.get("content") or "")[:TEXT_MOST] + via = c.get("via") or {} + if (c.get("kind") or "chat") == "bridge": + label = str(via.get("label") or "a chat room") + room = via.get("room") or None + channel, direct = str(via.get("channel") or "bridge"), bool(via.get("private")) + who = via.get("userid") or via.get("user") + name = _short(via.get("user") or via.get("userid") or c.get("title") or "Someone", 200) + key = REF_UNSAFE.sub("_", f"{channel}:{who}")[:200] if who else f"chat-{r['chat']}" + actor = {"type": "Person", "id": f"lens:chat_user/{key}", "name": name} + where = f"{label} · {room}" if room else label + summary = f"{name} wrote in {where}: “{_short(text, 120)}”" + seen = ( + {"space": spaces[0], "audience": None, "owner": None} + if spaces + else {"space": None, "audience": "owner", "owner": c.get("account")} + ) + else: + actor = people.get((c.get("account"), None)) or {"type": "Person", "id": "lens:account", "name": "Someone"} + channel, room, direct = "web", None, True + summary = f"{actor['name']} wrote: “{_short(text, 120)}”" + seen = {"space": spaces[0] if spaces else None, "audience": "owner", "owner": c.get("account")} + obj = store.clean( + { + "type": "Note", + "id": f"lens:chat_message/{r['n']}", + "name": _short(text, 120) or "a message", + "url": f"/chat/{r['chat']}", + "text": text, + "channel": channel, + "room": room, + "chat": r["chat"], + "direct": direct, + "attachments": len(r["attachments"]) if isinstance(r.get("attachments"), list) and r["attachments"] else None, + } + ) + out.append( + events.build( + f"said-{r['n']}", + "chat.message", + actor=actor, + obj=obj, + summary=summary, + source="chat_messages", + pos=[r["created_at"], r["n"]], + published=published, + spaces=spaces if len(spaces) > 1 else None, + **seen, + ) + ) + return out + + +def _said_rows(db, pos, op, limit, since=None, until=None): + """Messages past `pos` ([created_at, id]; op ">" oldest first, "<" newest first): those at its time with a later + id, then those after it, so the (role, created_at) index's range and order do the work.""" + order = "created_at, id" if op == ">" else "created_at DESC, id DESC" + bounds, p = "", {} + if since: + bounds += " AND created_at > $since" + p["since"] = events_content._sec(since) + if until: + bounds += " AND created_at <= $until" + p["until"] = events_content._sec(until) + rows = [] + if pos is not None and pos[1] is not None: + rows += db.rows( + f"SELECT {SAID_FIELDS} FROM chat_message WHERE {SAID_WHERE} AND created_at = $pt AND id {op} $pi{bounds} " + f"ORDER BY {order} LIMIT {int(limit)}", + pt=pos[0], + pi=R("chat_message", pos[1]), + **p, + ) + where = "true" if pos is None else f"created_at {op} $pt" + if pos is not None: + p["pt"] = pos[0] + rows += db.rows( + f"SELECT {SAID_FIELDS} FROM chat_message WHERE {SAID_WHERE} AND {where}{bounds} ORDER BY {order} LIMIT {int(limit)}", **p + ) + return rows[: int(limit)] + + +class ChatSource(events.Source): + """What people said to Lens (chat_message rows with role user) as chat.message, read by (created_at, id).""" + + name = "chat_messages" + + def read(self, db, cfg, pos, spaces, limit, hint=None): + return _said(db, _said_rows(db, pos, ">", limit)) # follow keeps the namespaces asked for + + def page(self, db, cfg, before, who, flt, limit): + return _said(db, _said_rows(db, before, "<", limit, flt.get("since"), flt.get("until"))) + + def get(self, db, cfg, activity_id): + got = self.get_many(db, cfg, [activity_id]) + return got[0] if got else None + + def get_many(self, db, cfg, ids): + ns = [int(m.group(1)) for m in (SAID_RX.match(i) for i in ids) if m] + if not ns: + return [] + return _said( + db, db.rows(f"SELECT {SAID_FIELDS} FROM chat_message WHERE id IN $rs AND {SAID_WHERE}", rs=[R("chat_message", n) for n in ns]) + ) + + +events.register_source(ChatSource()) + + _SENSOR_OBJECT = { "stream": _s("Stream"), "value": _s("Value", "number"), @@ -533,3 +678,46 @@ def _sample(aid, kind, as_type, actor, obj, summary, source, pos, untrusted, spa JOINED_UNTRUSTED, ), ) +CHAT_UNTRUSTED = ("actor.name", "object.name", "object.text", "object.room") +events.kind( + "chat.message", + label="Message to Lens", + source="chat_messages", + as_type="Create", + object_type="Note", + fields={ + "object": { + "type": "object", + "properties": { + "text": _s("What was said"), + "channel": _s("Where", enum=["web", "telegram", "slack", "bridge", "mailbox"]), + "room": _s("Room"), + "chat": _s("Conversation", "integer"), + "direct": _s("A private chat with Lens", "boolean"), + "attachments": _s("Files sent with it", "integer"), + }, + } + }, + untrusted=CHAT_UNTRUSTED, + sample=_sample( + "said-81", + "chat.message", + "Create", + {"type": "Person", "id": "lens:chat_user/telegram:5528", "name": "sam"}, + { + "type": "Note", + "id": "lens:chat_message/81", + "name": "groceries: milk, eggs", + "url": "/chat/12", + "text": "groceries: milk, eggs", + "channel": "telegram", + "chat": 12, + "direct": True, + }, + "sam wrote in Telegram: “groceries: milk, eggs”", + "chat_messages", + ["2026-10-15T16:21:58+00:00", 81], + CHAT_UNTRUSTED, + space=3, + ), +) diff --git a/fastapi_backend/app/domain/store.py b/fastapi_backend/app/domain/store.py index 05e3858f3..76ead7244 100644 --- a/fastapi_backend/app/domain/store.py +++ b/fastapi_backend/app/domain/store.py @@ -1372,6 +1372,8 @@ def close(self): "DEFINE INDEX IF NOT EXISTS chat_account ON chat FIELDS account", "DEFINE TABLE IF NOT EXISTS chat_message SCHEMALESS", "DEFINE INDEX IF NOT EXISTS chat_message_chat ON chat_message FIELDS chat", + # what people said to Lens, by time, as chat.message (events_streams.py) + "DEFINE INDEX IF NOT EXISTS chat_message_said ON chat_message FIELDS role, created_at", # what each namespace's assistant remembers (ns_assistant.py); its name and instructions are space.assistant "DEFINE TABLE IF NOT EXISTS assistant_memory SCHEMALESS", "DEFINE INDEX IF NOT EXISTS assistant_memory_space ON assistant_memory FIELDS space", diff --git a/fastapi_backend/tests/api/test_chat_activities.py b/fastapi_backend/tests/api/test_chat_activities.py new file mode 100644 index 000000000..ed8811d6b --- /dev/null +++ b/fastapi_backend/tests/api/test_chat_activities.py @@ -0,0 +1,151 @@ +"""What people say to Lens as `chat.message` activities (docs/automations.md, Kinds): a question in the web app is its +person's alone; one said to the Telegram bot belongs to the namespace its room answers in, else to the account the bot +answers as; what isn't said to Lens is never stored, so it never is an activity; an automation's own question to its +agent isn't a person's.""" + +from __future__ import annotations + +import time + +import pytest + +from app.domain import channels, chat, events, store, telegram +from tests import fake_llm, fake_telegram +from tests.api._assist import start_llm +from tests.helpers import make_user, seed + +update = fake_telegram.update +KINDS = ["chat.message"] + + +@pytest.fixture(autouse=True) +def fresh(): + events._seen.clear() + events._claimed.clear() + yield + + +@pytest.fixture +def tg(): + srv, url = fake_telegram.start() + telegram._BOTS.clear() + yield fake_telegram.Handler, url + srv.shutdown() + + +@pytest.fixture +def llm(cfg): + srv = start_llm(cfg) + yield fake_llm.Handler + srv.shutdown() + + +@pytest.fixture +def app(cfg, db, llm): + from app.main import create_app + + return create_app(cfg, db, background=False) + + +@pytest.fixture +def bot(app, db, cfg, folder, tg): + seed(db, cfg, folder) + make_user(db, "bot@x.io", "bot password 1", roles={"pods": "viewer"}) + h, url = tg + cfg["telegram"] = { + **cfg["telegram"], + "enabled": True, + "api_url": url, + "token": fake_telegram.TOKEN, + "account": "bot@x.io", + "users": ["alice", "bob"], + "rooms": ["-100 = pods"], + } + cfg["ai"]["tools"] = False + return h + + +def _start(db, cfg, consumer): + """A follower's first look marks where it starts: what's said in that second isn't seen, so wait for the next.""" + assert events.follow(db, cfg, consumer, kinds=KINDS)[0] == [] + t = store.now() + while store.now() == t: + time.sleep(0.05) + + +def test_a_question_in_the_web_app_is_its_persons_alone(db, cfg): + me = make_user(db, "me@x.io", "my password 1", roles={"pods": "editor"}) + other = make_user(db, "other@x.io", "other password 1", roles={"pods": "editor"}) + pods = store.ns_id(db, "pods") + _start(db, cfg, "test.chat") + cid = chat.create(db, me, "Trip", scope={"namespaces": ["pods"]}) + chat.add(db, cid, "user", "When does the shipment leave?") + chat.add(db, cid, "assistant", "On Friday.") + agent = chat.create(db, me, "Automation 4: ...") + chat.add(db, agent, "user", "File this where it belongs.", origin={"automation": 4, "run": 1, "node": "a"}) + + got, commit = events.follow(db, cfg, "test.chat", kinds=KINDS) + commit() + assert len(got) == 1 + a = events.check_activity(got[0]) + assert a["kind"] == "chat.message" and a["type"] == "Create" and a["space"] == pods + assert a["actor"] == {"type": "Person", "id": f"lens:account/{me}", "name": "me"} + assert a["object"] == { + "type": "Note", + "id": f"lens:chat_message/{a['id'].removeprefix('said-')}", + "name": "When does the shipment leave?", + "url": f"/chat/{cid}", + "text": "When does the shipment leave?", + "channel": "web", + "chat": cid, + "direct": True, + } + assert a["summary"] == "me wrote: “When does the shipment leave?”" + assert a["lens"]["audience"] == "owner" and a["lens"]["owner"] == me and "object.text" in a["lens"]["untrusted"] + # its person's alone, even for admins and others in the namespace + assert events.get(db, cfg, a["id"], events.viewer(me, [pods]))["id"] == a["id"] + assert events.get(db, cfg, a["id"], events.viewer(other, [pods])) is None + assert events.get(db, cfg, a["id"], events.viewer(other, None, admin=True)) is None + assert [x["id"] for x in events.page(db, cfg, events.viewer(me, [pods]), flt={"kinds": KINDS})["items"]] == [a["id"]] + assert events.page(db, cfg, events.viewer(other, [pods]), flt={"kinds": KINDS})["items"] == [] + assert events.follow(db, cfg, "test.chat", kinds=KINDS)[0] == [] + + +def test_what_is_said_to_the_telegram_bot(db, cfg, bot, llm): + pods = store.ns_id(db, "pods") + account = db.one("SELECT VALUE record::id(id) FROM account WHERE email = 'bot@x.io'") + _start(db, cfg, "test.telegram") + bot.updates = [ + update("groceries: milk, eggs"), # a private chat with alice + update("lunch anyone?", chat_id=-100, chat_type="group", title="Team", user="bob", user_id=8), # not for Lens + update("@lens_test_bot when does the shipment leave?", chat_id=-100, chat_type="group", title="Team", user="bob", user_id=8), + ] + assert channels.tick(db, cfg, cfg, "api-1", telegram.TELEGRAM) == 2 + + got, commit = events.follow(db, cfg, "test.telegram", kinds=KINDS) + commit() + for a in got: + events.check_activity(a) + assert [a["object"]["text"] for a in got] == ["groceries: milk, eggs", "when does the shipment leave?"] + mine, room = got + assert mine["actor"] == {"type": "Person", "id": "lens:chat_user/telegram:7", "name": "alice"} + assert {k: mine["object"][k] for k in ("channel", "direct")} == {"channel": "telegram", "direct": True} + assert "room" not in mine["object"] + assert mine["space"] is None and mine["lens"]["audience"] == "owner" and mine["lens"]["owner"] == account + assert mine["summary"] == "alice wrote in Telegram: “groceries: milk, eggs”" + assert room["actor"]["id"] == "lens:chat_user/telegram:8" and room["actor"]["name"] == "bob" + assert {k: room["object"][k] for k in ("channel", "room", "direct")} == {"channel": "telegram", "room": "Team", "direct": False} + assert room["space"] == pods and "audience" not in room["lens"] # the room answers in pods + assert room["summary"] == "bob wrote in Telegram · Team: “when does the shipment leave?”" + assert events.get(db, cfg, room["id"], events.viewer(99, [pods]))["id"] == room["id"] + assert events.get(db, cfg, mine["id"], events.viewer(99, [pods])) is None + assert events.get(db, cfg, mine["id"], events.viewer(account, [pods]))["id"] == mine["id"] + + # a follower of one namespace hears only that namespace's rooms + bot.updates = [ + update("and on Monday?"), + update("@lens_test_bot and on Monday?", chat_id=-100, chat_type="group", title="Team", user="bob", user_id=8), + ] + assert channels.tick(db, cfg, cfg, "api-1", telegram.TELEGRAM) == 2 + got, _ = events.follow(db, cfg, "test.telegram", kinds=KINDS, spaces=[pods]) + assert [(a["actor"]["name"], a["object"]["text"]) for a in got] == [("bob", "and on Monday?")] diff --git a/threat-model.json b/threat-model.json index aa45d1b96..8e9bd0477 100644 --- a/threat-model.json +++ b/threat-model.json @@ -457,7 +457,7 @@ "parameter": "prompt text; model-chosen tool calls", "attacker_controllable": "yes", "component": "assistant", - "caller_must_enforce": "none — P-RETRIEVAL", + "caller_must_enforce": "none — P-RETRIEVAL; what people say to Lens reaches automations as outside text (chat.message, lens.untrusted)", "provenance": "documented", "source": "docs/assistant.md \"limited to what the person can read\"" }, diff --git a/threat-model.md b/threat-model.md index 72b762014..5a6edd772 100644 --- a/threat-model.md +++ b/threat-model.md @@ -290,7 +290,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `POST /imports/web`, extension `http` tools, notification targets | URL | yes: editors (web, tools), owners (targets) | resource-name, x-network-destination | none | *(maintainer, 2026-10)* | | admin settings: model, embeddings, speech, decisions, Fedora, bridge, Telegram bot, Slack app tokens, assistant mailbox, telemetry, IIIF import, rclone remotes, IMAP, Gmail (a Google refresh token or a Workspace service account key), iCal, MQTT bridge, OIDC issuer | URL, credentials, `api_key_env` | admin only; documented guards still apply (P-ADMIN-GUARD) | resource-name, x-credential | admin is trusted | *(maintainer, 2026-10)* | | replies from remote servers | web page HTML and scripts, IIIF manifests, model replies, OIDC discovery, calendars and their redirects | yes, the remote | data, topology | treat as untrusted | *(inferred, Q17)* | -| chat, `/graph/ask`, MCP tools | prompt text; model-chosen tool calls | yes; also indirectly through content | data, x-prompt-injection | none | *(documented, docs/assistant.md "limited to what the person can read")* | +| chat, `/graph/ask`, MCP tools | prompt text; model-chosen tool calls | yes; also indirectly through content | data, x-prompt-injection | none; what people say to Lens, in the web app or a chat app, reaches automations as outside text (`chat.message`, `lens.untrusted`) | *(documented, docs/assistant.md "limited to what the person can read")* | | Lens extension side panel | answers shown there (model text that can repeat third-party text), the sign-in coming back | yes, indirectly through content | data, x-prompt-injection | none; answers render as Markdown cleaned by DOMPurify (no scripts, images or forms; only http and https links), and the sign-in checks its `state` and PKCE | *(documented, extensions/browser/src/core/assistant.js answerHtml docstring "cleaned of anything that could run or fetch on its own")* | | extensions (prompt, graph and `http` tools, hooks), custom nodes including `template` nodes (Q23) | tool spec, graph JSON, template text | yes, any writer for their own; sharing is gated | callback/code (declarative) | person enables only extensions they trust | *(documented, docs/assistant.md "other people's run only when an admin shared them")* | | Python code tools, shipped and admin-created templates, workflows, pipelines, routines | Python, Jinja source, regexes | admin only | callback/code | admin is trusted | *(maintainer, 2026-10)* | diff --git a/threat-model.yaml b/threat-model.yaml index 63f88e7f1..237d01d42 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -2,7 +2,7 @@ # Authority order: prose > yaml > json. Regenerate whenever the prose changes. schema: threat-model-sidecar/v2 project: lens -prose_version: "threat-model.md@sha256:bcdebb916f791169f475437f97fea2d166bfca909704eba3c4f48b3b5786c2f1" +prose_version: "threat-model.md@sha256:a396f25c854ef342e1d269daf00bed3ac90f3346087c8a02c56deb8875878e00" model_status: under-review triage_policy: strict confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8} @@ -386,7 +386,7 @@ entry_points: attacker_controllable: true x-condition: "also indirectly through content" control_kinds: [data, x-prompt-injection] - caller_must_enforce: "none — P-RETRIEVAL" + caller_must_enforce: "none — P-RETRIEVAL; what people say to Lens reaches automations as outside text (chat.message, lens.untrusted)" provenance: {kind: documented, source: 'docs/assistant.md "limited to what the person can read"'} - id: "Lens extension side panel" component: assistant