From 3ff67427628ecf29f9c593506c6fb19d95dc78ab Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sun, 18 Oct 2026 06:11:42 +0000 Subject: [PATCH 1/3] feat(sandbox): run the assistant's code on compute nodes sandbox.register_nodes(list_fn, driver_fn) lets the mesh hand over the install's nodes and a Driver for each node's own gateway. run_python and run_bash then take a node, sandbox_nodes lists where code can run, and a conversation's sandbox stays where it last ran; moving starts a fresh one and deletes the old. Sweep, forget and clear act on each sandbox where it is. Without registered nodes nothing changes. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01GbBS222wEEQgkuui6CHo6c --- CHANGELOG.md | 5 + README.md | 3 +- docs/assistant.md | 20 ++ fastapi_backend/app/domain/ai_tools.py | 33 ++- fastapi_backend/app/domain/sandbox.py | 330 ++++++++++++++++++---- fastapi_backend/tests/api/test_sandbox.py | 80 ++++++ nextjs-frontend/components/chat/steps.tsx | 2 + threat-model.json | 4 +- threat-model.md | 2 +- threat-model.yaml | 4 +- 10 files changed, 417 insertions(+), 66 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d10f77d6..ce7877cbc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,11 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **The code sandbox can run on compute nodes.** Once the mesh registers the install's compute nodes, `run_python` + and `run_bash` take a `node` to run on, `sandbox_nodes` lists where code can run and why a node can't now, and a + conversation's sandbox stays where it last ran (moving starts a fresh one). Each node's sandboxes are stopped, + swept and deleted through its own gateway, and each run's ledger row names its node. Until nodes are registered + nothing changes ([Compute nodes](docs/assistant.md#compute-nodes)). - **The assistant sends mail, adds events and writes files, after your yes.** New tools `send_mail` (send, or save a draft, from Gmail; `reply_to` keeps a message's thread), `add_event` (an event on Google Calendar, inviting people if asked) and `write_file` (a text file on a storage connection, replacing one only when asked). Each always waits diff --git a/README.md b/README.md index 92728108e..ceb1f83b6 100644 --- a/README.md +++ b/README.md @@ -242,7 +242,8 @@ These are open pull requests, not yet part of Lens: ([#406](https://github.com/adeelahmad/lense/pull/406)). - **Automations:** more operators for conditions ([#434](https://github.com/adeelahmad/lense/pull/434)). - **Nodes:** Lens's own private network between the machines it runs on, a Nebula mesh it issues the certificates - for ([#546](https://github.com/adeelahmad/lense/pull/546)). + for ([#546](https://github.com/adeelahmad/lense/pull/546)); the assistant's code sandbox can already run on the + compute nodes it will register ([Compute nodes](docs/assistant.md#compute-nodes)). ## Planned diff --git a/docs/assistant.md b/docs/assistant.md index 259070c90..0c0ac94d4 100644 --- a/docs/assistant.md +++ b/docs/assistant.md @@ -530,6 +530,26 @@ people it's turned on for: those who can edit a namespace, and admins (`sandbox. - **Its client.** Lens's own client for the gateway (the `sandbox` Python extra, OpenShell's SDK, Apache-2.0) is fetched once the sandbox is on ([Components](components.md)). +### Compute nodes + +In progress: the seam is built, and the mesh that joins the nodes is being built in its own pull request. Once the +install's mesh registers compute nodes (its other machines, each with its own sandbox gateway, reached only over the +mesh's own network), the assistant can run a conversation's code on one of them: + +- **`sandbox_nodes`** lists where code can run: this machine (`here`) and each compute node, with whether it can run + code now and, if not, why (it hasn't reported in the last 3 minutes, it isn't on the mesh, its sandbox gateway isn't + running), its machine, its groups, and which one has this conversation's sandbox. +- **`node`** on `run_python` and `run_bash` (a node's id or name, or `here`) runs there. Left out, code runs where the + conversation's sandbox already is: this machine until a run asks for a node. A sandbox that moves starts fresh where + it moves to, without earlier files, and the one it left is deleted. The model is told to pick a node only when the + person asks for a machine or for more power than this one has. +- **The same rules everywhere.** A node's sandbox has the same image, limits and network setting as this machine's, + is stopped when idle and deleted when unused or with its conversation, by its own gateway. Each run's ledger row + names its node (`sandbox.run`, `node`). Turning off this machine's gateway leaves the nodes' sandboxes alone, and a + node removed from the mesh takes its sandboxes with it. +- **Nothing changes without nodes:** until the mesh registers some, there is no `sandbox_nodes` and no `node`, and + every sandbox runs on this machine's gateway. + ### The gateway Built: the Docker Compose install has a `sandbox` profile that runs OpenShell's gateway (release 0.1.4) next to Lens. diff --git a/fastapi_backend/app/domain/ai_tools.py b/fastapi_backend/app/domain/ai_tools.py index 715a93cbf..3131599b4 100644 --- a/fastapi_backend/app/domain/ai_tools.py +++ b/fastapi_backend/app/domain/ai_tools.py @@ -713,7 +713,7 @@ def catalog(box): if browsermod.enabled(box.cfg): offered |= {t for t in GROUPS["browser"]["tools"] if t not in off} if sandboxmod.offered(box.cfg, box.admin, bool(box.editable)): # every conversation of theirs has its sandbox - offered |= {t for t in GROUPS["code"]["tools"] if t not in off} + offered |= {t[0] for t in sandboxmod.tools() if t[0] not in off} if researchmod.available(box.cfg): # every conversation of theirs can start research offered |= {t for t in GROUPS["research"]["tools"] if t not in off} # code that can reach another host can carry out what it's given @@ -841,6 +841,8 @@ def system_note(self): out = self._home_note() + (browsermod.NOTE if self.browser else "") if self.sandbox: out += sandboxmod.NOTE.format(network=sandboxmod.network_note(self.cfg)) + if sandboxmod.nodes_on(): + out += sandboxmod.NODES_NOTE given = {s["function"]["name"] for s in self.specs()} if self.web and webmod.NAMES & given: out += webmod.NOTE @@ -918,7 +920,7 @@ def specs(self): if self.browser: tools += browsermod.TOOLS if self.sandbox: - tools += sandboxmod.TOOLS + tools += sandboxmod.tools() if self.web: tools += webmod.TOOLS if self.chat is not None and researchmod.available(self.cfg): @@ -1156,26 +1158,33 @@ def _sandbox(self, fn, *args): except sandboxmod.SandboxError as e: raise ValueError(str(e)) from None - def _sandbox_run(self, kind, source, files): + def _sandbox_run(self, kind, source, files, node=None): source = str(source or "") files = [str(f) for f in ([files] if isinstance(files, str) else files or [])] + node = str(node).strip() if node is not None and str(node).strip() else None if sandboxmod.sends_out(self.cfg) and self.wary() and f"run_{kind}" not in self.allowed_out: # code that reaches another host can carry out what the answer read: after text from outside Lens, which # may have put it there, a run asks first first = next((x.strip() for x in source.splitlines() if x.strip()), "")[:80] - args = store.clean({"code" if kind == "python" else "command": source, "files": files or None}) + args = store.clean({"code" if kind == "python" else "command": source, "files": files or None, "node": node}) lang = "Python" if kind == "python" else "bash" where = sandboxmod.reaches(self.cfg) - return self._approval(f"run_{kind}", args, f"Run {lang} in the code sandbox, which can reach {where}: {first}" + self.why()) - out = self._sandbox(sandboxmod.run, self.account, kind, source, files) + on = f" on {node}" if node else "" + return self._approval(f"run_{kind}", args, f"Run {lang} in the code sandbox{on}, which can reach {where}: {first}" + self.why()) + out = self._sandbox(sandboxmod.run, self.account, kind, source, files, None, node) self.detail = sandboxmod.shown(out) # the output under the code, on the step the person sees return out, sandboxmod.summary(kind, source, out) - def t_run_python(self, code, files=None): - return self._sandbox_run("python", code, files) + def t_run_python(self, code, files=None, node=None): + return self._sandbox_run("python", code, files, node) - def t_run_bash(self, command, files=None): - return self._sandbox_run("bash", command, files) + def t_run_bash(self, command, files=None, node=None): + return self._sandbox_run("bash", command, files, node) + + def t_sandbox_nodes(self): + out = sandboxmod.node_list(self.db, self.chat) + n = sum(1 for x in out["nodes"] if x["id"] != sandboxmod.HERE) + return out, f"Looked at where code can run ({n} compute node{'s' if n != 1 else ''})" def t_sandbox_files(self): out = self._sandbox(sandboxmod.listing, self.account) @@ -2290,7 +2299,9 @@ def _carry_out(db, cfg, a, user, editable, decision, base, admin, readable, via= kind = a["tool"].removeprefix("run_") source = args.get("code") if kind == "python" else args.get("command") try: - result = {"result": sandboxmod.run(db, cfg, a["chat"], a.get("account") or user["id"], kind, source, args.get("files") or [])} + account = a.get("account") or user["id"] + got = sandboxmod.run(db, cfg, a["chat"], account, kind, source, args.get("files") or [], None, args.get("node")) + result = {"result": got} except sandboxmod.SandboxError as e: raise ValueError(str(e)) from None elif a["tool"] in ( diff --git a/fastapi_backend/app/domain/sandbox.py b/fastapi_backend/app/domain/sandbox.py index 2bf15931c..cc4751c80 100644 --- a/fastapi_backend/app/domain/sandbox.py +++ b/fastapi_backend/app/domain/sandbox.py @@ -17,6 +17,12 @@ turned on), over mutual TLS with the client certificate the Compose profile's gateway made (tls_dir), or a stand-in that tests set with use_driver. The Compose profile `sandbox` runs the gateway on this machine; the machine's upgrade agent starts and stops it (turn, preflight says what stands in the way). + +Compute nodes: other machines of the install, joined over its own mesh network, can run conversations' sandboxes too. +The mesh registers them (register_nodes: its list of nodes, and a driver for a node's own gateway, reached over the +mesh). run_python and run_bash then take a node, sandbox_nodes lists them, and a conversation's sandbox stays where it +last ran until it's asked to run elsewhere (moving starts a fresh one there). Nothing changes until nodes are +registered: every sandbox runs on this machine's gateway. """ from __future__ import annotations @@ -110,6 +116,11 @@ "items": _S, "description": "ids of files attached to this conversation to copy into the sandbox first, by their own names", } +NODE = { + "type": "string", + "description": 'a compute node\'s id or name from sandbox_nodes to run on ("here": this machine); left out, where ' + "this conversation's sandbox already is. A sandbox that moves starts fresh there, without earlier files", +} # (name, description, parameters, required, changes Lens data) TOOLS = [ ( @@ -117,7 +128,7 @@ "Run Python 3 code in this conversation's own sandbox, to compute, analyse, convert or make files. Its files " "stay between runs in this conversation; it has no Lens data, keys or access. Print what you need: you get " "stdout, stderr and the exit code back. files: attachments to copy in first.", - {"code": _S, "files": FILES}, + {"code": _S, "files": FILES, "node": NODE}, ["code"], False, ), @@ -126,7 +137,7 @@ "Run a bash script in this conversation's own sandbox (the same one run_python uses, with its files): Unix " "tools, pip install when the network allows it. You get stdout, stderr and the exit code back. files: " "attachments to copy in first.", - {"command": _S, "files": FILES}, + {"command": _S, "files": FILES, "node": NODE}, ["command"], False, ), @@ -137,6 +148,15 @@ [], False, ), + ( + "sandbox_nodes", + "The machines that can run this conversation's code: this one (\"here\") and the install's compute nodes, " + "each with whether it can run code now (and why not), its machine and groups, and which one has this " + "conversation's sandbox.", + {}, + [], + False, + ), ( "share_sandbox_file", "Give the person a file the sandbox made (a path from sandbox_files) to download from your answer: a chart, a " @@ -155,6 +175,15 @@ ), ] NAMES = {t[0] for t in TOOLS} +NODE_TOOLS = {"sandbox_nodes"} # offered only once compute nodes are registered (nodes_on) + + +def tools(): + """The sandbox's tools as the model gets them: without nodes, no sandbox_nodes and no node to pick.""" + if nodes_on(): + return TOOLS + return [(n, d, {k: v for k, v in p.items() if k != "node"}, r, c) for n, d, p, r, c in TOOLS if n not in NODE_TOOLS] + NOTE = ( "\n\nYou can run Python and bash with run_python and run_bash, in a sandbox of this conversation's own that keeps " @@ -164,6 +193,10 @@ "it showed. share_sandbox_file gives the person a file it made to download; import_sandbox_file puts one into a " "namespace, once the person approves." ) +NODES_NOTE = ( + " Code can also run on the install's compute nodes (sandbox_nodes lists them): pass node to run_python or run_bash " + "only when the person asks for a machine, or for more power than this one has." +) NETWORK_NOTE = { "off": "It has no network: nothing can be downloaded or sent, so don't try pip install.", "packages": "Its network reaches only the Python and npm package registries (pip install works).", @@ -408,8 +441,9 @@ def use_driver(make): @contextlib.contextmanager -def _gateway(cfg, driver=None, timeout=30.0) -> Iterator[Driver]: - """The gateway, open for the calls inside; its errors come out as SandboxError, said plainly.""" +def _gateway(cfg, driver=None, timeout=30.0, where=None) -> Iterator[Driver]: + """The gateway, open for the calls inside; its errors come out as SandboxError, said plainly. `where` names a + node's gateway in them (default: this machine's, at sandbox.gateway).""" d = driver or (_driver(cfg) if _driver else None) own = d is None try: @@ -419,8 +453,9 @@ def _gateway(cfg, driver=None, timeout=30.0) -> Iterator[Driver]: except SandboxError: raise except Exception as e: - log.warning("sandbox: the gateway at %s failed: %s", settings(cfg)["gateway"], e) - raise SandboxError(f"the code sandbox's gateway ({settings(cfg)['gateway']}) didn't work: {_said(e)}") from None + at = where or settings(cfg)["gateway"] + log.warning("sandbox: the gateway at %s failed: %s", at, e) + raise SandboxError(f"the code sandbox's gateway ({at}) didn't work: {_said(e)}") from None finally: if own and d is not None: with contextlib.suppress(Exception): @@ -443,6 +478,164 @@ def check(cfg, driver=None, timeout=30.0): return None +# ---------- compute nodes: the install's other machines, over its mesh (mesh.py registers them) ---------- +HERE = "here" # this machine, as the model names it +FRESH_SECONDS = 180 # a node that reported within this is up (nodes report every minute) +_nodes: tuple | None = None # (list_fn(db) -> the install's nodes, driver_fn(db, cfg, node) -> its gateway's Driver) + + +def register_nodes(list_fn, driver_fn): + """Let conversations' code run on compute nodes. `list_fn(db)` gives the install's nodes (mesh.nodes rows: id, + name, role primary|worker|compute, ip, groups, last_seen, status {nebula, sandbox, machine, ...}, revoked_at), and + `driver_fn(db, cfg, node)` a Driver for that node's own gateway, reached over the mesh. (None, None): this machine + only again.""" + global _nodes + _nodes = (list_fn, driver_fn) if list_fn and driver_fn else None + + +def nodes_on(): + """Whether compute nodes are registered, so the model may pick where code runs.""" + return _nodes is not None + + +def _fresh(when): + try: + t = dt.datetime.fromisoformat(str(when or "").replace("Z", "+00:00")) + except ValueError: + return False + if t.tzinfo is None: + t = t.replace(tzinfo=dt.timezone.utc) + return (dt.datetime.now(dt.timezone.utc) - t).total_seconds() <= FRESH_SECONDS + + +def _why_not(node): + """Why a node can't run code now, in words; None when it can.""" + st = node.get("status") or {} + if node.get("role") != "compute": + return "it isn't a compute node" + if node.get("revoked_at"): + return "it was removed from the mesh" + if not _fresh(node.get("last_seen")): + return f"it hasn't reported since {node.get('last_seen') or 'it joined'}" + if not st.get("nebula"): + return "it isn't connected to the mesh" + if not st.get("sandbox"): + return "its sandbox gateway isn't running" + return None + + +def ready(node): + """Whether a node can run code now: a compute node, not removed, that reported in the last few minutes with the + mesh up and its sandbox gateway running.""" + return _why_not(node) is None + + +def _all(db): + """The install's nodes, but those removed (none without nodes registered).""" + if not _nodes: + return [] + try: + return [n for n in _nodes[0](db) or [] if not n.get("revoked_at")] + except Exception as e: + log.warning("sandbox: couldn't list the compute nodes: %s", e) + return [] + + +def _named(node): + return str(node.get("name") or f"node {node['id']}") + + +def _find(db, which): + """The node `which` names (its id, else its name), ready to run code; None for this machine.""" + w = str(which if which is not None else "").strip() + if not w or w.casefold() in (HERE, "this machine", "local"): + return None + if not _nodes: + raise SandboxError("only this machine runs code: no compute nodes have joined") + rows = _all(db) + hit = [n for n in rows if str(n.get("id")) == w] or [n for n in rows if str(n.get("name") or "").casefold() == w.casefold()] + if not hit: + raise SandboxError(f"there's no node {which}: sandbox_nodes lists them") + node = hit[0] + if node.get("role") == "primary": # the primary is the machine Lens runs on + return None + why = _why_not(node) + if why: + raise SandboxError(f"{_named(node)} can't run code now: {why}") + return node + + +def _node_of(db, row): + """The node a conversation's sandbox is on (its row's `node`): None for this machine, or when that node is gone.""" + at = (row or {}).get("node") + if at is None: + return None + return next((n for n in _all(db) if str(n.get("id")) == str(at)), None) + + +def _place(db, chat, which=None): + """Where a conversation's code runs: on the node asked for, else where its sandbox is (this machine at first, or + when its node was removed). Raises when that node can't run code now.""" + if which is not None and str(which).strip(): + return _find(db, which) + row = _row(db, chat) + node = _node_of(db, row) + if node is not None and not ready(node): + raise SandboxError( + f"this conversation's sandbox is on {_named(node)}, which can't run code now ({_why_not(node)}): run on " + f'another node, or here (node "{HERE}"), in a fresh sandbox' + ) + return node + + +@contextlib.contextmanager +def _at(db, cfg, node, driver=None) -> Iterator[Driver]: + """The gateway of this machine (node None) or of a compute node, open for the calls inside.""" + if node is None: + with _gateway(cfg, driver) as d: + yield d + return + try: + d = _nodes[1](db, cfg, node) if _nodes else None + except Exception as e: + raise SandboxError(f"Lens couldn't reach {_named(node)}: {_said(e)}") from None + if d is None: + raise SandboxError(f"Lens couldn't reach {_named(node)}") + try: + with _gateway(cfg, d, where=_named(node)) as g: + yield g + finally: + with contextlib.suppress(Exception): + d.close() + + +def node_list(db, chat=None): + """What sandbox_nodes says: this machine, then each compute node, whether it can run code now (and why not), and + which one has the conversation's sandbox.""" + row = _row(db, chat) if chat is not None else None + at = (row or {}).get("node") + out = [{"id": HERE, "name": "this machine", "ready": True, "has_sandbox": bool(row) and at is None}] + for n in _all(db): + if n.get("role") != "compute": + continue + st = n.get("status") or {} + out.append( + store.clean( + { + "id": n["id"], + "name": n.get("name"), + "ready": ready(n), + "why_not": _why_not(n), + "machine": st.get("machine"), + "groups": n.get("groups") or None, + "last_seen": n.get("last_seen"), + "has_sandbox": bool(row) and str(at) == str(n["id"]), + } + ) + ) + return {"nodes": out} + + # ---------- the gateway on this machine: the Compose profile `sandbox` ---------- PROFILE = "sandbox" GATEWAY = "openshell-gateway:17670" # where the profile's gateway answers on the network it shares with Lens @@ -556,7 +749,7 @@ def gateway_view(db, cfg, probe=True, driver=None): def clear(db, cfg, driver=None): """Delete every sandbox Lens made on the gateway, files and all (the gateway is going away). Returns how many went; when the gateway doesn't answer, nothing is deleted here and the sweep deletes them once it's back.""" - rows = db.rows("SELECT record::id(id) AS chat, name FROM code_sandbox") + rows = [r for r in db.rows("SELECT record::id(id) AS chat, name, node FROM code_sandbox") if r.get("node") is None] try: with _gateway(cfg, driver) as d: names = set(d.names()) | {r.get("name") or name_for(r["chat"]) for r in rows} @@ -565,7 +758,8 @@ def clear(db, cfg, driver=None): except SandboxError as e: log.info("sandbox: %s; its sandboxes are deleted later", e) return 0 - db.q("DELETE code_sandbox") + for r in rows: # the compute nodes' sandboxes stay: their gateways aren't going away + db.q("DELETE $r", r=R("code_sandbox", int(r["chat"]))) return len(names) @@ -621,23 +815,26 @@ def _row(db, chat): return db.one("SELECT * FROM $r", r=R("code_sandbox", int(chat))) -def _ready(db, cfg, d, chat, account): - """The conversation's sandbox, made, started or remade so it's running with today's settings. Returns what became of - it: "new" (made now), "remade" (its network, hosts or image changed, or it was lost or broken: earlier files are gone) or - "kept".""" +def _ready(db, cfg, d, chat, account, node=None): + """The conversation's sandbox, made, started or remade on `d` (this machine's gateway, or compute node `node`'s) so + it's running with today's settings. Returns what became of it: "new" (made now), "remade" (its network, hosts or + image changed, or it was lost or broken: earlier files are gone), "moved" (it ran elsewhere before: a fresh one + here) or "kept".""" s, name = settings(cfg), name_for(chat) row = _row(db, chat) + at = node["id"] if node else None + moved = bool(row) and row.get("node") != at phase, until = d.phase(name), time.monotonic() + READY_SECONDS while phase in ("stopping", "deleting") and time.monotonic() < until: time.sleep(1) phase = d.phase(name) hosts = s["hosts"] if s["network"] == "hosts" else [] changed = bool(row) and (row.get("network") != s["network"] or row.get("image") != s["image"] or (row.get("hosts") or []) != hosts) - if not row or changed or phase in (None, "error", "stopping", "deleting"): - if phase is not None: # its settings changed, it broke, or it's another's from before (an id used again) + if not row or changed or moved or phase in (None, "error", "stopping", "deleting"): + if phase is not None: # its settings changed, it broke, it's from before it moved, or another's (an id used again) d.delete(name, wait=True) d.create(name, s) - became = "new" if not row else "remade" + became = "new" if not row else "moved" if moved else "remade" else: if phase == "stopped": d.start(name) @@ -652,6 +849,7 @@ def _ready(db, cfg, d, chat, account): "chat": int(chat), "account": account, "name": name, + "node": at, "network": s["network"], "hosts": hosts, "image": s["image"], @@ -722,10 +920,24 @@ def _fit(text, chars, end=False): return (text[-n:] if end else text[:n]) if n else "" -def run(db, cfg, chat, account, kind, source, files=(), driver=None): - """Run `source` (Python, or a bash script) in the conversation's sandbox, after copying in `files` (attachment ids). - What it printed comes back as the model reads it: the start of stdout and the end of stderr (where a traceback - ends), cut to fit RESULT_CHARS. The run is a row in the activity ledger (sandbox.run), with its time.""" +def _leave(db, cfg, row, driver=None): + """The conversation's sandbox moves: the one it had where it ran before is deleted, files and all. When that + machine can't be reached it stays there (a node removed from the mesh takes its sandboxes with it).""" + old = None if row.get("node") is None else _node_of(db, row) + if row.get("node") is not None and old is None: + return + try: + with _at(db, cfg, old, driver) as d: + d.delete(row.get("name") or name_for(row["chat"])) + except SandboxError as e: + log.info("sandbox: the sandbox it moved from stays: %s", e) + + +def run(db, cfg, chat, account, kind, source, files=(), driver=None, node=None): + """Run `source` (Python, or a bash script) in the conversation's sandbox, after copying in `files` (attachment ids): + on compute node `node` (its id or name) when given, else where its sandbox is (_place). What it printed comes back + as the model reads it: the start of stdout and the end of stderr (where a traceback ends), cut to fit RESULT_CHARS. + The run is a row in the activity ledger (sandbox.run), with its time and node.""" if kind not in ("python", "bash"): raise ValueError(f"the sandbox runs python or bash, not {kind}") source = str(source or "") @@ -734,19 +946,22 @@ def run(db, cfg, chat, account, kind, source, files=(), driver=None): s = settings(cfg) bundle, landed = _bundle(db, cfg, chat, account, files) if files else (b"", []) name = name_for(chat) - with _gateway(cfg, driver) as d: - became = _ready(db, cfg, d, chat, account) + at = _place(db, chat, node) + row = _row(db, chat) + if row and row.get("node") != (at["id"] if at else None): + _leave(db, cfg, {**row, "chat": chat}, driver) + with _at(db, cfg, at, driver) as d: + became = _ready(db, cfg, d, chat, account, at) if bundle: put = d.exec(name, ["python3", "-c", _UNTAR], bundle, s["seconds"], 4000) if put.exit_code != 0: raise SandboxError(f"the files couldn't be copied in: {_text(put.stderr)[-300:].strip() or 'no reason given'}") command = ["python3", "-"] if kind == "python" else ["bash", "-s"] t0 = time.monotonic() - with activity.call( - "sandbox.run", cfg, refs=[activity.ref("chat", int(chat))], detail={"language": kind, "network": s["network"]} - ) as row: + detail = {"language": kind, "network": s["network"], **({"node": at["id"]} if at else {})} + with activity.call("sandbox.run", cfg, refs=[activity.ref("chat", int(chat))], detail=detail) as led: ran = d.exec(name, command, source.encode(), s["seconds"], MAX_OUTPUT) - row.detail = {**(row.detail or {}), "exit_code": ran.exit_code} + led.detail = {**(led.detail or {}), "exit_code": ran.exit_code} took = round(time.monotonic() - t0, 2) db.q("UPDATE $r SET used_at = $t", r=R("code_sandbox", int(chat)), t=store.now()) out, err = _text(ran.stdout), _text(ran.stderr) @@ -765,12 +980,14 @@ def run(db, cfg, chat, account, kind, source, files=(), driver=None): result["cut_short"] = "the output was longer than this: print less, or write it to a file and read parts of it" if landed: result["copied_in"] = landed + if at: + result["node"] = _named(at) + where = f" on {_named(at)}" if at else " here" if became == "moved" else "" if became != "kept": - result["sandbox"] = ( - "a new sandbox for this conversation" - if became == "new" - else "a fresh sandbox (the settings changed, or it had stopped working): files from earlier runs are gone" - ) + result["sandbox"] = { + "new": f"a new sandbox for this conversation{where}", + "moved": f"a fresh sandbox{where}, where it runs from now on: files from earlier runs are gone", + }.get(became, "a fresh sandbox (the settings changed, or it had stopped working): files from earlier runs are gone") return result @@ -810,8 +1027,9 @@ def listing(db, cfg, chat, account, driver=None): """The files in the conversation's sandbox (none yet when it has no sandbox).""" if not _row(db, chat): return {"files": [], "more": 0, "note": "this conversation has no sandbox yet: run_python or run_bash makes it"} - with _gateway(cfg, driver) as d: - _ready(db, cfg, d, chat, account) + at = _place(db, chat) + with _at(db, cfg, at, driver) as d: + _ready(db, cfg, d, chat, account, at) ran = d.exec(name_for(chat), ["python3", "-c", _LIST, str(LIST_MAX)], None, settings(cfg)["seconds"], MAX_OUTPUT) if ran.exit_code != 0: raise SandboxError(f"the sandbox couldn't list its files: {_text(ran.stderr)[-300:].strip() or 'no reason given'}") @@ -826,8 +1044,9 @@ def fetch(db, cfg, chat, account, path, driver=None): p = _path(path) if not _row(db, chat): raise SandboxError("this conversation has no sandbox yet") - with _gateway(cfg, driver) as d: - _ready(db, cfg, d, chat, account) + at = _place(db, chat) + with _at(db, cfg, at, driver) as d: + _ready(db, cfg, d, chat, account, at) ran = d.exec(name_for(chat), ["python3", "-c", _CAT, p, str(FILE_BYTES)], None, settings(cfg)["seconds"], FILE_BYTES + 1) if ran.exit_code == 3: raise SandboxError(f"the sandbox has no file {p}: see sandbox_files") @@ -919,9 +1138,11 @@ def forget(db, cfg, chat, driver=None): row = _row(db, chat) if not row: return False + node = _node_of(db, row) try: - with _gateway(cfg, driver) as d: - d.delete(row.get("name") or name_for(chat)) + if row.get("node") is None or node is not None: # a removed node took its sandboxes with it + with _at(db, cfg, node, driver) as d: + d.delete(row.get("name") or name_for(chat)) except SandboxError as e: log.info("sandbox: %s; it's deleted later", e) db.q("UPDATE $r SET used_at = $t", r=R("code_sandbox", int(chat)), t="1970-01-01T00:00:00+00:00") @@ -940,7 +1161,7 @@ def sweep(db, cfg, driver=None, every=SWEEP_SECONDS): if every and now - _swept["at"] < every: return 0 _swept["at"] = now - rows = db.rows("SELECT record::id(id) AS chat, name, state, used_at FROM code_sandbox") + rows = db.rows("SELECT record::id(id) AS chat, name, node, state, used_at FROM code_sandbox") if not rows: return 0 s = settings(cfg) @@ -950,17 +1171,28 @@ def sweep(db, cfg, driver=None, every=SWEEP_SECONDS): if not due: return 0 changed = 0 - try: - with _gateway(cfg, driver) as d: - for r in due: - name = r.get("name") or name_for(r["chat"]) - if (r.get("used_at") or "") < old: - d.delete(name) - db.q("DELETE $r", r=R("code_sandbox", int(r["chat"]))) - else: - d.stop(name) - db.q("UPDATE $r SET state = 'stopped'", r=R("code_sandbox", int(r["chat"]))) + nodes = {str(n["id"]): n for n in _all(db)} + places: dict[Any, list] = {} # by where they are: this machine (None), or a node's id + for r in due: + places.setdefault(r.get("node"), []).append(r) + for at, these in places.items(): + node = None if at is None else nodes.get(str(at)) + if at is not None and node is None: # its node was removed from the mesh, and its sandboxes with it + for r in these: + db.q("DELETE $r", r=R("code_sandbox", int(r["chat"]))) changed += 1 - except SandboxError as e: - log.info("sandbox: the sweep stopped: %s", e) + continue + try: + with _at(db, cfg, node, driver) as d: + for r in these: + name = r.get("name") or name_for(r["chat"]) + if (r.get("used_at") or "") < old: + d.delete(name) + db.q("DELETE $r", r=R("code_sandbox", int(r["chat"]))) + else: + d.stop(name) + db.q("UPDATE $r SET state = 'stopped'", r=R("code_sandbox", int(r["chat"]))) + changed += 1 + except SandboxError as e: + log.info("sandbox: the sweep stopped%s: %s", f" on {_named(node)}" if node else "", e) return changed diff --git a/fastapi_backend/tests/api/test_sandbox.py b/fastapi_backend/tests/api/test_sandbox.py index ee7c73bca..bf1dd8fcd 100644 --- a/fastapi_backend/tests/api/test_sandbox.py +++ b/fastapi_backend/tests/api/test_sandbox.py @@ -560,3 +560,83 @@ def __init__(self, address, tls=None, timeout=None): tls = seen["tls"] assert (tls.ca_path, tls.cert_path, tls.key_path) == tuple(tmp_path / n for n in ("ca.crt", "tls.crt", "tls.key")) assert isinstance(sandbox.landlock_abi(), int | None) # this machine's own, read without failing + + +def test_code_runs_on_a_compute_node_the_mesh_registered(app, db, cfg, folder, new_client, llm, gateway): + s = Assist(app, db, cfg, folder, new_client) + c, h = s.cl["editor"] + cid = c.post("/api/v1/chats", headers=h, json={}).json()["id"] + name = f"lens-chat-{cid}" + + def said(llm): # what the tools gave the model (a long conversation's summary is asked for after the answer) + last = [r for r in llm.seen if any(m["role"] == "tool" for m in r["messages"])][-1] + return [json.loads(unwrapped(m["content"])) for m in last["messages"] if m["role"] == "tool"] + + # without nodes, nothing changes: no sandbox_nodes, and nowhere else to run + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "echo here > a.txt"})]}, {"content": "Ok."}]) + asked = next(r for r in llm.seen if r.get("tools")) + run = next(t for t in asked["tools"] if t["function"]["name"] == "run_bash") + assert "sandbox_nodes" not in offered(llm) and "node" not in run["function"]["parameters"]["properties"] + assert name in gateway.boxes + + srv, node_gw, addr = fake_openshell.start(folder / "node-gateway") + now = dt.datetime.now(dt.UTC).isoformat(timespec="seconds") + long_ago = (dt.datetime.now(dt.UTC) - dt.timedelta(hours=2)).isoformat(timespec="seconds") + up = {"nebula": True, "sandbox": {"version": "1"}, "machine": "x86_64, 32 GB"} + nodes = [ + {"id": 1, "name": "lens", "role": "primary", "ip": "10.42.0.1", "last_seen": now, "status": up}, + {"id": 2, "name": "gpu", "role": "compute", "ip": "10.42.0.2", "groups": ["gpu"], "last_seen": now, "status": up}, + {"id": 3, "name": "old", "role": "compute", "ip": "10.42.0.3", "last_seen": long_ago, "status": up}, + {"id": 4, "name": "gone", "role": "compute", "ip": "10.42.0.4", "last_seen": now, "status": up, "revoked_at": now}, + ] + reached = [] + + def driver_fn(db_, cfg_, node): + reached.append(node["ip"]) + return sandbox.OpenShell({**cfg_, "sandbox": {**cfg_["sandbox"], "gateway": addr}}) + + sandbox.register_nodes(lambda db_: nodes, driver_fn) + try: + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "sandbox_nodes", {})]}, {"content": "Ok."}]) + listed = {n["name"]: n for n in said(llm)[0]["nodes"]} + assert set(listed) == {"this machine", "gpu", "old"} and listed["this machine"]["has_sandbox"] + assert listed["gpu"]["ready"] and listed["gpu"]["machine"] == "x86_64, 32 GB" and not listed["gpu"]["has_sandbox"] + assert not listed["old"]["ready"] and listed["old"]["why_not"].startswith("it hasn't reported since") + assert "compute nodes" in llm.seen[-1]["messages"][0]["content"] + + # asked to run on gpu, it moves there: a fresh sandbox, reached through the node's own gateway + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "ls", "node": "gpu"})]}, {"content": "Ok."}]) + ran = said(llm)[0] + assert (ran["node"], ran["stdout"]) == ("gpu", "") and ran["sandbox"].startswith("a fresh sandbox on gpu") + assert name in node_gw.boxes and name not in gateway.boxes and reached[-1] == "10.42.0.2" + assert db.one("SELECT node FROM $r", r=R("code_sandbox", cid))["node"] == 2 + last = db.rows("SELECT detail, at FROM activity WHERE action = 'sandbox.run' ORDER BY at DESC LIMIT 1")[0] + assert last["detail"]["node"] == 2 + + # and stays there + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "echo gpu > b.txt\nls"})]}, {"content": "Ok."}]) + assert said(llm)[0]["stdout"] == "b.txt\n" and "sandbox" not in said(llm)[0] + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "sandbox_files", {})]}, {"content": "Ok."}]) + assert [f["path"] for f in said(llm)[0]["files"]] == ["b.txt"] + + # a node that can't run code says why; one removed or unknown isn't there + for which, why in (("old", "old can't run code now: it hasn't reported since"), ("gone", "there's no node gone")): + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "ls", "node": which})]}, {"content": "Ok."}]) + assert why in said(llm)[0]["error"] + + # back here: the node's sandbox goes, and a fresh one is made on this machine + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "ls", "node": "lens"})]}, {"content": "Ok."}]) + assert said(llm)[0]["sandbox"].startswith("a fresh sandbox here") and "node" not in said(llm)[0] + assert name not in node_gw.boxes and name in gateway.boxes + + # an idle sandbox on a node is stopped there, and deleting the conversation deletes it there + ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "ls", "node": "2"})]}, {"content": "Ok."}]) + t = (dt.datetime.now(dt.UTC) - dt.timedelta(minutes=16)).isoformat(timespec="seconds") + db.q("UPDATE $r SET used_at = $t", r=R("code_sandbox", cid), t=t) + assert sandbox.sweep(db, app.state.settings.current(), every=0) == 1 + assert node_gw.boxes[name]["phase"] == fake_openshell.STOPPED + assert c.delete(f"/api/v1/chats/{cid}", headers=h).status_code == 200 + assert name not in node_gw.boxes and not db.one("SELECT * FROM $r", r=R("code_sandbox", cid)) + finally: + sandbox.register_nodes(None, None) + srv.stop(0) diff --git a/nextjs-frontend/components/chat/steps.tsx b/nextjs-frontend/components/chat/steps.tsx index e3efe054c..2043d97dd 100644 --- a/nextjs-frontend/components/chat/steps.tsx +++ b/nextjs-frontend/components/chat/steps.tsx @@ -21,6 +21,7 @@ import { Quote, Search, Send, + Server, Shapes, SquareTerminal, Waypoints, @@ -49,6 +50,7 @@ const ICON: Record = { propose_topic_change: PencilLine, run_python: SquareTerminal, run_bash: SquareTerminal, + sandbox_nodes: Server, share_sandbox_file: Download, search_mail: Mail, read_mail: Mail, diff --git a/threat-model.json b/threat-model.json index 4425b71e9..089b026c4 100644 --- a/threat-model.json +++ b/threat-model.json @@ -710,10 +710,10 @@ }, { "entry_point": "code sandbox", - "parameter": "assistant run_python, run_bash, sandbox_files, share_sandbox_file and import_sandbox_file: code the model writes, attachment ids, a path in the sandbox; sandbox.* settings (admins)", + "parameter": "assistant run_python, run_bash, sandbox_files, sandbox_nodes, share_sandbox_file and import_sandbox_file: code the model writes, attachment ids, a path in the sandbox, a compute node's id or name; sandbox.* settings (admins)", "attacker_controllable": "yes", "component": "assistant", - "caller_must_enforce": "none — off by default; the code runs in the conversation's own OpenShell sandbox with none of Lens's data or keys and no network unless sandbox.network allows it (P-SANDBOX); with the network on, a run after outside text asks first (P-INJECTION-EGRESS); files in only from the person's attachments, out only through an approved import or as a download for that person", + "caller_must_enforce": "none — off by default; the code runs in the conversation's own OpenShell sandbox with none of Lens's data or keys and no network unless sandbox.network allows it (P-SANDBOX); with the network on, a run after outside text asks first (P-INJECTION-EGRESS); files in only from the person's attachments, out only through an approved import or as a download for that person; with compute nodes registered by the mesh, a run may go to a ready compute node's own gateway, reached only over the mesh, with the same sandbox settings, and only where the model passed node or the conversation's sandbox already is", "provenance": "documented", "source": "docs/assistant.md \"Code sandbox\"" }, diff --git a/threat-model.md b/threat-model.md index ef87eeaf6..ac2841f15 100644 --- a/threat-model.md +++ b/threat-model.md @@ -326,7 +326,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `PUT /api/v1/admin/sandbox`, MCP `set_sandbox_gateway` | `on` | admin only (MCP: with the write scope) | x-env-config | admin is trusted; audited (`lens.sandbox`); refused when the machine can't run the gateway (preflight); reaches the machine only as an inbox request (`action=services`, `profile=sandbox`) the agent checks again; off deletes every sandbox first | *(documented, docs/assistant.md "The gateway")* | | MCP `edit_entity`, `edit_topic`, `merge_topics`, `delete_topic`, `undo_topic_change`, `undo_graph_change`; assistant `propose_topic_change` | entity and topic ids, names, labels, definitions | yes, an agent with an editor's write-scope token; indirectly through content the model reads | data, x-prompt-injection | the curate permission in each namespace touched and a write-capable principal (the same route functions as the web app); the assistant's changes wait for the person's approval; merges and deletes are kept to undo, and audited | *(documented, docs/mcp.md "curate entities and topics as an editor does")* | | `/api/v1/automations` (save, try, run now, decide, draft from a sentence), the MCP and assistant automation tools; the activities an automation runs on | triggers, `where` and If match trees, the event workflow graph (action params are Jinja templates), `policy`, `limits`, an activity id; the text of the activities themselves | yes: any editor, for automations in namespaces they edit; indirectly through activity text (mail, titles, chat) that reaches action params | callback/code (declarative), data, x-prompt-injection | none; it runs as its maker with their permissions at run time (P-AUTOMATION-ACTOR), actions reaching outside Lens ask first, outside text marks intents untrusted and breaks up links (P-INJECTION-EGRESS), and chains and bursts are bounded (P-AUTOMATION-LOOP) | *(documented, docs/automations.md "Running automations")* | -| Assistant code sandbox tools (`run_python`, `run_bash`, `sandbox_files`, `share_sandbox_file`, `import_sandbox_file`) and the `sandbox.*` settings | code and scripts the model writes, attachment ids, a path in the sandbox; the gateway address, image and hosts it may reach (admins) | yes: the people `sandbox.who` names, and indirectly through content the model reads | callback/code, data, x-prompt-injection | none; off by default; the code runs in the conversation's own OpenShell sandbox, which holds none of Lens's data or keys and reaches no network unless `sandbox.network` allows it (P-SANDBOX); with the network on, a run after outside text asks first (P-INJECTION-EGRESS); files go in only from the person's own attachments to that conversation, and come out only as an import the person approves or as a download for that person (`share_sandbox_file`) | *(documented, docs/assistant.md "Code sandbox")* | +| Assistant code sandbox tools (`run_python`, `run_bash`, `sandbox_files`, `sandbox_nodes`, `share_sandbox_file`, `import_sandbox_file`) and the `sandbox.*` settings | code and scripts the model writes, attachment ids, a path in the sandbox, a compute node's id or name; the gateway address, image and hosts it may reach (admins) | yes: the people `sandbox.who` names, and indirectly through content the model reads | callback/code, data, x-prompt-injection | none; off by default; the code runs in the conversation's own OpenShell sandbox, which holds none of Lens's data or keys and reaches no network unless `sandbox.network` allows it (P-SANDBOX); with the network on, a run after outside text asks first (P-INJECTION-EGRESS); files go in only from the person's own attachments to that conversation, and come out only as an import the person approves or as a download for that person (`share_sandbox_file`); with compute nodes registered by the mesh, a run may go to a ready compute node's own gateway, reached only over the mesh, with the same sandbox settings, and only where the model passed `node` or the conversation's sandbox already is | *(documented, docs/assistant.md "Code sandbox")* | | `GET /api/v1/chats/{cid}/files/{fid}`: a file the code sandbox shared (`share_sandbox_file`) | a file id; the file's bytes, which code the model wrote produced | yes: the person whose conversation it is; indirectly through content the model reads | data, x-prompt-injection | none; served only to the conversation's person (404 for anyone else, or for an id of another conversation) as an `attachment` with `nosniff` and a `sandbox` content policy, never rendered by Lens; the id is random (96 bits) and checked against the conversation; kept encrypted (AES-256-GCM with a key derived from the server's secret, bound to the file and its conversation), at most 20 a conversation and 25 MB each, and deleted with the conversation or the person's account | *(documented, docs/assistant.md "Code sandbox")* | | assistant and MCP `lens_help` | a question's words, a docs page's name | yes, any signed-in person or agent with a read token; indirectly through content the model reads | data, x-prompt-injection | it only reads: the docs come from a fixed folder (a page is picked by name from the ones indexed, never opened by path), searched in an in-memory index; this server's facts are the version, addresses, on/off switches, the chat model's name and the asker's own namespaces, never keys, tokens or model server addresses, and the settings page of each only for admins | *(documented, docs/assistant.md "Questions about Lens itself")* | | Deep research: `/research `, `research: true` on a question, `POST /chats/{cid}/research`, `/research/{id}` (revise, stop, resume, note, podcast), the assistant's and MCP's `start_research`, MCP's `research_status`, and the `research.*` settings | a goal, answers to follow-up questions, plan edits (sub-questions, sources, tool calls, outline); settings (admins) | yes: anyone who may use the assistant, in their own conversations; indirectly through what the run reads, through the model | data, x-prompt-injection, size | none; the plan waits for the person's own approval (P-RESEARCH); the run uses only the read tools of its plan's sources, as the person with their current access, within its tool calls, minutes and budgets; its web searches and page reads don't ask again, bounded by the plan and kept in the ledger; one run per person at a time; the report's claims are checked against what they cite and marked when unsupported | *(documented, docs/assistant.md "Deep research")* | diff --git a/threat-model.yaml b/threat-model.yaml index d71ef7eb7..adec49c48 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -679,11 +679,11 @@ entry_points: - id: "code sandbox" component: assistant parameters: - - name: "assistant run_python, run_bash, sandbox_files, share_sandbox_file and import_sandbox_file: code the model writes, attachment ids, a path in the sandbox; sandbox.* settings (admins)" + - name: "assistant run_python, run_bash, sandbox_files, sandbox_nodes, share_sandbox_file and import_sandbox_file: code the model writes, attachment ids, a path in the sandbox, a compute node's id or name; sandbox.* settings (admins)" attacker_controllable: true x-condition: "the people sandbox.who names (editors and admins by default); indirectly through content the model reads" control_kinds: [callback/code, data, x-prompt-injection] - caller_must_enforce: "none — off by default; the code runs in the conversation's own OpenShell sandbox with none of Lens's data or keys and no network unless sandbox.network allows it (P-SANDBOX); with the network on, a run after outside text asks first (P-INJECTION-EGRESS); files in only from the person's attachments, out only through an approved import or as a download for that person" + caller_must_enforce: "none — off by default; the code runs in the conversation's own OpenShell sandbox with none of Lens's data or keys and no network unless sandbox.network allows it (P-SANDBOX); with the network on, a run after outside text asks first (P-INJECTION-EGRESS); files in only from the person's attachments, out only through an approved import or as a download for that person; with compute nodes registered by the mesh, a run may go to a ready compute node's own gateway, reached only over the mesh, with the same sandbox settings, and only where the model passed node or the conversation's sandbox already is" provenance: {kind: documented, source: 'docs/assistant.md "Code sandbox"'} - id: "code sandbox shared files" component: assistant From 61be5127f83fb60b835c47c631bf4381eb0a4cb7 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sun, 18 Oct 2026 07:02:25 +0000 Subject: [PATCH 2/3] style: format the compute node sandbox test Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01GbBS222wEEQgkuui6CHo6c --- fastapi_backend/tests/api/test_sandbox.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fastapi_backend/tests/api/test_sandbox.py b/fastapi_backend/tests/api/test_sandbox.py index bf1dd8fcd..ecf15d016 100644 --- a/fastapi_backend/tests/api/test_sandbox.py +++ b/fastapi_backend/tests/api/test_sandbox.py @@ -621,7 +621,9 @@ def driver_fn(db_, cfg_, node): # a node that can't run code says why; one removed or unknown isn't there for which, why in (("old", "old can't run code now: it hasn't reported since"), ("gone", "there's no node gone")): - ask(c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "ls", "node": which})]}, {"content": "Ok."}]) + ask( + c, h, cid, llm, [{"content": "", "tool_calls": [call(1, "run_bash", {"command": "ls", "node": which})]}, {"content": "Ok."}] + ) assert why in said(llm)[0]["error"] # back here: the node's sandbox goes, and a fresh one is made on this machine From ece8ecef837e2b9a21df97c5e2aadabfeb6c88d1 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sun, 18 Oct 2026 07:52:29 +0000 Subject: [PATCH 3/3] docs(readme): list the assistant's answers about Lens itself as built v0.15.0 released #400, so the requirements check fails while it stays under In progress. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01GbBS222wEEQgkuui6CHo6c --- README.md | 8 ++++---- docs/requirements.md | 1 + 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 05cf49bf0..49749dc9e 100644 --- a/README.md +++ b/README.md @@ -88,11 +88,13 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p cost), opened from its header. It reads answers aloud with a speech provider, macOS `say` or StyleTTS2 on this machine (opt-in), any message in one click. Long conversations are summarised before they fill the model's context, and `/clear` and `/compact` start afresh or summarise now. Conversations title themselves after their first answer, unless - you name them. Each namespace can have its own assistant with + you name them. It answers questions about Lens itself too, from its docs and this server's settings, citing both. + Each namespace can have its own assistant with its own name, instructions and memory, answering in Telegram and Slack (built in, with voice notes and files), in Discord, Matrix, WhatsApp and other chat rooms through Matterbridge, and by email at an address of its own that you email or forward things to. ([Chat](docs/assistant.md), [web search](docs/assistant.md#web-search-and-reading-pages), - [mail, calendars and files](docs/assistant.md#mail-calendars-and-files), [code sandbox](docs/assistant.md#code-sandbox), [answer records](docs/assistant.md#what-each-answer-records), + [mail, calendars and files](docs/assistant.md#mail-calendars-and-files), [code sandbox](docs/assistant.md#code-sandbox), + [Lens itself](docs/assistant.md#questions-about-lens-itself), [answer records](docs/assistant.md#what-each-answer-records), [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), [speech providers](docs/speech-providers.md); [checks](docs/requirements.md#ask-and-get-cited-answers)) - **Extend the assistant.** Add tools (a prompt, a web request, a graph drawn on the canvas, or sandboxed Python), @@ -241,8 +243,6 @@ still loading. ([Components](docs/components.md), These are open pull requests, not yet part of Lens: -- **The assistant** answering questions about Lens itself, from its docs and this server - ([#400](https://github.com/adeelahmad/lense/pull/400)). - **Chat:** a conversation exported as Markdown, PDF or JSON ([#474](https://github.com/adeelahmad/lense/pull/474)). - **Pictures:** low-quality images enhanced before they're read ([#479](https://github.com/adeelahmad/lense/pull/479)). - **Web clipper:** signing in, or using your API token, not only an import webhook's diff --git a/docs/requirements.md b/docs/requirements.md index 18ce5ad34..9ba51c2d3 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -90,6 +90,7 @@ request, to another check that proves the same thing; loosening a requirement or | Any message read aloud in one click, one at a time, its audio kept so a replay costs nothing | [chat-read-aloud.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-read-aloud.test.tsx) `plays an answer by the speech model in one click, stops on the next, and keeps its audio`, `reads one message at a time: starting another stops the first`, `reads a long answer piece by piece, asking for the next while one plays` | | Conversations title themselves after their first answer, and a name you give is kept | [test_chat_titles.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chat_titles.py) `test_a_first_answer_titles_the_conversation`, `test_a_title_you_give_is_kept`, `test_agents_rename_and_retitle_over_mcp` | | Long conversations summarised before they fill the model's context, and `/clear` and `/compact` | [test_compaction.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_compaction.py) `test_older_messages_are_folded_into_a_summary`, `test_the_context_window_is_the_model_servers`, `test_a_long_conversation_is_folded_in_runs_that_fit_the_summary_model`, `test_a_conversation_that_doesnt_fit_is_folded_in_and_asked_again`, `test_what_tools_returned_is_cut_to_fit_the_model`, `test_clear_starts_afresh_and_compact_folds_everything_now`, `test_commands_in_the_app`; [chat-summary-mark.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-summary-mark.test.tsx) `says how many messages the summary stands for, and shows it on request` | +| It answers questions about Lens itself, from its docs and this server's settings, citing both, for agents over MCP too | [test_lens_help.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_lens_help.py) `test_the_assistant_answers_about_lens_itself`, `test_the_docs_are_found_by_words`, `test_admins_see_where_each_setting_is`, `test_agents_ask_lens_over_mcp` | | A namespace's own assistant, with its own name, instructions and memory | [test_ns_assistant.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_ns_assistant.py) `test_off_by_default_and_owners_turn_it_on`, `test_remembers_across_conversations_with_citations`; [ns-assistant.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/ns-assistant.test.tsx) `saving a name and forgetting a memory` | | Telegram and Slack built in, with voice notes and files | [test_telegram.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_telegram.py) `test_it_answers_private_chats_and_what_is_said_to_it_in_groups`, `test_voice_notes_are_heard`, `test_files_are_held_for_the_assistant_to_import`; [test_slack.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_slack.py) `test_it_answers_direct_messages_mentions_and_the_slash_command`, `test_voice_clips_are_heard_and_files_held` | | Other chat rooms through Matterbridge | [test_bridge.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_bridge.py) `test_it_answers_what_is_said_to_it_and_nothing_else`; [test_matterbridge_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_matterbridge_run.py) `test_chat_networks_are_set_in_the_app_and_become_matterbridges_config`, `test_the_bridge_talks_to_the_matterbridge_lens_runs`; [matterbridge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/matterbridge.yml) `Lens writes a config, and Matterbridge starts with it` |