Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,8 +248,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
Lens joins it over a private network of its own, a Nebula mesh it issues the certificates for: one install line
with a one-time token, and the machine needs nothing else. Worker nodes run jobs as their own user of the
database, never with Lens's keys; compute nodes run the assistant's code on a sandbox gateway of their own.
Settings › Nodes and Network show each machine and the mesh, and agents do the same over the API. Not yet tried
between two real machines. ([Nodes and the mesh](docs/mesh.md); [checks](docs/requirements.md#run-on-more-machines))
Settings › Nodes and Network show each machine and the mesh, and agents do the same over the API, MCP and the
assistant. Not yet tried between two real machines. ([Nodes and the mesh](docs/mesh.md);
[checks](docs/requirements.md#run-on-more-machines))
- **Keep it up to date and backed up.** Lens checks for a new release once a day and tells admins by email or a
notification. One click in Settings › Updates (or one API call) upgrades it through a small agent on the machine,
showing each step, and rolls back when the new version doesn't start; nothing installs unless an admin says so. The
Expand Down
7 changes: 7 additions & 0 deletions changelog.d/mesh-tools.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
- **Agents manage the mesh's machines** (in progress, as the mesh is). Over MCP and in the assistant, an admin's agent
reads the mesh (`mesh`: its CA, this machine's address, every machine and how it's doing, the join tokens not used
yet, never their secrets) and adds a machine (`add_mesh_node`, which answers with the install line), renames one or
changes its namespaces, takes one off and brings it back, and withdraws a join token. In the assistant, adding a
machine and taking one off always wait for your yes, and the install line is shown to you once on the approval
card, never kept or sent to the model; the rest happens at once. Over MCP every change needs an admin's
write-scope token and is audited ([Agents](docs/mesh.md#agents)).
9 changes: 7 additions & 2 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -980,12 +980,17 @@ Admins also get the server tools: `server_status` (what's set up and what's miss
and the items they can refer to, never a value), the news feed tools: `news_feeds`, `subscribe_feed` (follow a feed into a namespace in one step,
[News feeds](processing.md#news-feeds-rss-and-atom)) and `unsubscribe_feed`, and the people tools: `list_people`, `invite_person` (an account, optionally with a role in one
namespace; Lens emails the sign-in link when mail is set up) and `change_person` (a role in a namespace, or disabling
and enabling an account). Removing an account for good stays in the web app and the API. In an ordinary conversation
and enabling an account). Removing an account for good stays in the web app and the API. The mesh tools manage the
machines Lens runs on ([Nodes and the mesh](mesh.md#agents)): `mesh` reads it, `add_mesh_node` makes the install line
for another machine (shown to the person on the approval card, once), `remove_mesh_node` takes one off, and
`change_mesh_node`, `restore_mesh_node` and `withdraw_mesh_token` rename a machine or change its namespaces, bring a
removed one back, and withdraw a join token, at once. The [auto scaling](scaling.md#agents) tools are here too. In an ordinary conversation
their changes are approval cards. In a **setup conversation**
(`POST /api/v1/chats` with `"kind": "setup"`, or **Finish with the assistant** in the setup wizard once a model is
connected) the admin has asked the assistant to set the server up, so it makes the changes itself and says what it
changed. Every change is audited with `assistant` in its detail, and all of them stay changeable in Settings.
Turning telemetry on or off, changing the Secrets section, inviting people and changing their access always ask first, and so does every change once the answer has read text from outside
Turning telemetry on or off, changing the Secrets section, inviting people and changing their access, and adding a
machine to the mesh or taking one off always ask first, and so does every change once the answer has read text from outside
Lens ([third-party content](#third-party-content-and-prompt-injection)).

### Routine choices
Expand Down
13 changes: 10 additions & 3 deletions docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ Each namespace can have its own server, at `/mcp/<namespace>` (`https://lens.exa
that should see that namespace and nothing else: a family assistant, a project's coding agent, a client's app. It
has the same tools, but sees only that namespace, whatever else the person who signs the agent in can read. Tools
that take a namespace use this one when none is named, `list_namespaces` and `list_files` show only it, and the tools
about the server itself (`file_storage`, `monitoring`, `services`, `mcp_servers`, `sandbox_gateway`, `install_settings`
and the ones that change them) aren't there, even for admins. Nor are the person's own conversations (`list_chats`,
about the server itself (`file_storage`, `monitoring`, `services`, `mcp_servers`, `sandbox_gateway`, `install_settings`,
`mesh` and the ones that change them) aren't there, even for admins. Nor are the person's own conversations (`list_chats`,
`read_chat`, `rename_chat`, `export_chat`) and projects (`list_projects`, `read_project`, `create_project`,
`update_project`, `schedule_project`), which belong to no namespace. An admin counts as an owner of the namespace there.

Expand Down Expand Up @@ -164,6 +164,11 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
| `install_plugin` | install one as your own extension, private until you share it, or bring yours to the version listed now; `preview` installs nothing and says what it would add and what Lens leaves out. Needs a write-scope token |
| `install_settings` | whether other devices on the network reach Lens and its edition (full or lean), whether the machine's upgrade agent can change them, and the install line to run when it can't. Admins only |
| `set_install_settings` | open Lens to the network or keep it to this machine, or change the edition: the machine sets itself up again and Lens restarts ([Updates](updates.md#setting-the-machine-up-again)). Admins only, with a write-scope token |
| `mesh` | Lens's network between its machines ([Nodes and the mesh](mesh.md#agents)): whether it's on, its CA, this machine's address, every machine and how it's doing, and the join tokens not used yet, never their secrets. Admins only |
| `add_mesh_node` | add a machine: a join token and the install line to run on it, shown once; turns the mesh on. Admins only, with a write-scope token |
| `change_mesh_node` | rename a machine or change the namespaces it serves (`every_namespace` for every one). Admins only, with a write-scope token |
| `remove_mesh_node` | take a machine off the mesh; `restore_mesh_node` brings it back while its certificate is good. Admins only, with a write-scope token |
| `withdraw_mesh_token` | withdraw a join token not used up yet. Admins only, with a write-scope token |
| `news_feeds` | the news feeds (RSS or Atom) Lens follows, the namespaces each brings posts into and how the last scan went; addresses stay sealed. Admins only |
| `subscribe_feed` | follow a news feed into a namespace in one step: its address or a page of the site, read first and kept only if it works; `backfill` also brings in the posts it lists now ([News feeds](processing.md#news-feeds-rss-and-atom)). Admins only, with a write-scope token |
| `unsubscribe_feed` | stop following a news feed; the posts it brought in stay. Admins only, with a write-scope token |
Expand Down Expand Up @@ -192,7 +197,9 @@ queues an episode, `create_feed` and `publish_to_feed`, which make and fill a [p
editors), `enhance`, which enhances an image and reads it again (the process permission),
`set_namespace_integrations`, which turns a namespace's own MCP server or chat completions on or off (owners), `set_picture_descriptions`, which changes what describes pictures (admins), `set_file_storage`, which changes where new files are kept (admins), `set_secret_reference`, which points a secret field at a password manager's item (admins), `install_plugin`, which
installs a plugin from a marketplace as your own extension, `add_mcp_server`, which adds an
MCP server (admins), `subscribe_feed` and `unsubscribe_feed`, which follow and stop following news feeds (admins), `set_mail_routing`, which changes where an email account's messages go (admins), and `set_monitoring`, `set_service`,
MCP server (admins), `subscribe_feed` and `unsubscribe_feed`, which follow and stop following news feeds (admins), `set_mail_routing`, which changes where an email account's messages go (admins), `add_mesh_node`,
`change_mesh_node`, `remove_mesh_node`, `restore_mesh_node` and `withdraw_mesh_token`, which manage the machines on
the [mesh](mesh.md#agents) (admins), and `set_monitoring`, `set_service`,
`set_sandbox_gateway` (the [code sandbox](assistant.md#code-sandbox)'s gateway) and `set_install_settings`, which have the machine change how Lens runs (admins).
Importing stays in the web app and the API.

Expand Down
26 changes: 24 additions & 2 deletions docs/mesh.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,17 @@ Status: **in progress**. Built: Lens as the mesh's certificate authority, join t
token, each machine's Nebula config, a node's renewals and reports, removing a node, the `mesh` settings, the API
under `/api/v1/mesh`, Nebula running on the primary in the `mesh` container, relaying the database and the API,
`lens node` with the install line that joins another machine, worker nodes running jobs, compute nodes running the
assistant's code, and Settings › Network and Nodes in the web app. Not yet tried between two real machines.
assistant's code, Settings › Network and Nodes in the web app, and the same over MCP and in the assistant
([Agents](#agents)). Not yet tried between two real machines.

Lens runs on one machine until you add another: a Mac with a GPU to transcribe on, a box to run the assistant's code
on, a server with more disk. Those machines talk to this install over a private network of Lens's own, a
[Nebula](https://github.com/slackhq/nebula) mesh (MIT), never over the LAN or the internet in the clear: every packet
between them is encrypted and signed with certificates Lens issued, and Nebula's firewall lets through only what each
role needs. A single-machine install never turns it on.

**Off until an admin adds a node.** Adding a node (Settings › Nodes, or `POST /api/v1/mesh/tokens`) makes the mesh's
**Off until an admin adds a node.** Adding a node (Settings › Nodes, `POST /api/v1/mesh/tokens`, or an agent's
`add_mesh_node`) makes the mesh's
certificate authority the first time and turns `mesh.enabled` on. Nothing goes in `.env`; nothing is asked at
install.

Expand Down Expand Up @@ -222,6 +224,26 @@ In the web app:
asks nothing: the row stays, dimmed and marked Removed, with Undo while its certificate is good. Join tokens not
used yet are listed under it, each with Withdraw. The list refreshes every 15 seconds.

## Agents

The same for an admin's agent, over [MCP](mcp.md) and in the [assistant](assistant.md#setting-up-and-running-the-server-by-chat)
(admins, in the server tools):

| Tool | | Over MCP | In the assistant |
| --- | --- | --- | --- |
| `mesh` | whether it's on, the CA's fingerprint and expiry, this machine's mesh address, every machine (id, name, role, address, online and last seen, Nebula, its Lens version and machine, the namespaces it serves, on a compute node whether its sandbox gateway runs, its error, when it was removed) and the join tokens not used yet, never their secrets | reads | reads |
| `add_mesh_node` | a join token and the install line for another machine (`role` worker or compute, optionally `name`, `namespaces`, `hours`, `uses`), shown once; turns the mesh on when it's off | at once, answering with the line | always waits for the person's yes; the line is shown to them on the approval card, once |
| `change_mesh_node` | renames a machine or changes the namespaces it serves | at once | at once |
| `remove_mesh_node` | takes a machine off the mesh | at once | always waits for the person's yes |
| `restore_mesh_node` | brings a removed machine back while its certificate is good | at once | at once |
| `withdraw_mesh_token` | withdraws a join token not used up yet | at once | at once |

Over MCP the changes need an admin's token with the write scope; they're audited as the web app's are, with `mcp` in
the detail (`mesh.token.create`, `mesh.node.change`, `mesh.node.remove`, `mesh.node.restore`, `mesh.token.withdraw`),
and the install line uses the address the agent called Lens at. In the assistant the install line uses the web app's
address the person is on, and it isn't kept: not in the approval, the audit log or anything the model is sent. What
happens at once still asks first once the answer has read text from outside Lens.

## What keeps it safe

- Nothing listens until an admin adds a node, and single-machine installs never do.
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,7 @@ request, to another check that proves the same thing; loosening a requirement or
| A worker node runs jobs as its own user of Lens's database, never root and never with the archive's secret, reading only its namespaces' files over the mesh, never a vault's | [test_mesh_workers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mesh_workers.py) `test_a_worker_node_runs_lens_worker_as_its_own_user_of_the_database`, `test_its_jobs_read_files_from_here_over_the_mesh`, `test_a_node_serving_some_namespaces_takes_only_their_jobs`, `test_only_a_worker_node_gets_the_database`, `test_a_node_runs_only_the_steps_a_node_can` |
| A compute node runs the assistant's code on a gateway of its own, which Lens reaches over the mesh with mutual TLS, holding its certificate sealed | [test_mesh_compute.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mesh_compute.py) `test_a_compute_nodes_install_line_starts_its_gateway`, `test_a_compute_node_hands_over_its_gateway_and_the_assistant_may_run_code_there`, `test_only_a_compute_node_gives_a_gateway_and_only_a_certificate`, `test_lens_reaches_the_nodes_gateway_with_its_certificate_checked_by_name` |
| Settings › Nodes adds a machine with Enter, shows its install line once, lists each machine with how it's doing, renames in place and removes with Undo; Settings › Network shows the mesh and its settings | [settings-mesh.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/settings-mesh.test.tsx) `adds a node with Enter and shows the install line once, with its terms`, `lists each machine with how it's doing`, `removes a node without asking, and undoes it`, `renames a node with Enter`, `says whether a compute node's sandbox gateway runs`, `shows that it runs, and what Nebula said`, `checks the network and the hosts as the server does` |
| Agents read the mesh and add, rename, remove, restore and withdraw over MCP and in the assistant; adding or removing a machine in the assistant waits for the admin's yes, and a join token's secret never reaches the model | [test_mesh_agents.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mesh_agents.py) `test_mcp_clients_add_and_manage_mesh_nodes`, `test_the_assistant_adds_and_removes_machines_once_the_person_approves`, `test_the_assistant_renames_restores_and_withdraws_at_once`, `test_a_setup_conversation_still_asks_before_a_machine_joins_or_goes`, `test_only_admins_get_the_mesh_tools`; [approval-install-line.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/approval-install-line.test.tsx) `shows its install line this once, until the person is done with it` |

## Keep it up to date and backed up

Expand Down
Loading
Loading