Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/mcp-runner.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: MCP runner

# The MCP runner (the optional mcp service, mcp-runner/): it builds, starts as docker-compose runs it (read-only, no
# capabilities), writes its token to the volume it shares with Lens, and runs real stdio MCP servers from npm and PyPI.
on:
pull_request:
paths:
- mcp-runner/**
- .github/workflows/mcp-runner.yml
push:
branches: [main]
paths:
- mcp-runner/**
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

permissions:
contents: read

jobs:
mcp-runner:
name: Build and run servers
runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4

- name: Build
run: docker build -t lens-mcp:ci mcp-runner

- name: Start it as Lens's compose file does
run: |
docker network create mcp
docker run -d --name mcp --network mcp --read-only --tmpfs /tmp --cap-drop ALL \
--security-opt no-new-privileges:true -v mcp-data:/data -v mcp-key:/key lens-mcp:ci
sleep 3
docker logs mcp

- name: Runs an npm server and a PyPI server
run: |
token="$(docker run --rm -v mcp-key:/key alpine:3.20 cat /key/token)"
call() {
docker run --rm --network mcp curlimages/curl:8.10.1 -fsS --max-time 240 -X "$1" \
-H "Authorization: Bearer $token" -H "Content-Type: application/json" "http://mcp:8090$2" -d "$3"
}
call PUT /servers/memory '{"command": "npx", "args": ["-y", "@modelcontextprotocol/server-memory@2026.8.31"]}'
call POST /servers/memory/mcp '{"jsonrpc": "2.0", "id": 1, "method": "tools/list"}' | tee out.json
grep -q create_entities out.json
call PUT /servers/time '{"command": "uvx", "args": ["mcp-server-time==2026.8.18"]}'
call POST /servers/time/mcp '{"jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": {"name": "get_current_time", "arguments": {"timezone": "UTC"}}}' | tee out.json
grep -q UTC out.json
# and refuses anyone without the token
code="$(docker run --rm --network mcp curlimages/curl:8.10.1 -s -o /dev/null -w '%{http_code}' -X PUT http://mcp:8090/servers/x -d '{}')"
test "$code" = 401

- name: What it said
if: failure()
run: docker logs mcp

- name: Stop
if: always()
run: docker rm -f mcp || true
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
assistant and the MCP server can ask for it (`find_similar`). It works without search by meaning, by key words,
topics and entities ([Processing](docs/processing.md#similar-content)).

- **MCP servers that run as a program.** Most MCP servers in marketplaces run as a program (`npx` or `uvx`), not on
the network. Admins can now add one as an MCP server extension with its `command`, `args` and `env`, as in Claude's
`.mcp.json`, and a plugin's such servers come along when an admin installs it. They run in the MCP runner, an opt-in
container with none of Lens's data or keys (`COMPOSE_PROFILES=mcp`), start on first use and stop after 15 idle
minutes ([Assistant](docs/assistant.md#the-mcp-runner)).

## 0.8.1 <small>October 12, 2026</small> {id="0.8.1"}

- **The installer stops waiting when Lens can't start, and shows why.** A microsandbox install that couldn't start Lens
Expand Down
11 changes: 11 additions & 0 deletions DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@

Plans and progress for work in flight. Newest first.

## 2026-10-14 · MCP servers that run as a program

Admins add a stdio MCP server (`command`, `args`, `env`, as in Claude's `.mcp.json`) as an `mcp` extension, and a
plugin's stdio servers come along when an admin installs it. They run in the MCP runner (`mcp-runner/`, the opt-in
`mcp` Compose profile), a container with none of Lens's data or keys, reached over HTTP with a token on a volume only
it and the backend share. Recorded in the threat model.

Refine later: turn the runner on from Settings through the machine agent, as monitoring does, instead of
`COMPOSE_PROFILES=mcp` in `.env`; the runner on installs that don't use Docker (microsandbox, Cloudron, NAS packages);
plugin servers that run a program from the plugin's own folder; an egress allow-list for the runner's network.

## 2026-10-10 · Pasted transcripts fetch no speech model

Found by the clicks survey: on a fresh install, importing a pasted transcript queued the standard pipeline from
Expand Down
30 changes: 29 additions & 1 deletion docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ x-backend: &backend
LENS_GRAFANA_URL: ${LENS_GRAFANA_URL:-}
# the volume shared with the matterbridge service (its config, written from Settings → Chat rooms)
LENS_MATTERBRIDGE_DIR: /matterbridge
# the MCP runner (the optional `mcp` service, docs/assistant.md): MCP servers that run as a program, sandboxed there
LENS_MCP_RUNNER_URL: http://mcp:8090
LENS_MCP_TOKEN_FILE: /mcp-key/token
# upgrading from the web app (docs/updates.md): the folder install.sh's upgrade agent watches, and the install folder
LENS_CONTROL_DIR: /control
LENS_HOST_DIR: ${LENS_DIR:-}
Expand All @@ -78,11 +81,12 @@ x-backend: &backend
- ./.lens-control:/control
- ${AUDIO_DIR:-./local-audio}:/audio:ro
- matterbridge-data:/matterbridge
- mcp-key:/mcp-key:ro
depends_on:
surrealdb:
condition: service_healthy
restart: unless-stopped
networks: [lens]
networks: [lens, mcp]

services:
surrealdb:
Expand Down Expand Up @@ -160,6 +164,26 @@ services:
restart: unless-stopped
networks: [lens]

# The MCP runner (mcp-runner/runner.py, docs/assistant.md): MCP servers that run as a program (stdio, usually npx or
# uvx), which admins add as extensions. Opt-in: `docker compose --profile mcp up -d`, or COMPOSE_PROFILES=mcp in .env.
# The sandbox: none of Lens's data or keys, no capabilities, a read-only image, and a network shared only with the
# backend (and the internet).
mcp:
image: lens-mcp:${LENS_VERSION:-prod}
build: ./mcp-runner
profiles: [mcp]
security_opt: ["no-new-privileges:true"]
cap_drop: [ALL]
read_only: true
tmpfs: [/tmp]
mem_limit: 2g
pids_limit: 512
volumes:
- mcp-data:/data
- mcp-key:/key
restart: unless-stopped
networks: [mcp]

# StyleTTS2 (docs/speech-providers.md#styletts2): spoken answers and podcasts read on this machine, in a voice like a
# short clip you put in its voices volume. Heavy (PyTorch and a 1 GB model, fetched on first use), so opt-in:
# `docker compose --profile styletts2 up -d`, then Settings → AI assistant → Voice → StyleTTS2.
Expand Down Expand Up @@ -267,6 +291,8 @@ services:
volumes:
surreal-data:
matterbridge-data:
mcp-data:
mcp-key:
styletts2-data:
styletts2-voices:
archive-data:
Expand All @@ -278,3 +304,5 @@ volumes:
networks:
lens:
driver: bridge
mcp:
driver: bridge
29 changes: 28 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ x-backend: &backend
OPENAPI_OUTPUT_FILE: ./shared-data/openapi.json
# the volume shared with the matterbridge service (its config, written from Settings → Chat rooms)
LENS_MATTERBRIDGE_DIR: /matterbridge
# the MCP runner (the optional `mcp` service, docs/assistant.md): MCP servers that run as a program, sandboxed there
LENS_MCP_RUNNER_URL: http://mcp:8090
LENS_MCP_TOKEN_FILE: /mcp-key/token
volumes:
- ./fastapi_backend:/app
# the image's own packages, not the source tree's: anonymous, so `make dev` (up --build --renew-anon-volumes)
Expand All @@ -55,10 +58,11 @@ x-backend: &backend
- ./local-shared-data:/app/shared-data
- ${AUDIO_DIR:-./local-audio}:/audio:ro
- matterbridge-data:/matterbridge
- mcp-key:/mcp-key:ro
depends_on:
surrealdb:
condition: service_healthy
networks: [lens]
networks: [lens, mcp]

services:
surrealdb:
Expand Down Expand Up @@ -141,6 +145,25 @@ services:
- matterbridge-data:/matterbridge
networks: [lens]

# The MCP runner (mcp-runner/runner.py, docs/assistant.md): MCP servers that run as a program (stdio, usually npx or
# uvx), which admins add as extensions. Opt-in: `docker compose --profile mcp up -d`, or COMPOSE_PROFILES=mcp in .env.
# The sandbox: none of Lens's data or keys, no capabilities, a read-only image, and a network shared only with the
# backend (and the internet).
mcp:
image: lens-mcp:dev
build: ./mcp-runner
profiles: [mcp]
security_opt: ["no-new-privileges:true"]
cap_drop: [ALL]
read_only: true
tmpfs: [/tmp]
mem_limit: 2g
pids_limit: 512
volumes:
- mcp-data:/data
- mcp-key:/key
networks: [mcp]

# StyleTTS2 (docs/speech-providers.md#styletts2): spoken answers and podcasts read on this machine, in a voice like a
# short clip you put in its voices volume. Heavy (PyTorch and a 1 GB model, fetched on first use), so opt-in:
# `docker compose --profile styletts2 up -d`, then Settings → AI assistant → Voice → StyleTTS2.
Expand Down Expand Up @@ -172,6 +195,8 @@ services:
volumes:
surreal-data:
matterbridge-data:
mcp-data:
mcp-key:
styletts2-data:
styletts2-voices:
archive-data:
Expand All @@ -180,3 +205,5 @@ volumes:
networks:
lens:
driver: bridge
mcp:
driver: bridge
20 changes: 17 additions & 3 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,8 +216,9 @@ shared with namespaces (admins can share with everyone), and switched on or off
- **MCP server** (admins add them): a server that speaks the Model Context Protocol over Streamable HTTP, at the
address an admin gives (this machine and the local network included, as for the model's address), with the headers
it needs (a token). Lens lists its tools when it's saved, and the assistant calls them as `<server>__<tool>`: one the
server marks read-only answers at once, the others ask for approval. Servers that run as a program (stdio) aren't
run.
server marks read-only answers at once, the others ask for approval. A server that runs as a program (stdio:
`command`, `args` and `env` instead of an address, as in Claude's `.mcp.json`, usually `npx` or `uvx`) runs in the
MCP runner, below.
- **Plugin**: a bundle of tools, skills, hooks and MCP servers, shared and switched off as one.

Make one however suits you:
Expand Down Expand Up @@ -263,9 +264,22 @@ version. A skill (a folder with SKILL.md) is a skill, and the assistant reads it
scripts) when the instructions point at them; nothing in them runs. A command (`commands/*.md`) is a skill the
assistant follows when someone asks for it, and an agent (`agents/*.md`) is a prompt tool with the agent's instructions
as its system prompt. Claude Code's hooks run commands on the computer, so they're left out. A plugin's MCP servers on
the network (`.mcp.json`) come along when an admin installs it; those that run as a program are left out. Repositories are read from GitHub or GitLab (or the archive link) through the same guard as web
the network (`.mcp.json`) come along when an admin installs it, and those that run as a program too when the MCP
runner is on (ones that run a program shipped inside the plugin are left out). Repositories are read from GitHub or GitLab (or the archive link) through the same guard as web
tools: public addresses only.

### The MCP runner

MCP servers that run as a program run in a container of their own, the MCP runner
([`mcp-runner/`](https://github.com/adeelahmad/lense/tree/main/mcp-runner)), never next to Lens. It's off until you
turn it on: add `COMPOSE_PROFILES=mcp` to Lens's `.env` and run the install command again (or
`docker compose --profile mcp up -d`). The container is the sandbox: it has none of Lens's data, keys or database, runs
as an unprivileged user with every capability dropped, a read-only image and memory and process limits, and shares a
network only with Lens's backend (and the internet, which `npx` and `uvx` need to fetch the servers). Each server gets
an empty folder as its home and only the environment its extension gives it; Lens talks to the runner over HTTP with
a token the runner writes to a volume only the two share. A server starts on first use, stays up while it's used,
and stops after 15 idle minutes. Installs that don't run on Docker don't have the runner yet.

`ai.extensions: false` turns every extension off.

## Setting up and running the server by chat
Expand Down
39 changes: 27 additions & 12 deletions fastapi_backend/app/domain/extensions.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,22 +264,36 @@ def _check_hook(spec, me, db=None):


def _check_mcp(spec, me, db=None):
"""An MCP server: its address, the headers it needs (a token), and its tools as it listed them (mcp_client.py)."""
if set(spec) - {"url", "headers", "tools"}:
raise ValueError("an MCP server is {url, headers, tools}")
"""An MCP server: its address and the headers it needs (a token), or the program it runs as (command, args, env:
run in the MCP runner's sandbox), and its tools as it listed them (mcp_client.py)."""
if set(spec) - {"url", "headers", "command", "args", "env", "tools"}:
raise ValueError("an MCP server is {url, headers, tools}, or {command, args, env, tools} for one that runs as a program")
if not me["admin"]:
raise ValueError("only admins add MCP servers")
url = _text(spec.get("url"), "the MCP server's address", 2000, required=True)
if not re.match(r"^https?://[^/?#\s]+", url):
raise ValueError("an MCP server's address starts with http:// or https:// (servers that run as a program aren't run)")
headers = spec.get("headers") or {}
if not isinstance(headers, dict) or not all(isinstance(k, str) and isinstance(v, str) for k, v in headers.items()):
raise ValueError("headers are names and text values")

def words(v, what):
if not isinstance(v, dict) or not all(isinstance(k, str) and isinstance(x, str) for k, x in v.items()):
raise ValueError(f"{what} are names and text values")
return v

if spec.get("command") is not None:
if spec.get("url") or spec.get("headers"):
raise ValueError("an MCP server has an address or a command, not both")
server = {"command": _text(spec.get("command"), "the MCP server's command", 200, required=True)}
args = spec.get("args") or []
if not isinstance(args, list) or len(args) > 50 or not all(isinstance(a, str) and len(a) <= 2000 for a in args):
raise ValueError("an MCP server's args are a list of words")
server.update({"args": args or None, "env": words(spec.get("env") or {}, "env values") or None})
else:
url = _text(spec.get("url"), "the MCP server's address (or its command)", 2000, required=True)
if not re.match(r"^https?://[^/?#\s]+", url):
raise ValueError("an MCP server's address starts with http:// or https://")
server = {"url": url, "headers": words(spec.get("headers") or {}, "headers") or None}
tools = spec.get("tools")
if tools is None: # asked of the server: what it offers now
from . import mcp_client

tools = mcp_client.discover({"url": url, "headers": headers})
tools = mcp_client.discover(store.clean(server))
if tools is not None:
if not isinstance(tools, list) or len(tools) > 100:
raise ValueError("an MCP server's tools are a list of at most 100")
Expand All @@ -297,7 +311,7 @@ def _check_mcp(spec, me, db=None):
)
for t in tools
]
return store.clean({"url": url, "headers": headers or None, "tools": tools})
return store.clean({**server, "tools": tools})


def mcp_tools(item):
Expand All @@ -306,7 +320,8 @@ def mcp_tools(item):
s, out = item["spec"], []
for t in s.get("tools") or []:
name = re.sub(r"[^A-Za-z0-9_-]", "_", f"{item['name']}__{t['name']}")[:64]
run = store.clean({"type": "mcp", "url": s["url"], "headers": s.get("headers"), "tool": t["name"]})
where = {k: s.get(k) for k in ("url", "headers", "command", "args", "env")}
run = store.clean({"type": "mcp", **where, "tool": t["name"]})
spec = {
"params": [],
"schema": t.get("schema") or {"type": "object"},
Expand Down
31 changes: 21 additions & 10 deletions fastapi_backend/app/domain/marketplace.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
What becomes what: a skill is a skill, with its text files (references, examples, scripts to read; nothing runs); a
command is a skill the assistant follows when someone asks for it; an agent is a prompt tool with the agent's
instructions as its system prompt. Claude Code's hooks run shell commands on the computer, so they're left out; MCP
servers on the network come along when an admin installs it (servers that run as a program don't); `preview` lists
what's left out and why. A plugin becomes one Lens plugin, owned by the person
servers come along when an admin installs it (those that run as a program run in the MCP runner's sandbox, when it
runs: mcp_client.py); `preview` lists what's left out and why. A plugin becomes one Lens plugin, owned by the person
who installed it and private until they share it, and installing it again brings the new version.

Only admins add marketplaces: that's the approval. People who can write extensions install from the ones added, and
Expand Down Expand Up @@ -404,7 +404,7 @@ def _norm(p):

def _mcp(found, meta, admin, report):
"""A plugin's MCP servers (.mcp.json, or mcpServers in its manifest) as MCP server items, their tools listed now.
Servers on the network only (HTTP); only admins add them."""
On the network (HTTP), or run as a program in the MCP runner when it runs; only admins add them."""
conf = meta.get("mcpServers")
if isinstance(conf, str):
conf = found.get(_norm(conf))
Expand All @@ -431,15 +431,26 @@ def _mcp(found, meta, admin, report):
c = c if isinstance(c, dict) else {}
kind = c.get("type") or ("stdio" if c.get("command") else "http")
if kind == "stdio" or c.get("command"):
report.append(
{"kind": "mcp", "name": name, "why": "it runs a program on the computer; Lens reaches MCP servers on the network"}
)
continue
if kind not in ("http", "streamable-http") or not isinstance(c.get("url"), str):
args = [str(x) for x in c.get("args") or [] if isinstance(x, (str, int, float))]
env = {k: str(v) for k, v in (c.get("env") or {}).items() if isinstance(k, str) and isinstance(v, (str, int, float))}
if not isinstance(c.get("command"), str):
report.append({"kind": "mcp", "name": name, "why": "it says it runs a program but not which"})
continue
if any("CLAUDE_PLUGIN_ROOT" in x for x in [c["command"], *args, *env.values()]):
report.append(
{"kind": "mcp", "name": name, "why": "it runs a program from the plugin's own folder, which Lens doesn't run"}
)
continue
if not mcp_client.runner():
report.append({"kind": "mcp", "name": name, "why": mcp_client.NO_RUNNER})
continue
server = store.clean({"command": c["command"], "args": args or None, "env": env or None})
elif kind not in ("http", "streamable-http") or not isinstance(c.get("url"), str):
report.append({"kind": "mcp", "name": name, "why": f"Lens speaks MCP over HTTP; this one uses {kind}"})
continue
headers = {k: v for k, v in (c.get("headers") or {}).items() if isinstance(k, str) and isinstance(v, str)}
server = {"url": c["url"], "headers": headers or None}
else:
headers = {k: v for k, v in (c.get("headers") or {}).items() if isinstance(k, str) and isinstance(v, str)}
server = {"url": c["url"], "headers": headers or None}
try:
tools = mcp_client.discover(server)
except ValueError as e:
Expand Down
Loading
Loading