Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **MCP server: agents search, read and cite the archive.** Add `https://<your Lens>/mcp` to Claude, Cursor, VS Code
or another MCP client, sign in on Lens's consent page (OAuth), and the agent sees what you see: your namespaces,
the collections you were given a role on, and their graphs. Read-only tools: `search` (moments said, on screen or
written, with links to each), `list_namespaces`, `list_recordings`, `list_speakers`, `get_recording` (summary,
chapters, entities), `get_transcript` (paged), `fetch` (the whole text), `cite` (the words, who said them, the
time and a link, as Markdown), `list_entities`, `get_entity`, `explore_graph` and `find_path`. Links open the
recording at that moment in the web app. Streamable HTTP without sessions, in both protocol eras (the 2025
`initialize` handshake and 2026-07-28's per-request envelope); an API token works for clients that can't sign in
(docs/mcp.md).
- **Opt-in telemetry.** Lens can send OpenTelemetry traces and metrics about its own work to a collector you choose:
API requests by route, jobs and each step, routines, workflow runs, and model calls with their tokens and an
estimated cost (from per-model prices you set). It is off by default and never on unless you turn it on, in
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ recording.
- **Publishing:** an embeddable transcript player, static reports, and IIIF Presentation 3 with Content Search, Change
Discovery and the Authorization Flow.
- **Multi-user:** roles per namespace (viewer, editor, owner), API tokens, share links, and an audit log.
- **Agents:** an MCP server (`/mcp`) that agents sign in to with OAuth, to search, read and cite what the person can
read.

## Stack

Expand Down Expand Up @@ -43,7 +45,7 @@ Start with [Get started](docs/get-started.md), then:
- [Database](docs/database.md): SurrealDB modes, schema, and the traps the code avoids
- [Configuration](docs/configuration.md)
- [Processing](docs/processing.md), [Video](docs/video.md), [Chat and batch runs](docs/assistant.md), [IIIF](docs/iiif.md)
- [API](docs/api.md), [what the design needs from the API next](docs/backend-gaps.md)
- [API](docs/api.md), [MCP server](docs/mcp.md), [what the design needs from the API next](docs/backend-gaps.md)
- [Deployment](docs/deployment.md), [Contributing](CONTRIBUTING.md), [Security](SECURITY.md)

Built on the [Next.js FastAPI Template](https://github.com/vintasoftware/nextjs-fastapi-template) (MIT).
3 changes: 2 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ GET /api/v1/oauth/grants
DELETE /api/v1/oauth/grants/{grant_id}
```

Lens is an OAuth 2.1 authorization server for API and MCP clients ([Authentication](authentication.md#oauth)).
Lens is an OAuth 2.1 authorization server for API and MCP clients ([Authentication](authentication.md#oauth)). The MCP
server itself is at `/mcp`, outside `/api/v1` ([MCP server](mcp.md)).

* `POST /oauth/register {redirect_uris, client_name?, client_uri?, token_endpoint_auth_method?}` (RFC 7591, no sign-in,
201) registers an app: `client_id` (`lc_…`), and a `client_secret` (`ls_…`, shown once) when it asked for
Expand Down
4 changes: 2 additions & 2 deletions docs/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ curl -H "Authorization: Bearer la_…" https://lens.example.org/api/v1/resources

## OAuth

For apps that sign people in instead of asking them for a key: MCP clients (Claude, Cursor and others), desktop and
web apps. Lens is the OAuth 2.1 authorization server itself: it issues the tokens, and people sign in with the Lens
For apps that sign people in instead of asking them for a key: MCP clients (Claude, Cursor and others, connecting
to [Lens's MCP server](mcp.md)), desktop and web apps. Lens is the OAuth 2.1 authorization server itself: it issues the tokens, and people sign in with the Lens
account they have. Nothing here creates an account.

```
Expand Down
75 changes: 75 additions & 0 deletions docs/mcp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# MCP server

Lens is a [Model Context Protocol](https://modelcontextprotocol.io) server, so agents (Claude, Cursor, VS Code,
ChatGPT and other MCP clients) can search the archive, read recordings and cite them. An agent signs in as a person
and sees exactly what that person sees: their namespaces, the collections they were given a role on, and the graphs
those open. Everything it can do only reads.

## Connecting

The server is at `/mcp` on the web app's address: `https://lens.example.org/mcp` (`http://localhost:3000/mcp`
locally). Add it to the client as a remote (Streamable HTTP) server, and the client sends you to Lens to sign in.

| Client | |
|---|---|
| Claude (claude.ai, desktop) | Settings → Connectors → Add custom connector, with the address |
| Claude Code | `claude mcp add --transport http lens https://lens.example.org/mcp` |
| Cursor | `~/.cursor/mcp.json`: `{"mcpServers": {"lens": {"url": "https://lens.example.org/mcp"}}}` |
| VS Code | `.vscode/mcp.json`: `{"servers": {"lens": {"type": "http", "url": "https://lens.example.org/mcp"}}}` |
| MCP Inspector | `npx @modelcontextprotocol/inspector`, transport Streamable HTTP |

Signing in is OAuth ([Authentication](authentication.md#oauth)): the client finds out where to sign in from the 401 it
gets first (`WWW-Authenticate: Bearer resource_metadata="…/.well-known/oauth-protected-resource/mcp"`), registers
itself, and opens Lens's consent page, where you sign in if you aren't and choose Allow. Read access is all it needs.
The client renews its token by itself; you see it under API tokens → Apps with access, and take its access away
there.

A client that can't sign in can send an API token instead (`Authorization: Bearer la_…`, made under API tokens), for
example `claude mcp add --transport http lens https://lens.example.org/mcp --header "Authorization: Bearer la_…"`.

For the addresses to be right, set `FRONTEND_URL` to the address people use, and list the web app as a trusted proxy
when it runs on another machine or container ([Configuration](configuration.md#trusted-proxies)); a reverse proxy in
front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](authentication.md#oauth)).

## Tools

| Tool | |
|---|---|
| `search` | moments where words are said, shown on screen or written, best first: the recording, speaker, time, the matching text (matches in **bold**) and a link to that moment. Words, `"phrases"`, `OR`; by namespace, speaker or recording |
| `list_namespaces` | the namespaces you can read, with their recordings, hours, speakers and your role |
| `list_recordings` | recordings, newest first, by namespace, title words, dates, speaker, tag or kind |
| `list_speakers` | the speakers of your namespaces, the ones who talk most first |
| `get_recording` | one recording: when, how long, speakers, summary and key points, chapters, the entities and keywords most mentioned, its link |
| `get_transcript` | a recording's lines in order (speaker, time, text, link), a page at a time, from a line or a time window |
| `fetch` | a recording's whole text at once, as `[m:ss] Speaker: …` lines |
| `cite` | a citation for a moment (a line or a second, and a few lines from there): the words, who said them, the recording and date, the time, the link, and all of it as Markdown |
| `list_entities` | people, organisations, products, places, events, works and topics mentioned, by name, type, namespace or recording |
| `get_entity` | one entity: its names, how often and when it is mentioned, what it is mentioned with, who mentions it most, and the lines that mention it |
| `explore_graph` | the knowledge graph around an entity (`e12`) or a speaker (`s3`), one or two steps out |
| `find_path` | the shortest chain of links between two of them, with lines that show each link |

Results are JSON (as `structuredContent`, and the same as text). Links open the recording's page in the web app at
that moment (`/resources/<id>?t=<seconds>`), on the address the client used. Mistakes the agent can fix (a wrong
argument, something it can't read) come back as a tool result with `isError`, so it can try again; what isn't the
person's to read answers "not found", as the API does. `search` and `fetch` follow the shape ChatGPT's connectors
expect (`results` with `id`, `title`, `url`; `fetch` by that `id`).

There are no tools that change the archive yet: importing, editing and curating stay in the web app and the API.

## Protocol

Streamable HTTP without sessions: every request is a `POST /mcp` answered with JSON on the same response. `GET` and
`DELETE` (a stream from the server, ending a session) answer 405. Both eras of the protocol are served:

* the `initialize` handshake (versions 2024-11-05 to 2025-11-25), with `MCP-Protocol-Version` on the requests after
it; notifications get 202, and a 2025-03-26 client may batch;
* 2026-07-28's per-request envelope: no handshake, the version and the client's capabilities in each request's
`params._meta`, repeated with the method (and the tool's name) in the `MCP-Protocol-Version`, `Mcp-Method` and
`Mcp-Name` headers; `server/discover` lists the versions. Results carry `resultType`, and protocol errors their
HTTP status (400, or 404 for an unknown method).

The `MCP-Protocol-Version` header decides the era: none or a handshake version is the first, anything else the
second, where a version Lens doesn't speak is answered `-32022` with the ones it does. Requests from a browser must
come from the web app's own address or one in `CORS_ORIGINS` (403 otherwise), which keeps other sites from using a
signed-in browser to reach it. Implemented in `fastapi_backend/app/api/mcp.py` (protocol) and `mcp_tools.py` (tools);
the tests run the official MCP SDK's client against it in both eras.
Loading
Loading