From 723402e25f642da07f94e40e492d668bcefea326 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Fri, 2 Oct 2026 14:00:47 +0000 Subject: [PATCH] feat(mcp): an MCP server for agents to search, read and cite the archive /mcp speaks Streamable HTTP without sessions, in both protocol eras (the initialize handshake and 2026-07-28's per-request envelope). Agents sign in with Lens's OAuth (the 401 names /.well-known/oauth-protected-resource/mcp, which asks for read only) or an API token, and read what the person can through the API's own checks. Read-only tools: search, list_namespaces, list_recordings, list_speakers, get_recording, get_transcript, fetch, cite, list_entities, get_entity, explore_graph and find_path; moments carry links that open the recording there in the web app. The web app proxies /mcp. Tested with the official MCP SDK client in both eras. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01TUqMhwUCvF9dsP5JzVvMJf --- CHANGELOG.md | 9 + README.md | 4 +- docs/api.md | 3 +- docs/authentication.md | 4 +- docs/mcp.md | 75 ++ fastapi_backend/app/api/mcp.py | 260 ++++++ fastapi_backend/app/api/mcp_tools.py | 902 +++++++++++++++++++++ fastapi_backend/app/api/v1/routes/oauth.py | 23 +- fastapi_backend/app/main.py | 2 + fastapi_backend/pyproject.toml | 1 + fastapi_backend/tests/api/test_mcp.py | 301 +++++++ fastapi_backend/tests/api/test_oauth.py | 5 +- fastapi_backend/uv.lock | 118 +++ mkdocs.yml | 1 + nextjs-frontend/app/mcp/route.ts | 6 + nextjs-frontend/proxy.ts | 4 +- 16 files changed, 1702 insertions(+), 16 deletions(-) create mode 100644 docs/mcp.md create mode 100644 fastapi_backend/app/api/mcp.py create mode 100644 fastapi_backend/app/api/mcp_tools.py create mode 100644 fastapi_backend/tests/api/test_mcp.py create mode 100644 nextjs-frontend/app/mcp/route.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index ffafa4848..f43ec4624 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,15 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **MCP server: agents search, read and cite the archive.** Add `https:///mcp` to Claude, Cursor, VS Code + or another MCP client, sign in on Lens's consent page (OAuth), and the agent sees what you see: your namespaces, + the collections you were given a role on, and their graphs. Read-only tools: `search` (moments said, on screen or + written, with links to each), `list_namespaces`, `list_recordings`, `list_speakers`, `get_recording` (summary, + chapters, entities), `get_transcript` (paged), `fetch` (the whole text), `cite` (the words, who said them, the + time and a link, as Markdown), `list_entities`, `get_entity`, `explore_graph` and `find_path`. Links open the + recording at that moment in the web app. Streamable HTTP without sessions, in both protocol eras (the 2025 + `initialize` handshake and 2026-07-28's per-request envelope); an API token works for clients that can't sign in + (docs/mcp.md). - **Install on Proxmox VE with one command.** A [community helper script](https://community-scripts.org/docs/ct/detailed_guide) (`proxmox/ct/lens.sh`, run in the Proxmox host's shell) creates a Debian 13 LXC container running Lens without Docker: SurrealDB 3.2.4, the API, a job worker and the web app as systemd services, with fresh secrets and the web diff --git a/README.md b/README.md index 2e65fab45..9d78b9ff0 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,8 @@ recording. - **Publishing:** an embeddable transcript player, static reports, and IIIF Presentation 3 with Content Search, Change Discovery and the Authorization Flow. - **Multi-user:** roles per namespace (viewer, editor, owner), API tokens, share links, and an audit log. +- **Agents:** an MCP server (`/mcp`) that agents sign in to with OAuth, to search, read and cite what the person can + read. ## Stack @@ -43,7 +45,7 @@ Start with [Get started](docs/get-started.md), then: - [Database](docs/database.md): SurrealDB modes, schema, and the traps the code avoids - [Configuration](docs/configuration.md) - [Processing](docs/processing.md), [Video](docs/video.md), [Chat and batch runs](docs/assistant.md), [IIIF](docs/iiif.md) -- [API](docs/api.md), [what the design needs from the API next](docs/backend-gaps.md) +- [API](docs/api.md), [MCP server](docs/mcp.md), [what the design needs from the API next](docs/backend-gaps.md) - [Deployment](docs/deployment.md), [Contributing](CONTRIBUTING.md), [Security](SECURITY.md) Built on the [Next.js FastAPI Template](https://github.com/vintasoftware/nextjs-fastapi-template) (MIT). diff --git a/docs/api.md b/docs/api.md index 98b9fc296..fe2182c18 100644 --- a/docs/api.md +++ b/docs/api.md @@ -54,7 +54,8 @@ GET /api/v1/oauth/grants DELETE /api/v1/oauth/grants/{grant_id} ``` -Lens is an OAuth 2.1 authorization server for API and MCP clients ([Authentication](authentication.md#oauth)). +Lens is an OAuth 2.1 authorization server for API and MCP clients ([Authentication](authentication.md#oauth)). The MCP +server itself is at `/mcp`, outside `/api/v1` ([MCP server](mcp.md)). * `POST /oauth/register {redirect_uris, client_name?, client_uri?, token_endpoint_auth_method?}` (RFC 7591, no sign-in, 201) registers an app: `client_id` (`lc_…`), and a `client_secret` (`ls_…`, shown once) when it asked for diff --git a/docs/authentication.md b/docs/authentication.md index 5b6ba2f05..feae2ce44 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -56,8 +56,8 @@ curl -H "Authorization: Bearer la_…" https://lens.example.org/api/v1/resources ## OAuth -For apps that sign people in instead of asking them for a key: MCP clients (Claude, Cursor and others), desktop and -web apps. Lens is the OAuth 2.1 authorization server itself: it issues the tokens, and people sign in with the Lens +For apps that sign people in instead of asking them for a key: MCP clients (Claude, Cursor and others, connecting +to [Lens's MCP server](mcp.md)), desktop and web apps. Lens is the OAuth 2.1 authorization server itself: it issues the tokens, and people sign in with the Lens account they have. Nothing here creates an account. ``` diff --git a/docs/mcp.md b/docs/mcp.md new file mode 100644 index 000000000..18e15f744 --- /dev/null +++ b/docs/mcp.md @@ -0,0 +1,75 @@ +# MCP server + +Lens is a [Model Context Protocol](https://modelcontextprotocol.io) server, so agents (Claude, Cursor, VS Code, +ChatGPT and other MCP clients) can search the archive, read recordings and cite them. An agent signs in as a person +and sees exactly what that person sees: their namespaces, the collections they were given a role on, and the graphs +those open. Everything it can do only reads. + +## Connecting + +The server is at `/mcp` on the web app's address: `https://lens.example.org/mcp` (`http://localhost:3000/mcp` +locally). Add it to the client as a remote (Streamable HTTP) server, and the client sends you to Lens to sign in. + +| Client | | +|---|---| +| Claude (claude.ai, desktop) | Settings → Connectors → Add custom connector, with the address | +| Claude Code | `claude mcp add --transport http lens https://lens.example.org/mcp` | +| Cursor | `~/.cursor/mcp.json`: `{"mcpServers": {"lens": {"url": "https://lens.example.org/mcp"}}}` | +| VS Code | `.vscode/mcp.json`: `{"servers": {"lens": {"type": "http", "url": "https://lens.example.org/mcp"}}}` | +| MCP Inspector | `npx @modelcontextprotocol/inspector`, transport Streamable HTTP | + +Signing in is OAuth ([Authentication](authentication.md#oauth)): the client finds out where to sign in from the 401 it +gets first (`WWW-Authenticate: Bearer resource_metadata="…/.well-known/oauth-protected-resource/mcp"`), registers +itself, and opens Lens's consent page, where you sign in if you aren't and choose Allow. Read access is all it needs. +The client renews its token by itself; you see it under API tokens → Apps with access, and take its access away +there. + +A client that can't sign in can send an API token instead (`Authorization: Bearer la_…`, made under API tokens), for +example `claude mcp add --transport http lens https://lens.example.org/mcp --header "Authorization: Bearer la_…"`. + +For the addresses to be right, set `FRONTEND_URL` to the address people use, and list the web app as a trusted proxy +when it runs on another machine or container ([Configuration](configuration.md#trusted-proxies)); a reverse proxy in +front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](authentication.md#oauth)). + +## Tools + +| Tool | | +|---|---| +| `search` | moments where words are said, shown on screen or written, best first: the recording, speaker, time, the matching text (matches in **bold**) and a link to that moment. Words, `"phrases"`, `OR`; by namespace, speaker or recording | +| `list_namespaces` | the namespaces you can read, with their recordings, hours, speakers and your role | +| `list_recordings` | recordings, newest first, by namespace, title words, dates, speaker, tag or kind | +| `list_speakers` | the speakers of your namespaces, the ones who talk most first | +| `get_recording` | one recording: when, how long, speakers, summary and key points, chapters, the entities and keywords most mentioned, its link | +| `get_transcript` | a recording's lines in order (speaker, time, text, link), a page at a time, from a line or a time window | +| `fetch` | a recording's whole text at once, as `[m:ss] Speaker: …` lines | +| `cite` | a citation for a moment (a line or a second, and a few lines from there): the words, who said them, the recording and date, the time, the link, and all of it as Markdown | +| `list_entities` | people, organisations, products, places, events, works and topics mentioned, by name, type, namespace or recording | +| `get_entity` | one entity: its names, how often and when it is mentioned, what it is mentioned with, who mentions it most, and the lines that mention it | +| `explore_graph` | the knowledge graph around an entity (`e12`) or a speaker (`s3`), one or two steps out | +| `find_path` | the shortest chain of links between two of them, with lines that show each link | + +Results are JSON (as `structuredContent`, and the same as text). Links open the recording's page in the web app at +that moment (`/resources/?t=`), on the address the client used. Mistakes the agent can fix (a wrong +argument, something it can't read) come back as a tool result with `isError`, so it can try again; what isn't the +person's to read answers "not found", as the API does. `search` and `fetch` follow the shape ChatGPT's connectors +expect (`results` with `id`, `title`, `url`; `fetch` by that `id`). + +There are no tools that change the archive yet: importing, editing and curating stay in the web app and the API. + +## Protocol + +Streamable HTTP without sessions: every request is a `POST /mcp` answered with JSON on the same response. `GET` and +`DELETE` (a stream from the server, ending a session) answer 405. Both eras of the protocol are served: + +* the `initialize` handshake (versions 2024-11-05 to 2025-11-25), with `MCP-Protocol-Version` on the requests after + it; notifications get 202, and a 2025-03-26 client may batch; +* 2026-07-28's per-request envelope: no handshake, the version and the client's capabilities in each request's + `params._meta`, repeated with the method (and the tool's name) in the `MCP-Protocol-Version`, `Mcp-Method` and + `Mcp-Name` headers; `server/discover` lists the versions. Results carry `resultType`, and protocol errors their + HTTP status (400, or 404 for an unknown method). + +The `MCP-Protocol-Version` header decides the era: none or a handshake version is the first, anything else the +second, where a version Lens doesn't speak is answered `-32022` with the ones it does. Requests from a browser must +come from the web app's own address or one in `CORS_ORIGINS` (403 otherwise), which keeps other sites from using a +signed-in browser to reach it. Implemented in `fastapi_backend/app/api/mcp.py` (protocol) and `mcp_tools.py` (tools); +the tests run the official MCP SDK's client against it in both eras. diff --git a/fastapi_backend/app/api/mcp.py b/fastapi_backend/app/api/mcp.py new file mode 100644 index 000000000..4303da8c7 --- /dev/null +++ b/fastapi_backend/app/api/mcp.py @@ -0,0 +1,260 @@ +"""Lens as an MCP server (docs/mcp.md): agents search, read and cite the archive as the person who signed them in. + +One endpoint, ``/mcp``, speaks the Model Context Protocol over Streamable HTTP, without sessions, answering every +request on its own POST as JSON. Both eras of the protocol are served: + +* **Handshake era** (2024-11-05 to 2025-11-25): the client starts with ``initialize``, then sends requests with the + version it got in ``MCP-Protocol-Version``. Notifications and responses get 202; a 2025-03-26 client may batch. +* **Per-request era** (2026-07-28): no handshake. Each request carries its version and the client's capabilities in + ``params._meta``, and repeats the version, the method and (for ``tools/call``) the tool's name in the + ``MCP-Protocol-Version``, ``Mcp-Method`` and ``Mcp-Name`` headers, which must agree with the body. + ``server/discover`` says what this server speaks. Results say ``resultType``; protocol errors have HTTP statuses. + +The era is the ``MCP-Protocol-Version`` header's: none or a handshake version is the first, anything else the second +(where an unknown version is refused with the versions there are). Nothing streams from the server, so GET (the +server-to-client stream) and DELETE (ending a session) are 405. + +Who is asking comes from the ``Authorization: Bearer`` header, as for the API: an app's OAuth access token (``lo_…``, +what MCP clients get by signing in through app/api/v1/routes/oauth.py), an API token or a session. Without one the +answer is a 401 pointing at ``/.well-known/oauth-protected-resource/mcp`` (RFC 9728), where clients find out how to +sign in. The tools only read (app/api/mcp_tools.py), and only what the person can read through the API. +""" + +from __future__ import annotations + +import base64 +import binascii +import json +import logging +import re +import urllib.parse +from typing import Any + +from fastapi import APIRouter, HTTPException, Request +from fastapi.responses import JSONResponse, Response +from starlette.concurrency import run_in_threadpool + +from app.api import mcp_tools +from app.api.deps import Access, Db, optional_user +from app.api.v1.routes.oauth import MCP, public_base, web_app_base +from app.config import settings +from app.domain import __version__ + +log = logging.getLogger(__name__) + +router = APIRouter(include_in_schema=False, tags=["mcp"]) + +PATH = MCP +# newest first: an initialize asking for another version gets the newest (the client then decides whether to go on) +HANDSHAKE = ("2025-11-25", "2025-06-18", "2025-03-26", "2024-11-05") +PER_REQUEST = ("2026-07-28",) +MAX_BODY = 1_000_000 +SERVER_INFO = {"name": "lens", "title": settings.PROJECT_NAME, "version": __version__} +CAPABILITIES = {"tools": {"listChanged": False}} +INSTRUCTIONS = ( + "Lens is a private archive of recorded speech, video, documents and images, with transcripts, speakers, the people, " + "organisations and things mentioned (entities) and a knowledge graph between them. You see what the person who " + "signed you in can read. Find moments with search, read a recording with get_recording and get_transcript, and " + "follow entities and the graph with list_entities, get_entity, explore_graph and find_path. When you quote the " + "archive, cite each quote with the link given next to it (or ask cite for one): it opens the recording at that moment." +) +# the per-request era's envelope (params._meta) and headers +META_VERSION = "io.modelcontextprotocol/protocolVersion" +META_CAPABILITIES = "io.modelcontextprotocol/clientCapabilities" +META_SERVER = "io.modelcontextprotocol/serverInfo" +H_VERSION, H_METHOD, H_NAME = "mcp-protocol-version", "mcp-method", "mcp-name" + +PARSE_ERROR, INVALID_REQUEST, METHOD_NOT_FOUND, INVALID_PARAMS, INTERNAL_ERROR = -32700, -32600, -32601, -32602, -32603 +HEADER_MISMATCH, UNSUPPORTED_VERSION = -32020, -32022 +# the per-request era answers protocol errors with these statuses (other errors and results are 200) +STATUS = { + PARSE_ERROR: 400, + INVALID_REQUEST: 400, + INVALID_PARAMS: 400, + HEADER_MISMATCH: 400, + UNSUPPORTED_VERSION: 400, + METHOD_NOT_FOUND: 404, +} + + +class Fault(Exception): + """A JSON-RPC error to answer with.""" + + def __init__(self, code: int, message: str, data: Any = None) -> None: + super().__init__(message) + self.code, self.message, self.data = code, message, data + + def error(self, mid: Any) -> dict[str, Any]: + err: dict[str, Any] = {"code": self.code, "message": self.message} + if self.data is not None: + err["data"] = self.data + return {"jsonrpc": "2.0", "id": mid, "error": err} + + +def _unsupported(requested: Any) -> Fault: + return Fault(UNSUPPORTED_VERSION, "Unsupported protocol version", {"supported": list(PER_REQUEST), "requested": requested}) + + +def _origin_ok(request: Request) -> bool: + """Browsers say where a page that calls us came from: only Lens's own addresses and the configured CORS origins + may (MCP's guard against DNS rebinding). Clients that aren't browsers send no Origin.""" + origin = request.headers.get("origin") + if not origin: + return True + if origin == "null": + return False + mine = {public_base(request), web_app_base(request), settings.FRONTEND_URL.rstrip("/"), *(o.rstrip("/") for o in settings.CORS_ORIGINS)} + if origin.rstrip("/") in mine: + return True + host = (urllib.parse.urlsplit(origin).hostname or "").lower() + allowed = [h.lower() for h in request.app.state.settings.current()["server"]["allowed_hosts"]] + return bool(host) and "*" not in allowed and host in allowed + + +def dispatch(ctx: mcp_tools.Context, method: Any, params: Any, per_request: bool) -> dict[str, Any]: + """The result of one request, in either era; Fault for a protocol error.""" + if not isinstance(params, dict): + raise Fault(INVALID_PARAMS, "params is an object") + if method == "initialize" and not per_request: + asked = params.get("protocolVersion") + return { + "protocolVersion": asked if asked in HANDSHAKE else HANDSHAKE[0], + "capabilities": CAPABILITIES, + "serverInfo": SERVER_INFO, + "instructions": INSTRUCTIONS, + } + if method == "server/discover" and per_request: + return {"supportedVersions": [*PER_REQUEST, *HANDSHAKE], "capabilities": CAPABILITIES, "instructions": INSTRUCTIONS} + if method == "ping" and not per_request: # the per-request era has no ping + return {} + if method == "tools/list": + return {"tools": mcp_tools.listing()} + if method == "tools/call": + name, args = params.get("name"), params.get("arguments") or {} + if not isinstance(name, str) or name not in mcp_tools.TOOLS: + raise Fault(INVALID_PARAMS, f"unknown tool: {name}") + if not isinstance(args, dict): + raise Fault(INVALID_PARAMS, "arguments is an object") + try: + return mcp_tools.call(ctx, name, args) + except Exception: + log.exception("MCP tool %s failed", name) + raise Fault(INTERNAL_ERROR, "the tool failed; the server log has the details") from None + raise Fault(METHOD_NOT_FOUND, f"method not found: {method}") + + +def handle(message: Any, ctx: mcp_tools.Context) -> dict[str, Any] | None: + """The handshake era: the answer to one JSON-RPC message, or None for notifications and responses.""" + if not isinstance(message, dict) or message.get("jsonrpc") != "2.0": + return Fault(INVALID_REQUEST, "not a JSON-RPC 2.0 message").error(None) + if "method" not in message or "id" not in message: + return None # a response (we never ask the client anything) or a notification: nothing to say + mid = message["id"] + if not isinstance(mid, (str, int)) or isinstance(mid, bool): + return Fault(INVALID_REQUEST, "id is a string or a number").error(None) + try: + return {"jsonrpc": "2.0", "id": mid, "result": dispatch(ctx, message["method"], message.get("params") or {}, False)} + except Fault as f: + return f.error(mid) + + +def _header_value(value: str | None) -> str | None: + """A header's value, decoding the =?base64?…?= form clients use for what isn't plain ASCII (None if it's broken).""" + m = re.fullmatch(r"=\?base64\?([A-Za-z0-9+/]*={0,2})\?=", value or "") + if not m: + return value + try: + raw = base64.b64decode(m.group(1), validate=True) + return raw.decode() if base64.b64encode(raw).decode() == m.group(1) else None + except (binascii.Error, UnicodeDecodeError): + return None + + +def _envelope(message: dict[str, Any], request: Request) -> None: + """The per-request era's checks on a request, in the protocol's order: the envelope in params._meta, the + headers agreeing with the body, then the version.""" + meta = (message.get("params") or {}).get("_meta") if isinstance(message.get("params"), dict) else None + if not isinstance(meta, dict): + raise Fault(INVALID_PARAMS, f"params._meta must be an object carrying {META_VERSION!r} and {META_CAPABILITIES!r}") + missing = [k for k in (META_VERSION, META_CAPABILITIES) if k not in meta] + if missing: + raise Fault(INVALID_PARAMS, f"params._meta is missing {', '.join(missing)}") + for h in (H_VERSION, H_METHOD, H_NAME): + if len(request.headers.getlist(h)) > 1: + raise Fault(HEADER_MISMATCH, f"{h} header appears more than once") + version = meta[META_VERSION] + if request.headers.get(H_VERSION) != version: + raise Fault(HEADER_MISMATCH, f"{H_VERSION} header does not match the request envelope's protocol version") + if request.headers.get(H_METHOD) != message["method"]: + raise Fault(HEADER_MISMATCH, f"{H_METHOD} header does not match the request body's method") + if message["method"] == "tools/call": + name = message["params"].get("name") + if name is not None and _header_value(request.headers.get(H_NAME)) != name: + raise Fault(HEADER_MISMATCH, f"{H_NAME} header does not match the request body's 'name' parameter") + if not isinstance(version, str): + raise Fault(INVALID_PARAMS, "the protocol-version envelope value must be a string") + if version not in PER_REQUEST: + raise _unsupported(version) + + +def handle_per_request(message: Any, request: Request, ctx: mcp_tools.Context) -> tuple[dict[str, Any] | None, int]: + """The per-request era: (the answer, its HTTP status), or (None, 202) for a notification.""" + if not isinstance(message, dict) or message.get("jsonrpc") != "2.0" or not isinstance(message.get("method"), str): + f = Fault(INVALID_REQUEST, "Body must be a single JSON-RPC request or notification object") + return f.error(None), 400 + if "id" not in message: + version = request.headers.get(H_VERSION) + if version not in PER_REQUEST: + return _unsupported(version).error(None), 400 + return None, 202 # this era defines no notifications to the server; they're acknowledged and dropped + mid = message["id"] + if not isinstance(mid, (str, int)) or isinstance(mid, bool): + return Fault(INVALID_REQUEST, "id is a string or a number").error(None), 400 + try: + _envelope(message, request) + result = dispatch(ctx, message["method"], message.get("params") or {}, True) + except Fault as f: + return f.error(mid), STATUS.get(f.code, 200) + result.setdefault("resultType", "complete") + if message["method"] in ("tools/list", "server/discover"): + result.update(ttlMs=0, cacheScope="private") # what the tools are doesn't change, but who may use them does + result["_meta"] = {**result.get("_meta", {}), META_SERVER: SERVER_INFO} + return {"jsonrpc": "2.0", "id": mid, "result": result}, 200 + + +@router.post(PATH) +async def post(request: Request, db: Db) -> Response: + if not _origin_ok(request): + return JSONResponse(Fault(INVALID_REQUEST, "this origin may not use the MCP server").error(None), status_code=403) + accept = request.headers.get("accept", "") + if accept and not any(t in accept for t in ("application/json", "application/*", "*/*")): + return Response(status_code=406) # every answer here is JSON + user = await run_in_threadpool(optional_user, request, db) # the database blocks + if not user: + raise HTTPException(401, "sign in first", headers={"WWW-Authenticate": "Bearer"}) + body = await request.body() + if len(body) > MAX_BODY: + return JSONResponse(Fault(INVALID_REQUEST, "the message is too large").error(None), status_code=413) + try: + message = json.loads(body) + except (ValueError, UnicodeDecodeError, RecursionError): + return JSONResponse(Fault(PARSE_ERROR, "Parse error").error(None), status_code=400) + ctx = mcp_tools.Context(request, db, request.app.state.settings.current(), Access(request, db, user), web_app_base(request)) + version = request.headers.get(H_VERSION) + if version is not None and version not in HANDSHAKE: + answer, status = await run_in_threadpool(handle_per_request, message, request, ctx) + return JSONResponse(answer, status_code=status) if answer is not None else Response(status_code=status) + if isinstance(message, list): + if not message: + return JSONResponse(Fault(INVALID_REQUEST, "an empty batch").error(None), status_code=400) + answers = [a for m in message if (a := await run_in_threadpool(handle, m, ctx)) is not None] + return JSONResponse(answers) if answers else Response(status_code=202) + answer = await run_in_threadpool(handle, message, ctx) + return JSONResponse(answer) if answer is not None else Response(status_code=202) + + +@router.get(PATH) +@router.delete(PATH) +def no_stream() -> Response: + """No stream from the server and no sessions to end: every answer comes back on its POST.""" + return Response(status_code=405, headers={"Allow": "POST"}) diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py new file mode 100644 index 000000000..8e8f08a7b --- /dev/null +++ b/fastapi_backend/app/api/mcp_tools.py @@ -0,0 +1,902 @@ +"""The MCP server's tools (app/api/mcp.py, docs/mcp.md): searching, reading and citing the archive. + +Every tool only reads, and reads through the API's own checks (app/api/deps.Access and the route functions), so an +agent sees exactly what the person who signed it in sees: their namespaces, the collections they were given a role on, +and the graphs those open. Results are JSON objects (``structuredContent``, and the same as text for clients that only +read text). Moments carry ``url``, a link to the web app's page that opens the recording there, so answers can cite. +""" + +from __future__ import annotations + +import datetime as dt +import html +import json +import re +from collections.abc import Callable +from dataclasses import dataclass, field +from typing import Any + +from fastapi import HTTPException, Request + +from app.api.deps import Access, Principal +from app.api.v1.routes import entities as entity_routes +from app.api.v1.routes import namespaces as namespace_routes +from app.api.v1.routes import recordings as recording_routes +from app.api.v1.routes import search as search_routes +from app.domain import analyze, library, render, store +from app.domain import entities as ents +from app.domain import speakers as spk +from app.domain.store import DB +from app.schemas.entities import EntityList +from app.schemas.search import SearchResults + +R = store.R +FETCH_CHARS = 100_000 # the most of a transcript fetch returns; get_transcript pages through the rest +TEXT_CHARS = 2000 # a line longer than this is cut (documents can have long blocks) + + +@dataclass +class Context: + """One MCP request: who is asking (through `acl`), the database and configuration, and the web app's address.""" + + request: Request + db: DB + cfg: dict[str, Any] + acl: Access + web: str + + @property + def user(self) -> Principal: + assert self.acl.user is not None # app/api/mcp.py answers 401 before any tool runs + return self.acl.user + + def link(self, rid: int, ms: int | None = None) -> str: + """The recording's page in the web app, at a moment (whole seconds, as the page reads `?t=`).""" + return f"{self.web}/resources/{rid}" + (f"?t={int(ms) // 1000}" if ms and ms >= 1000 else "") + + +class ToolError(Exception): + """A problem the agent can fix (a wrong argument, something it may not see): said to it as the tool's result.""" + + +@dataclass(frozen=True) +class Arg: + name: str + type: str # string, integer, number, boolean, or string[] + description: str + required: bool = False + default: Any = None + minimum: int | None = None + maximum: int | None = None + enum: tuple[str, ...] | None = None + max_length: int | None = None + + def schema(self) -> dict[str, Any]: + if self.type == "string[]": + s: dict[str, Any] = {"type": "array", "items": {"type": "string", **({"enum": list(self.enum)} if self.enum else {})}} + else: + s = {"type": self.type} + if self.enum: + s["enum"] = list(self.enum) + s["description"] = self.description + if self.minimum is not None: + s["minimum"] = self.minimum + if self.maximum is not None: + s["maximum"] = self.maximum + if self.max_length is not None: + s["maxLength"] = self.max_length + if self.default is not None: + s["default"] = self.default + return s + + def parse(self, value: Any) -> Any: + if value is None: + if self.required: + raise ToolError(f"{self.name} is required") + return self.default + t = self.type + if t == "integer": + if isinstance(value, float) and value.is_integer(): + value = int(value) + elif isinstance(value, str) and re.fullmatch(r"-?\d+", value.strip()): + value = int(value) + if not isinstance(value, int) or isinstance(value, bool): + raise ToolError(f"{self.name} is a whole number") + elif t == "number": + if isinstance(value, str): + try: + value = float(value) + except ValueError: + raise ToolError(f"{self.name} is a number") from None + if not isinstance(value, (int, float)) or isinstance(value, bool): + raise ToolError(f"{self.name} is a number") + elif t == "boolean": + if not isinstance(value, bool): + raise ToolError(f"{self.name} is true or false") + elif t == "string": + if isinstance(value, (int, float)) and not isinstance(value, bool): + value = str(value) + if not isinstance(value, str): + raise ToolError(f"{self.name} is a string") + value = value.strip() + if self.max_length is not None and len(value) > self.max_length: + raise ToolError(f"{self.name} is at most {self.max_length} characters") + if not value: + if self.required: + raise ToolError(f"{self.name} is required") + return self.default + elif t == "string[]": + if isinstance(value, str): + value = [x for x in value.split(",")] + if not isinstance(value, list) or not all(isinstance(x, str) for x in value): + raise ToolError(f"{self.name} is a list of strings") + value = [x.strip() for x in value if x.strip()] + if self.enum: + bad = [x for x in value if x not in self.enum] + if bad: + raise ToolError(f"{self.name}: {', '.join(bad)} isn't one of {', '.join(self.enum)}") + return value or self.default + if self.enum and t != "string[]" and value not in self.enum: + raise ToolError(f"{self.name} is one of {', '.join(self.enum)}") + if self.minimum is not None and value < self.minimum: + raise ToolError(f"{self.name} is at least {self.minimum}") + if self.maximum is not None and value > self.maximum: + raise ToolError(f"{self.name} is at most {self.maximum}") + return value + + +@dataclass(frozen=True) +class Tool: + name: str + title: str + description: str + args: tuple[Arg, ...] + run: Callable[..., dict[str, Any]] = field(compare=False) + + def definition(self) -> dict[str, Any]: + return { + "name": self.name, + "title": self.title, + "description": self.description, + "inputSchema": { + "type": "object", + "properties": {a.name: a.schema() for a in self.args}, + "required": [a.name for a in self.args if a.required], + "additionalProperties": False, + }, + "annotations": { + "title": self.title, + "readOnlyHint": True, + "destructiveHint": False, + "idempotentHint": True, + "openWorldHint": False, + }, + } + + +TOOLS: dict[str, Tool] = {} + + +def tool(name: str, title: str, description: str, *args: Arg) -> Callable[[Callable[..., dict[str, Any]]], Callable[..., dict[str, Any]]]: + def register(fn: Callable[..., dict[str, Any]]) -> Callable[..., dict[str, Any]]: + TOOLS[name] = Tool(name, title, description, args, fn) + return fn + + return register + + +def listing() -> list[dict[str, Any]]: + return [t.definition() for t in TOOLS.values()] + + +def _text(out: dict[str, Any], error: bool = False) -> dict[str, Any]: + result: dict[str, Any] = {"content": [{"type": "text", "text": json.dumps(out, ensure_ascii=False, default=str)}], "isError": error} + if not error: + result["structuredContent"] = out + return result + + +def call(ctx: Context, name: str, arguments: dict[str, Any]) -> dict[str, Any]: + """Run a tool. What the agent can fix (bad arguments, something not found or not theirs) comes back as a result + with isError, as MCP has it, so the model sees it and can try again.""" + t = TOOLS[name] + known = {a.name for a in t.args} + try: + unknown = sorted(set(arguments) - known) + if unknown: + raise ToolError(f"unknown argument {', '.join(unknown)}; {name} takes {', '.join(a.name for a in t.args) or 'none'}") + values = {a.name: a.parse(arguments.get(a.name)) for a in t.args} + return _text(t.run(ctx, **values)) + except ToolError as e: + return _text({"error": str(e)}, error=True) + except HTTPException as e: + if e.status_code == 404: + return _text({"error": "not found, or not something you can read"}, error=True) + return _text({"error": str(e.detail)}, error=True) + except KeyError: + return _text({"error": "not found, or not something you can read"}, error=True) + except ValueError as e: + return _text({"error": str(e)}, error=True) + + +# ---------- shared shapes ---------- +def _plain(snippet: str) -> str: + """A search snippet (escaped HTML with around matches) as text, with the matches in **bold**.""" + return html.unescape(snippet.replace("", "**").replace("", "**")) + + +def _cut(text: str | None, n: int = TEXT_CHARS) -> str: + text = text or "" + return text if len(text) <= n else text[: n - 1] + "…" + + +def _seconds(ms: int | None) -> float | None: + return None if ms is None else round(ms / 1000, 1) + + +def _day(value: str | None, name: str) -> dt.date | None: + if not value: + return None + try: + return dt.date.fromisoformat(value[:10]) + except ValueError: + raise ToolError(f"{name} is a date like 2024-05-31") from None + + +def _names(db: DB, ids: list[int | None]) -> dict[int, str]: + return render.speaker_names(db, [i for i in ids if i]) + + +def _line(ctx: Context, rid: int, s: dict[str, Any], names: dict[int, str]) -> dict[str, Any]: + out: dict[str, Any] = { + "line": s["idx"], + "at": store.tc(s.get("t0")), + "seconds": _seconds(s.get("t0")), + "end_seconds": _seconds(s.get("t1")), + "speaker": names.get(s["speaker"]) if s.get("speaker") else None, + "text": _cut(s.get("text")), + "url": ctx.link(rid, s.get("t0")), + } + if s.get("page") is not None: + out["page"] = s["page"] + 1 + return out + + +def _segments(db: DB, rid: int, where: str = "", limit: int | None = None, **params: Any) -> list[dict[str, Any]]: + sql = f"SELECT idx, t0, t1, speaker, text, page FROM segment WHERE recording = $r{where} ORDER BY idx" + if limit is not None: + sql += f" LIMIT {int(limit)}" + return db.rows(sql, r=rid, **params) + + +def _recording(ctx: Context, rid: int) -> dict[str, Any]: + """The recording row, if this person may read it (404 otherwise).""" + return ctx.acl.recording(rid) + + +# ---------- searching ---------- +@tool( + "search", + "Search the archive", + "Full-text search for moments in the archive: what was said in recordings (transcripts), text shown on screen in " + "videos or written in documents and images, supplementary files, and kinds of object seen. Words are all required " + '(matched after stemming), "quoted phrases" must appear as written, OR separates alternatives. Results are the best ' + "moments first, each with the recording, who said it, when in the recording, the matching text (matches in **bold**) " + "and a url that opens the recording at that moment. Use get_transcript for the lines around a moment.", + Arg("query", "string", 'what to look for: words, "a phrase", or alternatives separated by OR', required=True, max_length=500), + Arg("namespace", "string", "only this namespace (see list_namespaces)"), + Arg("speaker_id", "integer", "only what this speaker said (ids from list_speakers or get_recording)"), + Arg("recording_id", "integer", "only within this recording"), + Arg("limit", "integer", "how many moments", default=10, minimum=1, maximum=50), + Arg("offset", "integer", "skip this many (for the next page)", default=0, minimum=0, maximum=10000), +) +def search( + ctx: Context, query: str, namespace: str | None, speaker_id: int | None, recording_id: int | None, limit: int, offset: int +) -> dict[str, Any]: + if recording_id is not None: + _recording(ctx, recording_id) + res = search_routes.search_transcripts( + ctx.acl, + ctx.user, + ctx.db, + q=query, + ns=namespace, + speaker=speaker_id, + emotion=None, + recording=recording_id, + object=None, + limit=limit, + offset=offset, + facets=False, + ) + found = SearchResults.model_validate(res) # the route hands back the dict it signed + results = [] + for h in found.hits: + hit: dict[str, Any] = { + "id": str(h.recording_id), + "recording_id": h.recording_id, + "title": h.title, + "namespace": h.namespace, + "recorded_at": h.recorded_at, + "speaker": h.speaker, + "speaker_id": h.speaker_id, + "found": h.source, + "at": store.tc(h.t0) if h.t0 is not None else None, + "seconds": _seconds(h.t0), + "line": h.idx, + "page": h.page + 1 if h.page is not None else None, + "file": h.file_label, + "text": _plain(h.snippet), + "url": ctx.link(h.recording_id, h.t0), + } + results.append({k: v for k, v in hit.items() if v is not None}) + out: dict[str, Any] = {"query": found.query, "total": found.total, "results": results} + if found.capped: + out["capped"] = True + if offset + len(results) < found.total: + out["next_offset"] = offset + len(results) + return out + + +# ---------- browsing ---------- +@tool( + "list_namespaces", + "List namespaces", + "The namespaces (separate parts of the archive) you can read, with how many recordings and hours each has and your " + "role there. `partial` ones you see only some collections of.", +) +def list_namespaces(ctx: Context) -> dict[str, Any]: + rows = namespace_routes.list_namespaces(ctx.acl, ctx.user, ctx.db) + return { + "namespaces": [ + { + "name": n.name, + "role": n.role, + "partial": n.partial, + "recordings": n.recordings, + "hours": round((n.ms or 0) / 3_600_000, 1), + "speakers": n.speakers, + "graph": n.graph, + } + for n in rows + ] + } + + +SORTS = ("-date", "date", "title", "-title", "duration", "-duration") +MEDIA = ("audio", "video", "transcript", "document", "image") + + +@tool( + "list_recordings", + "List recordings", + "Recordings (and documents, images and transcripts) you can read, newest first by default, with filters. `query` " + "matches words in the title, the namespace or a speaker's name; use search for what is said in them.", + Arg("namespace", "string", "only this namespace"), + Arg("query", "string", "words that must all appear in the title, the namespace or a speaker's name", max_length=200), + Arg("date_from", "string", "recorded on or after this day (YYYY-MM-DD)"), + Arg("date_to", "string", "recorded on or before this day (YYYY-MM-DD)"), + Arg("speaker_id", "integer", "only recordings this speaker is in"), + Arg("tag", "string", "only recordings with this tag", max_length=100), + Arg("media", "string", "only this kind", enum=MEDIA), + Arg("sort", "string", "order: date, title or duration; - first for descending", default="-date", enum=SORTS), + Arg("limit", "integer", "how many", default=20, minimum=1, maximum=100), + Arg("offset", "integer", "skip this many (for the next page)", default=0, minimum=0), +) +def list_recordings( + ctx: Context, + namespace: str | None, + query: str | None, + date_from: str | None, + date_to: str | None, + speaker_id: int | None, + tag: str | None, + media: str | None, + sort: str, + limit: int, + offset: int, +) -> dict[str, Any]: + spaces, within = ctx.acl.scope(namespace) + rows, total = library.list_recordings( + ctx.db, + spaces, + within=within, + sort=sort, + limit=limit, + offset=offset, + q=query, + speakers=[speaker_id] if speaker_id is not None else None, + date_from=_day(date_from, "date_from"), + date_to=_day(date_to, "date_to"), + media=media, + tags=[tag] if tag else None, + cfg=ctx.cfg, + ) + out: dict[str, Any] = { + "total": total, + "recordings": [ + { + "recording_id": r["id"], + "title": r.get("title"), + "namespace": r.get("namespace"), + "recorded_at": r.get("recorded_at"), + "duration": store.tc(r.get("duration_ms")) if r.get("duration_ms") else None, + "media": r.get("media_kind"), + "status": r.get("status"), + "speakers": r.get("speakers") or None, + "tags": r.get("tags") or None, + "language": r.get("language"), + "url": ctx.link(r["id"]), + } + for r in rows + ], + } + if offset + len(rows) < total: + out["next_offset"] = offset + len(rows) + return out + + +@tool( + "list_speakers", + "List speakers", + "The speakers (voices) of the namespaces you have a role in, the ones who talk most first: their ids filter search " + "and list_recordings.", + Arg("namespace", "string", "only this namespace"), + Arg("query", "string", "only speakers whose name contains this", max_length=100), + Arg("limit", "integer", "how many", default=50, minimum=1, maximum=500), +) +def list_speakers(ctx: Context, namespace: str | None, query: str | None, limit: int) -> dict[str, Any]: + spaces = [ctx.acl.namespace(namespace)] if namespace else ctx.acl.spaces() + names = store.space_names(ctx.db) + rows = [ + { + "speaker_id": s["id"], + "name": s["display"], + "named": bool(s.get("name")), + "namespace": names.get(sid), + "recordings": s["recordings"], + "talk_minutes": round((s.get("talk_ms") or 0) / 60000, 1), + } + for sid in spaces + for s in spk.list_speakers(ctx.db, sid) + if s["recordings"] and (not query or query.lower() in (s["display"] or "").lower()) + ] + rows.sort(key=lambda s: -s["talk_minutes"]) + return {"total": len(rows), "speakers": rows[:limit]} + + +# ---------- reading ---------- +@tool( + "get_recording", + "Get a recording", + "One recording: what it is, when it was recorded, its speakers, summary, chapters (sections), the entities and " + "keywords most mentioned in it, and a url to it. Read what was said with get_transcript.", + Arg("recording_id", "integer", "the recording", required=True), +) +def get_recording(ctx: Context, recording_id: int) -> dict[str, Any]: + rec = recording_routes.get_recording(recording_id, ctx.acl, ctx.db, ctx.cfg) + raw = ( + ctx.db.one("SELECT recorded_at, duration_ms, source, media, language, pages, analyzed_at FROM $r", r=R("recording", recording_id)) + or {} + ) + sections = ctx.db.rows("SELECT idx, t0, t1, title FROM section WHERE recording = $r ORDER BY idx", r=recording_id) + found = _entities(ctx, None, None, None, recording_id, "mentions", 30, 0) # counted as the entity index counts them for this person + summary = rec.summary or {} + out: dict[str, Any] = { + "recording_id": recording_id, + "title": rec.title, + "namespace": rec.namespace, + "collection": " / ".join(c.name for c in rec.collection_path) or None, + "recorded_at": raw.get("recorded_at"), + "duration": store.tc(raw["duration_ms"]) if raw.get("duration_ms") else None, + "media": render.kind(raw), + "pages": raw.get("pages"), + "language": raw.get("language"), + "status": rec.status, + "tags": rec.tags or None, + "your_role": rec.role, + "speakers": [{"speaker_id": s.id, "name": s.name} for s in rec.speakers], + "summary": summary.get("summary"), + "key_points": summary.get("key_points") or None, + "topics": summary.get("topics") or None, + "action_items": summary.get("action_items") or None, + "sections": [ + { + "title": s.get("title"), + "at": store.tc(s.get("t0")), + "seconds": _seconds(s.get("t0")), + "url": ctx.link(recording_id, s.get("t0")), + } + for s in sections + ] + or None, + "entities": [{"entity_id": e["id"], "name": e["name"], "type": e["type_label"], "mentions": e["mentions"]} for e in found.items] + or None, + "keywords": [k[0] if isinstance(k, (list, tuple)) else k.get("word", k) for k in analyze.keywords(ctx.db, recording_id, 20)] + if raw.get("analyzed_at") + else None, + "url": ctx.link(recording_id), + } + if rec.email: + out["email"] = rec.email.model_dump(exclude_none=True) + if rec.web: + out["web_page"] = rec.web.model_dump(exclude_none=True) + return {k: v for k, v in out.items() if v is not None} + + +@tool( + "get_transcript", + "Read a transcript", + "The lines of a recording's transcript (or a document's or image's text), in order: who said what, when, each with " + "a url that opens the recording at that line. Read a long one in pages: from_line (next_line of the last page) or " + "a time window in seconds.", + Arg("recording_id", "integer", "the recording", required=True), + Arg("from_line", "integer", "start at this line (0 is the first)", minimum=0), + Arg("start_seconds", "number", "start at the line being said at this second", minimum=0), + Arg("end_seconds", "number", "stop before the lines that start after this second", minimum=0), + Arg("limit", "integer", "the most lines", default=200, minimum=1, maximum=1000), +) +def get_transcript( + ctx: Context, recording_id: int, from_line: int | None, start_seconds: float | None, end_seconds: float | None, limit: int +) -> dict[str, Any]: + rec = _recording(ctx, recording_id) + where, params = "", {} + if from_line is not None: + where += " AND idx >= $from" + params["from"] = from_line + if start_seconds is not None: + where += " AND t1 > $a" + params["a"] = int(start_seconds * 1000) + if end_seconds is not None: + where += " AND t0 <= $b" + params["b"] = int(end_seconds * 1000) + segs = _segments(ctx.db, recording_id, where, limit + 1, **params) + more, segs = len(segs) > limit, segs[:limit] + names = _names(ctx.db, [s.get("speaker") for s in segs]) + total = ctx.db.values("RETURN array::len((SELECT VALUE id FROM segment WHERE recording = $r))", r=recording_id) + out: dict[str, Any] = { + "recording_id": recording_id, + "title": rec.get("title"), + "lines_in_all": int(total[0]) if total else 0, + "lines": [_line(ctx, recording_id, s, names) for s in segs], + } + if more: + out["next_line"] = segs[-1]["idx"] + 1 + return out + + +@tool( + "fetch", + "Fetch a recording's text", + "A recording's whole text at once, as timestamped lines ([m:ss] Speaker: words), with its title and url; ids come " + "from search or list_recordings. A very long one is cut, and says where get_transcript can go on from.", + Arg("id", "string", "the recording id", required=True, max_length=40), +) +def fetch(ctx: Context, id: str) -> dict[str, Any]: + m = re.match(r"\s*(\d+)", id) + if not m: + raise ToolError("id is a recording id, like 12") + rid = int(m.group(1)) + rec = _recording(ctx, rid) + segs = _segments(ctx.db, rid) + names = _names(ctx.db, [s.get("speaker") for s in segs]) + parts, size, cut_at = [], 0, None + for s in segs: + who = names.get(s["speaker"]) if s.get("speaker") else None + where = f"p. {s['page'] + 1}" if s.get("page") is not None else store.tc(s.get("t0")) + line = f"[{where}] " + (f"{who}: " if who else "") + (s.get("text") or "") + if size + len(line) > FETCH_CHARS: + cut_at = s["idx"] + break + parts.append(line) + size += len(line) + 1 + space = store.space_names(ctx.db).get(rec["space"]) + meta: dict[str, Any] = { + "namespace": space, + "recorded_at": rec.get("recorded_at"), + "duration": store.tc(rec.get("duration_ms")) if rec.get("duration_ms") else None, + "lines": len(segs), + } + if cut_at is not None: + meta["truncated"] = True + meta["next_line"] = cut_at + return { + "id": str(rid), + "title": rec.get("title") or f"Recording {rid}", + "text": "\n".join(parts), + "url": ctx.link(rid), + "metadata": {k: v for k, v in meta.items() if v is not None}, + } + + +@tool( + "cite", + "Cite a moment", + "A citation for a moment of a recording: the exact words (one line, or a few from there), who said them, the " + "recording, when it was recorded, the time in it, a url that opens it there, and the whole citation as Markdown. " + "Name the moment by line (from search or get_transcript) or by second.", + Arg("recording_id", "integer", "the recording", required=True), + Arg("line", "integer", "the line (from search or get_transcript)", minimum=0), + Arg("seconds", "number", "or the moment, in seconds from the start", minimum=0), + Arg("lines", "integer", "how many lines to quote from there", default=1, minimum=1, maximum=10), +) +def cite(ctx: Context, recording_id: int, line: int | None, seconds: float | None, lines: int) -> dict[str, Any]: + if line is None and seconds is None: + raise ToolError("give line or seconds") + rec = _recording(ctx, recording_id) + if line is not None: + segs = _segments(ctx.db, recording_id, " AND idx >= $i", lines, i=line) + if not segs or segs[0]["idx"] != line: + raise ToolError(f"recording {recording_id} has no line {line}") + else: + ms = int((seconds or 0) * 1000) + # the line being said then, else the next one + segs = _segments(ctx.db, recording_id, " AND t1 > $ms", lines, ms=ms) + if not segs: + raise ToolError(f"nothing is said in recording {recording_id} at or after {store.tc(ms)}") + names = _names(ctx.db, [s.get("speaker") for s in segs]) + first = segs[0] + who = list(dict.fromkeys(names[s["speaker"]] for s in segs if s.get("speaker") and s["speaker"] in names)) + quote = " ".join((s.get("text") or "").strip() for s in segs) + title = rec.get("title") or f"Recording {recording_id}" + space = store.space_names(ctx.db).get(rec["space"]) + when = (rec.get("recorded_at") or "")[:10] or None + page = first.get("page") + at = f"p. {page + 1}" if page is not None else store.tc(first.get("t0")) + url = ctx.link(recording_id, first.get("t0")) + credit = ", ".join(x for x in (" and ".join(who) if who else None, f"*{title}*", when) if x) + return { + "quote": quote, + "speakers": who, + "recording_id": recording_id, + "title": title, + "namespace": space, + "recorded_at": rec.get("recorded_at"), + "line": first["idx"], + "at": at, + "seconds": _seconds(first.get("t0")), + "url": url, + "markdown": f"> {quote}\n>\n> — {credit}, [{at}]({url})", + } + + +# ---------- entities and the graph ---------- +ENTITY_TYPES = tuple(ents.TYPES) + + +def _entity_row(e: dict[str, Any]) -> dict[str, Any]: + out = { + "entity_id": e["id"], + "name": e["name"], + "type": e["type"], + "namespace": e.get("namespace"), + "aliases": e.get("aliases") or None, + "mentions": e.get("mentions"), + "recordings": e.get("recordings"), + "first": e.get("first"), + "last": e.get("last"), + } + return {k: v for k, v in out.items() if v is not None} + + +@tool( + "list_entities", + "Find entities", + "Entities: the people, organisations, products, places, events, works and topics mentioned in recordings, the most " + "mentioned first. Find one by name with query; its entity_id opens it in get_entity and the graph.", + Arg("query", "string", "a name, or part of one", max_length=200), + Arg("types", "string[]", "only these types", enum=ENTITY_TYPES), + Arg("namespace", "string", "only this namespace"), + Arg("recording_id", "integer", "only those mentioned in this recording"), + Arg("sort", "string", "order", default="mentions", enum=("mentions", "recordings", "recent", "rising", "name")), + Arg("limit", "integer", "how many", default=25, minimum=1, maximum=100), + Arg("offset", "integer", "skip this many (for the next page)", default=0, minimum=0), +) +def list_entities( + ctx: Context, + query: str | None, + types: list[str] | None, + namespace: str | None, + recording_id: int | None, + sort: str, + limit: int, + offset: int, +) -> dict[str, Any]: + res = _entities(ctx, query, types, namespace, recording_id, sort, limit, offset) + out: dict[str, Any] = {"total": res.total, "entities": [_entity_row(e) for e in res.items]} + if offset + len(res.items) < res.total: + out["next_offset"] = offset + len(res.items) + return out + + +def _entities( + ctx: Context, + query: str | None, + types: list[str] | None, + namespace: str | None, + recording_id: int | None, + sort: str, + limit: int, + offset: int, +) -> EntityList: + if namespace: + ctx.acl.namespace(namespace) + return entity_routes.list_entities( + ctx.user, + ctx.acl, + ctx.db, + q=query or "", + types=",".join(types or []), + namespaces=namespace or "", + speaker=None, + recording=recording_id, + date_from="", + date_to="", + min_mentions=1, + hidden=False, + sort=sort, + limit=limit, + offset=offset, + group=False, + ) + + +@tool( + "get_entity", + "Get an entity", + "One entity: its names, how often and when it is mentioned, the entities mentioned together with it, who mentions " + "it most, and the lines that mention it (newest first), each with a url to that moment.", + Arg("entity_id", "integer", "the entity (from list_entities, get_recording or the graph)", required=True), + Arg("mentions", "integer", "how many mentioning lines", default=10, minimum=0, maximum=100), + Arg("mentions_offset", "integer", "skip this many lines (for the next page)", default=0, minimum=0), + Arg("sort", "string", "the lines' order", default="newest", enum=("newest", "oldest", "most")), +) +def get_entity(ctx: Context, entity_id: int, mentions: int, mentions_offset: int, sort: str) -> dict[str, Any]: + d = entity_routes.get_entity(entity_id, ctx.user, ctx.acl, ctx.db) + con = entity_routes.get_entity_connections(entity_id, ctx.user, ctx.acl, ctx.db) + out: dict[str, Any] = { + "entity_id": d.id, + "name": d.name, + "type": d.type, + "type_label": d.type_label, + "namespace": d.namespace, + "aliases": d.aliases or None, + "mentions": d.mentions, + "recordings": d.recordings, + "speakers": d.speakers, + "first": d.first, + "last": d.last, + "same_as": [{"entity_id": x.id, "name": x.name, "namespace": x.namespace} for x in d.links] or None, + "related": [ + {"entity_id": x["id"], "name": x["name"], "type": x["type"], "together": x["together"]} + for x in (con.get("entities") or [])[:15] + ] + or None, + "mentioned_most_by": [ + {"speaker_id": x["id"], "name": x["name"], "mentions": x["mentions"]} for x in (con.get("speakers") or [])[:10] + ] + or None, + } + if mentions: + page = entity_routes.list_entity_mentions( + entity_id, + ctx.user, + ctx.acl, + ctx.db, + speaker=None, + recording=None, + date_from="", + date_to="", + sort=sort, + limit=mentions, + offset=mentions_offset, + ) + out["lines"] = [ + { + k: v + for k, v in { + "recording_id": m["recording_id"], + "title": m.get("title"), + "namespace": m.get("namespace"), + "recorded_at": m.get("recorded_at"), + "line": m.get("idx"), + "at": m.get("time"), + "seconds": _seconds(m.get("t0")), + "speaker": m.get("speaker"), + "text": _cut(m.get("text")), + "url": ctx.link(m["recording_id"], m.get("t0")), + }.items() + if v is not None + } + for m in page.items + ] + if mentions_offset + len(page.items) < page.total: + out["next_mentions_offset"] = mentions_offset + len(page.items) + return {k: v for k, v in out.items() if v is not None} + + +def _node(n: str | None, entity_id: int | None, speaker_id: int | None, name: str) -> str: + if n: + if not re.fullmatch(r"[es]\d+", n): + raise ToolError(f"{name} is a node id: e or s, like e12") + return n + if entity_id is not None: + return f"e{entity_id}" + if speaker_id is not None: + return f"s{speaker_id}" + raise ToolError(f"give {name}, an entity_id or a speaker_id") + + +def _scope(namespace: str | None) -> str: + return f"ns:{namespace}" if namespace else "all" + + +@tool( + "explore_graph", + "Explore the knowledge graph", + "The knowledge graph around an entity or a speaker: what and who it is connected to (mentioned together, or " + "mentioned by), the strongest first, one or two steps out. Without a namespace it covers the namespaces that share " + "their graph; a namespace with an isolated graph is explored on its own (namespace=...).", + Arg("node", "string", "the node: e or s", max_length=20), + Arg("entity_id", "integer", "or the entity"), + Arg("speaker_id", "integer", "or the speaker"), + Arg("namespace", "string", "only this namespace's graph"), + Arg("depth", "integer", "how many steps out", default=1, minimum=1, maximum=2), + Arg("types", "string[]", "only entities of these types", enum=ENTITY_TYPES), + Arg("limit", "integer", "the most nodes", default=40, minimum=1, maximum=200), +) +def explore_graph( + ctx: Context, + node: str | None, + entity_id: int | None, + speaker_id: int | None, + namespace: str | None, + depth: int, + types: list[str] | None, + limit: int, +) -> dict[str, Any]: + focus = _node(node, entity_id, speaker_id, "node") + g = entity_routes.explore_graph( + ctx.user, + ctx.acl, + ctx.db, + focus=focus, + scope=_scope(namespace), + depth=depth, + types=",".join(types or []), + kinds="", + min_weight=1, + limit=limit, + ) + return { + "focus": g["focus"], + "nodes": [{k: v for k, v in n.items() if v is not None} for n in g["nodes"]], + "edges": g["edges"], + "truncated": g.get("truncated", False), + } + + +@tool( + "find_path", + "Find how two things connect", + "The shortest chain of links between two entities or speakers in the knowledge graph, with lines from the " + "recordings that show each link (each with a url to that moment).", + Arg("from_node", "string", "one end: e or s", required=True, max_length=20), + Arg("to_node", "string", "the other end", required=True, max_length=20), + Arg("namespace", "string", "only this namespace's graph"), +) +def find_path(ctx: Context, from_node: str, to_node: str, namespace: str | None) -> dict[str, Any]: + a, b = _node(from_node, None, None, "from_node"), _node(to_node, None, None, "to_node") + res = entity_routes.find_graph_path(ctx.user, ctx.acl, ctx.db, a=a, b=b, scope=_scope(namespace)) + for link in res.get("links", []): + link["evidence"] = [ + { + "recording_id": e["recording_id"], + "at": e.get("time"), + "seconds": _seconds(e.get("t0")), + "text": _cut(e.get("text")), + "url": ctx.link(e["recording_id"], e.get("t0")), + } + for e in link.get("evidence", []) + ] + return res diff --git a/fastapi_backend/app/api/v1/routes/oauth.py b/fastapi_backend/app/api/v1/routes/oauth.py index a1d8bea5a..6267dff36 100644 --- a/fastapi_backend/app/api/v1/routes/oauth.py +++ b/fastapi_backend/app/api/v1/routes/oauth.py @@ -42,6 +42,7 @@ HOST = re.compile(r"[A-Za-z0-9.\-]+(:\d+)?|\[[0-9A-Fa-f:.]+\](:\d+)?") RESOURCE = re.compile(r"[A-Za-z0-9_\-]+(/[A-Za-z0-9_\-]+)*") NO_STORE = {"Cache-Control": "no-store", "Pragma": "no-cache"} +MCP, MCP_SCOPE = "/mcp", "read" # the MCP server (app/api/mcp.py), and the scope its tools need def _first(value: str | None) -> str: @@ -72,10 +73,14 @@ def public_base(request: Request) -> str: return settings.FRONTEND_URL.rstrip("/") -def _consent_page(request: Request) -> str: - """The consent page is the web app's: on this address when the request came through it, else FRONTEND_URL.""" +def web_app_base(request: Request) -> str: + """Where people open the web app's pages: this address when the request came through it, else FRONTEND_URL.""" through_web_app = bool(_first(request.headers.get("x-forwarded-host"))) - return (public_base(request) if through_web_app else settings.FRONTEND_URL.rstrip("/")) + "/oauth/authorize" + return public_base(request) if through_web_app else settings.FRONTEND_URL.rstrip("/") + + +def _consent_page(request: Request) -> str: + return web_app_base(request) + "/oauth/authorize" def _discovery(doc: dict[str, Any]) -> JSONResponse: @@ -103,10 +108,12 @@ def _client_credentials(request: Request, client_id: str | None, client_secret: async def bearer_challenge(request: Request, exc: StarletteHTTPException) -> Response: """401s that ask for a bearer token say where to find out how to get one (RFC 9728 §5.1), as MCP clients expect.""" if exc.status_code == 401 and (exc.headers or {}).get("WWW-Authenticate") == "Bearer": - meta = f"{public_base(request)}/.well-known/oauth-protected-resource" - exc = StarletteHTTPException( - 401, exc.detail, headers={**(exc.headers or {}), "WWW-Authenticate": f'Bearer resource_metadata="{meta}"'} - ) + # the MCP server is a resource of its own (/mcp), so clients that check what the metadata names see their URL; + # its tools only read, so that's all it asks for + mcp = request.url.path == MCP + meta = f"{public_base(request)}/.well-known/oauth-protected-resource" + (MCP if mcp else "") + challenge = f'Bearer resource_metadata="{meta}"' + (f', scope="{MCP_SCOPE}"' if mcp else "") + exc = StarletteHTTPException(401, exc.detail, headers={**(exc.headers or {}), "WWW-Authenticate": challenge}) return await http_exception_handler(request, exc) @@ -142,7 +149,7 @@ def protected_resource(request: Request, rest: str = "") -> JSONResponse: "resource": f"{base}/{rest}" if rest else base, "authorization_servers": [base], "bearer_methods_supported": ["header"], - "scopes_supported": list(oauth.SCOPES), + "scopes_supported": [MCP_SCOPE] if f"/{rest}" == MCP else list(oauth.SCOPES), "resource_name": settings.PROJECT_NAME, } return _discovery(doc) diff --git a/fastapi_backend/app/main.py b/fastapi_backend/app/main.py index 6f7343a44..47c4709a9 100644 --- a/fastapi_backend/app/main.py +++ b/fastapi_backend/app/main.py @@ -17,6 +17,7 @@ from starlette.exceptions import HTTPException as StarletteHTTPException from app.api.iiif import router as iiif_router +from app.api.mcp import router as mcp_router from app.api.pages import router as pages_router from app.api.v1.router import api_router from app.api.v1.routes.oauth import bearer_challenge @@ -81,6 +82,7 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]: app.include_router(api_router, prefix="/api/v1") app.include_router(oauth_well_known) + app.include_router(mcp_router) app.add_exception_handler(StarletteHTTPException, bearer_challenge) # type: ignore[arg-type] app.include_router(iiif_router) app.include_router(pages_router) diff --git a/fastapi_backend/pyproject.toml b/fastapi_backend/pyproject.toml index b28c3566d..b4296b588 100644 --- a/fastapi_backend/pyproject.toml +++ b/fastapi_backend/pyproject.toml @@ -64,6 +64,7 @@ dev = [ "mkdocs-material[imaging]>=9.6.9", "onnxruntime>=1.18", # the objects tests run a small YOLOX-shaped model "onnx>=1.16", # ... which they build + "mcp>=2.2", # the official MCP client, to test the MCP server with ] [tool.hatch.build.targets.wheel] diff --git a/fastapi_backend/tests/api/test_mcp.py b/fastapi_backend/tests/api/test_mcp.py new file mode 100644 index 000000000..ee17240cc --- /dev/null +++ b/fastapi_backend/tests/api/test_mcp.py @@ -0,0 +1,301 @@ +"""The MCP server (docs/mcp.md): agents sign in with OAuth, then search, read and cite what the person can read, over +Streamable HTTP in both eras of the protocol (the initialize handshake, and 2026-07-28's per-request envelope).""" + +from __future__ import annotations + +import json +from typing import Any + +import pytest +from fastapi.testclient import TestClient + +from app.api import mcp, mcp_tools +from tests.api.test_oauth import _bearer, _grant +from tests.helpers import login, make_user, seed + +MODERN = "2026-07-28" + + +@pytest.fixture +def env(client, db, cfg, folder): + ids = seed(db, cfg, folder) + make_user(db, "vi@x.io", "viewer password 1", roles={"pods": "viewer"}) + make_user(db, "both@x.io", "both password 1", roles={"pods": "viewer", "calls": "viewer"}) + hv = login(client, "vi@x.io", "viewer password 1") + _, tokens = _grant(client, hv, scope="read") # what an MCP client gets by signing in + return {"ids": ids, "h": _bearer(tokens), "tokens": tokens, "hv": hv, "hb": login(client, "both@x.io", "both password 1")} + + +def rpc(client: TestClient, h: dict[str, str], method: str, params: dict[str, Any] | None = None, **headers: str) -> Any: + r = client.post("/mcp", headers={**h, **headers}, json={"jsonrpc": "2.0", "id": 7, "method": method, "params": params or {}}) + assert r.status_code == 200, r.text + body = r.json() + assert body["id"] == 7 + return body + + +def tool(client: TestClient, h: dict[str, str], name: str, **args: Any) -> dict[str, Any]: + res = rpc(client, h, "tools/call", {"name": name, "arguments": args})["result"] + assert res["isError"] is False, res + assert json.loads(res["content"][0]["text"]) == res["structuredContent"] # the same, for clients that read text + return res["structuredContent"] + + +def tool_error(client: TestClient, h: dict[str, str], name: str, **args: Any) -> str: + res = rpc(client, h, "tools/call", {"name": name, "arguments": args})["result"] + assert res["isError"] is True and "structuredContent" not in res + return json.loads(res["content"][0]["text"])["error"] + + +def test_clients_are_sent_to_sign_in(client, app, env): + # without a token: a 401 that names this resource's own metadata (RFC 9728), which names Lens as the issuer + via = {"x-forwarded-host": "lens.example.org", "x-forwarded-proto": "https"} + web = TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 50000)) + r = web.post("/mcp", headers=via, json={"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}) + assert r.status_code == 401 + assert r.headers["www-authenticate"] == ( + 'Bearer resource_metadata="https://lens.example.org/.well-known/oauth-protected-resource/mcp", scope="read"' + ) + meta = web.get("/.well-known/oauth-protected-resource/mcp", headers=via).json() + assert meta["resource"] == "https://lens.example.org/mcp" and meta["authorization_servers"] == ["https://lens.example.org"] + assert meta["scopes_supported"] == ["read"] # all the tools need + assert client.post("/mcp", headers={"Authorization": "Bearer lo_nope"}, json={}).status_code == 401 + + # the app's read-only token is enough: everything here only reads + init = rpc(client, env["h"], "initialize", {"protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": {"name": "t"}})["result"] + assert init["protocolVersion"] == "2025-06-18" and init["capabilities"] == {"tools": {"listChanged": False}} + assert init["serverInfo"]["name"] == "lens" and "cite" in init["instructions"] + assert rpc(client, env["h"], "initialize", {"protocolVersion": "1999-01-01"})["result"]["protocolVersion"] == "2025-11-25" + tools = rpc(client, env["h"], "tools/list")["result"]["tools"] + assert [t["name"] for t in tools] == list(mcp_tools.TOOLS) + assert all(t["annotations"]["readOnlyHint"] and t["inputSchema"]["type"] == "object" for t in tools) + assert next(t for t in tools if t["name"] == "search")["inputSchema"]["required"] == ["query"] + # API tokens and sessions work too + assert rpc(client, env["hv"], "ping")["result"] == {} + + # taking the app's access away stops it at once + grant = client.get("/api/v1/oauth/grants", headers=env["hv"]).json()[0] + assert client.delete(f"/api/v1/oauth/grants/{grant['id']}", headers=env["hv"]).status_code == 200 + assert client.post("/mcp", headers=env["h"], json={"jsonrpc": "2.0", "id": 1, "method": "ping"}).status_code == 401 + + +def test_the_transport(client, env): + h = env["h"] + post = lambda body, **hd: client.post("/mcp", headers={**h, **hd}, content=body if isinstance(body, str) else json.dumps(body)) # noqa: E731 + # notifications and responses: 202, nothing to say + assert post({"jsonrpc": "2.0", "method": "notifications/initialized"}).status_code == 202 + assert post({"jsonrpc": "2.0", "id": 3, "result": {}}).status_code == 202 + # a batch (2025-03-26) gets an answer per request + r = post([{"jsonrpc": "2.0", "id": 1, "method": "ping"}, {"jsonrpc": "2.0", "method": "notifications/initialized"}]) + assert r.json() == [{"jsonrpc": "2.0", "id": 1, "result": {}}] + assert post([]).status_code == 400 + # JSON-RPC errors + assert post("{nope").json()["error"]["code"] == -32700 + assert post({"id": 1, "method": "ping"}).json()["error"]["code"] == -32600 + assert post({"jsonrpc": "2.0", "id": 1, "method": "resources/list"}).json()["error"]["code"] == -32601 + assert post({"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "drop_tables"}}).json()["error"]["code"] == -32602 + # nothing streams from the server, and there are no sessions + assert client.get("/mcp", headers=h).status_code == 405 and client.delete("/mcp", headers=h).status_code == 405 + # a page on another site can't use someone's browser to call it (DNS rebinding); the web app's own pages can + assert post({"jsonrpc": "2.0", "id": 1, "method": "ping"}, origin="https://evil.example").status_code == 403 + assert post({"jsonrpc": "2.0", "id": 1, "method": "ping"}, origin="http://localhost:3000").status_code == 200 + assert post({"jsonrpc": "2.0", "id": 1, "method": "ping"}, accept="text/html").status_code == 406 + assert post({"jsonrpc": "2.0", "id": 1, "method": "ping"}, **{"mcp-protocol-version": "2025-06-18"}).status_code == 200 + + +def _modern(client, h, method, params=None, *, name=None, meta=None, **headers): + params = dict(params or {}) + params["_meta"] = meta if meta is not None else {mcp.META_VERSION: MODERN, mcp.META_CAPABILITIES: {}} + hd = {**h, "mcp-protocol-version": MODERN, "mcp-method": method, **({"mcp-name": name} if name else {}), **headers} + return client.post("/mcp", headers=hd, json={"jsonrpc": "2.0", "id": "r1", "method": method, "params": params}) + + +def test_the_per_request_era(client, env): + h = env["h"] + r = _modern(client, h, "server/discover") + assert r.status_code == 200 + res = r.json()["result"] + assert res["supportedVersions"][0] == MODERN and "2025-06-18" in res["supportedVersions"] + assert (res["resultType"], res["ttlMs"], res["cacheScope"]) == ("complete", 0, "private") + assert res["_meta"][mcp.META_SERVER]["name"] == "lens" and res["capabilities"]["tools"] == {"listChanged": False} + assert _modern(client, h, "tools/list").json()["result"]["resultType"] == "complete" + r = _modern(client, h, "tools/call", {"name": "list_namespaces", "arguments": {}}, name="list_namespaces") + res = r.json()["result"] + assert res["resultType"] == "complete" and [n["name"] for n in res["structuredContent"]["namespaces"]] == ["pods"] + + # the headers must agree with the body, and the envelope must be there + bad = _modern(client, h, "tools/call", {"name": "list_namespaces", "arguments": {}}, name="search") + assert (bad.status_code, bad.json()["error"]["code"]) == (400, -32020) + bad = _modern(client, h, "tools/list", **{"mcp-method": "tools/call"}) + assert (bad.status_code, bad.json()["error"]["code"]) == (400, -32020) + bad = _modern(client, h, "tools/list", meta={mcp.META_VERSION: MODERN}) + assert (bad.status_code, bad.json()["error"]["code"]) == (400, -32602) + # a version there isn't: the versions there are + bad = _modern( + client, h, "tools/list", meta={mcp.META_VERSION: "2031-01-01", mcp.META_CAPABILITIES: {}}, **{"mcp-protocol-version": "2031-01-01"} + ) + assert (bad.status_code, bad.json()["error"]["code"]) == (400, -32022) + assert bad.json()["error"]["data"] == {"supported": [MODERN], "requested": "2031-01-01"} + # no handshake and no ping in this era; a name that isn't plain ASCII comes base64-encoded + assert _modern(client, h, "initialize").status_code == 404 and _modern(client, h, "ping").status_code == 404 + r = _modern(client, h, "tools/call", {"name": "list_namespaces", "arguments": {}}, name="=?base64?bGlzdF9uYW1lc3BhY2Vz?=") + assert r.status_code == 200 + # notifications are acknowledged and dropped + r = client.post("/mcp", headers={**h, "mcp-protocol-version": MODERN}, json={"jsonrpc": "2.0", "method": "notifications/cancelled"}) + assert r.status_code == 202 + + +def test_search_and_cite(client, env): + h, (ep1, ep2, call) = env["h"], env["ids"] + found = tool(client, h, "search", query="capsid", limit=2) + assert found["total"] == 4 and found["next_offset"] == 2 and len(found["results"]) == 2 + hit = found["results"][0] + assert hit["recording_id"] == ep1 and hit["namespace"] == "pods" and hit["found"] == "said" + assert "**capsid**" in hit["text"] and "" not in hit["text"] + assert hit["url"] == f"http://localhost:3000/resources/{ep1}" + (f"?t={int(hit['seconds'])}" if hit["seconds"] >= 1 else "") + assert {r["recording_id"] for r in tool(client, h, "search", query="capsid", limit=50)["results"]} == {ep1} # not the call + assert tool(client, h, "search", query='"exploit benchmark"', recording_id=ep2)["total"] >= 1 + # what isn't theirs is as if it weren't there + assert tool_error(client, h, "search", query="capsid", namespace="calls") == "not found, or not something you can read" + assert tool_error(client, h, "search", query="capsid", recording_id=call) == "not found, or not something you can read" + + c = tool(client, h, "cite", recording_id=ep1, line=1, lines=2) + assert c["quote"].startswith("Wow, really? Dyno Therapeutics") and c["quote"].endswith("by a wide margin.") + assert c["speakers"] == ["Bob", "Alice"] and c["title"] == "ep1" and c["line"] == 1 + assert c["url"] == f"http://localhost:3000/resources/{ep1}?t={int(c['seconds'])}" + assert c["markdown"].startswith("> Wow, really?") and f"[{c['at']}]({c['url']})" in c["markdown"] + at = tool(client, h, "cite", recording_id=ep1, seconds=c["seconds"] + 0.5) # the line being said then + assert at["line"] == 1 and at["quote"].startswith("Wow, really?") + assert tool_error(client, h, "cite", recording_id=ep1) == "give line or seconds" + assert tool_error(client, h, "cite", recording_id=ep1, line=99) == f"recording {ep1} has no line 99" + assert tool_error(client, h, "cite", recording_id=call, line=0) == "not found, or not something you can read" + + +def test_reading_recordings(client, env): + h, (ep1, ep2, call) = env["h"], env["ids"] + ns = tool(client, h, "list_namespaces")["namespaces"] + assert [(n["name"], n["role"], n["recordings"]) for n in ns] == [("pods", "viewer", 2)] + recs = tool(client, h, "list_recordings") + assert recs["total"] == 2 and {r["recording_id"] for r in recs["recordings"]} == {ep1, ep2} + assert tool(client, h, "list_recordings", query="ep2", sort="title")["recordings"][0]["recording_id"] == ep2 + assert tool_error(client, h, "list_recordings", date_from="last week") == "date_from is a date like 2024-05-31" + assert tool_error(client, h, "list_recordings", namespace="calls") == "not found, or not something you can read" + speakers = tool(client, h, "list_speakers")["speakers"] + assert {s["name"] for s in speakers} == {"Alice", "Bob", "Carol"} + alice = next(s["speaker_id"] for s in speakers if s["name"] == "Alice") + assert {r["speaker"] for r in tool(client, h, "search", query="capsid", speaker_id=alice)["results"]} == {"Alice"} + + rec = tool(client, h, "get_recording", recording_id=ep1) + assert (rec["title"], rec["namespace"], rec["media"], rec["your_role"]) == ("ep1", "pods", "transcript", "viewer") + assert {s["name"] for s in rec["speakers"]} == {"Alice", "Bob"} and rec["url"] == f"http://localhost:3000/resources/{ep1}" + assert "Dyno Therapeutics" in [e["name"] for e in rec["entities"]] and rec["sections"][0]["seconds"] == 0 + assert tool_error(client, h, "get_recording", recording_id=call) == "not found, or not something you can read" + + page = tool(client, h, "get_transcript", recording_id=ep1, limit=2) + assert page["lines_in_all"] == 6 and [x["line"] for x in page["lines"]] == [0, 1] and page["next_line"] == 2 + assert page["lines"][1]["speaker"] == "Bob" and page["lines"][1]["url"].endswith(f"/resources/{ep1}?t=6") + rest = tool(client, h, "get_transcript", recording_id=ep1, from_line=page["next_line"]) + assert [x["line"] for x in rest["lines"]] == [2, 3, 4, 5] and "next_line" not in rest + window = tool(client, h, "get_transcript", recording_id=ep1, start_seconds=7, end_seconds=12) + assert [x["line"] for x in window["lines"]] == [1, 2] + assert tool_error(client, h, "get_transcript", recording_id=ep1, limit=0) == "limit is at least 1" + assert tool_error(client, h, "get_transcript", recording_id="ep1") == "recording_id is a whole number" + assert tool_error(client, h, "get_transcript", recording_id=ep1, page=2).startswith("unknown argument page;") + + doc = tool(client, h, "fetch", id=str(ep2)) + assert doc["id"] == str(ep2) and doc["title"] == "ep2" and doc["url"] == f"http://localhost:3000/resources/{ep2}" + assert doc["text"].splitlines()[1] == "[0:04] Carol: The exploit benchmark measures how often a model finishes a working exploit." + assert doc["metadata"]["lines"] == 4 and "truncated" not in doc["metadata"] + assert tool_error(client, h, "fetch", id=str(call)) == "not found, or not something you can read" + + +def test_entities_and_the_graph(client, env, monkeypatch): + h, hb, (ep1, ep2, _) = env["h"], env["hb"], env["ids"] + found = tool(client, h, "list_entities", query="dyno") + assert found["total"] == 1 # the calls namespace has one too, which this person can't read + dyno = found["entities"][0] + assert (dyno["name"], dyno["type"], dyno["namespace"], dyno["recordings"]) == ("Dyno Therapeutics", "ORG", "pods", 2) + assert tool(client, h, "list_entities", types=["ORG"], recording_id=ep2)["entities"][0]["entity_id"] == dyno["entity_id"] + assert tool_error(client, h, "list_entities", types=["SPACESHIP"]).startswith("types: SPACESHIP isn't one of") + both = tool(client, hb, "list_entities", query="dyno")["entities"] + assert sorted(e["namespace"] for e in both) == ["calls", "pods"] + in_calls = next(e["entity_id"] for e in both if e["namespace"] == "calls") + assert tool_error(client, h, "get_entity", entity_id=in_calls) == "not found, or not something you can read" + + e = tool(client, h, "get_entity", entity_id=dyno["entity_id"], mentions=2) + assert e["name"] == "Dyno Therapeutics" and e["mentions"] == 4 and e["next_mentions_offset"] == 2 + assert {m["name"] for m in e["mentioned_most_by"]} == {"Alice", "Bob", "Carol"} + line = e["lines"][0] + assert line["recording_id"] in (ep1, ep2) and "Dyno Therapeutics" in line["text"] + assert line["url"].startswith(f"http://localhost:3000/resources/{line['recording_id']}") + + g = tool(client, h, "explore_graph", entity_id=dyno["entity_id"]) + assert g["focus"] == f"e{dyno['entity_id']}" and {n["label"] for n in g["nodes"] if n["kind"] == "speaker"} == {"Alice", "Bob", "Carol"} + assert tool(client, h, "explore_graph", node=f"e{dyno['entity_id']}", depth=2)["nodes"][0]["depth"] == 0 + assert tool_error(client, h, "explore_graph") == "give node, an entity_id or a speaker_id" + assert tool_error(client, h, "explore_graph", node="x1").startswith("node is a node id") + # the calls graph is isolated: explored on its own, by those who can read it + assert tool_error(client, h, "explore_graph", entity_id=in_calls, namespace="calls") == "not found, or not something you can read" + assert tool(client, hb, "explore_graph", entity_id=in_calls, namespace="calls")["focus"] == f"e{in_calls}" + assert tool_error(client, hb, "explore_graph", entity_id=in_calls) == "not found, or not something you can read" + + carol = next(n["id"] for n in g["nodes"] if n["label"] == "Carol") + path = tool(client, h, "find_path", from_node=f"e{dyno['entity_id']}", to_node=carol) + assert path["found"] is True and [n["id"] for n in path["nodes"]] == [f"e{dyno['entity_id']}", carol] + ev = path["links"][0]["evidence"][0] + assert ev["recording_id"] == ep2 and ev["url"].startswith(f"http://localhost:3000/resources/{ep2}") + + # a tool that breaks is a JSON-RPC error that says nothing about why (the log has it) + monkeypatch.setattr(mcp_tools.ents, "connections", lambda *a: 1 / 0) + err = rpc(client, h, "tools/call", {"name": "get_entity", "arguments": {"entity_id": dyno["entity_id"]}})["error"] + assert err == {"code": -32603, "message": "the tool failed; the server log has the details"} + + +def test_links_through_the_web_app(app, env): + via = {"x-forwarded-host": "lens.example.org", "x-forwarded-proto": "https", **env["h"]} + web = TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 50000)) + hit = tool(web, via, "search", query="exploit benchmark", limit=1)["results"][0] + assert hit["url"].startswith(f"https://lens.example.org/resources/{hit['recording_id']}") + + +def test_collection_roles(client, db, env): + """Someone with a role on one collection, and none in its namespace, reads just that collection's recordings.""" + ep1, ep2, _ = env["ids"] + make_user(db, "own@x.io", "owner password 1", roles={"pods": "owner"}) + make_user(db, "guest@x.io", "guest password 1") + ho, hg = login(client, "own@x.io", "owner password 1"), login(client, "guest@x.io", "guest password 1") + url = "/api/v1/namespaces/pods/collections" + cid = client.post(url, headers=ho, json={"name": "Talks"}).json()["id"] + assert client.post("/api/v1/recordings/collection", headers=ho, json={"recordings": [ep1], "collection": cid}).status_code == 200 + assert client.put(f"{url}/{cid}/members", headers=ho, json={"email": "guest@x.io", "role": "viewer"}).status_code == 200 + assert [n["partial"] for n in tool(client, hg, "list_namespaces")["namespaces"]] == [True] + assert [r["recording_id"] for r in tool(client, hg, "list_recordings")["recordings"]] == [ep1] + assert {r["recording_id"] for r in tool(client, hg, "search", query="capsid OR exploit", limit=50)["results"]} == {ep1} + assert tool(client, hg, "get_transcript", recording_id=ep1, limit=1)["lines"] + rec = tool(client, hg, "get_recording", recording_id=ep1) + assert rec["collection"] == "Talks" and rec["entities"][0]["mentions"] == 3 # of 4: ep2's isn't theirs + assert tool_error(client, hg, "get_transcript", recording_id=ep2) == "not found, or not something you can read" + assert tool(client, hg, "list_speakers")["speakers"] == [] # speakers are namespace-wide + + +def test_a_real_mcp_client(app, env): + """The official MCP SDK's client, over Streamable HTTP, in both eras: what Claude, Cursor and the others speak.""" + import anyio + import httpx2 + from mcp.client.client import Client + from mcp.client.streamable_http import streamable_http_client + + async def session(mode: str) -> tuple[str, list[str], dict[str, Any], str]: + http = httpx2.AsyncClient(transport=httpx2.ASGITransport(app=app), base_url="http://127.0.0.1", headers=env["h"]) + async with http, Client(streamable_http_client("http://127.0.0.1/mcp", http_client=http), mode=mode) as c: + tools = await c.list_tools() + res = await c.call_tool("search", {"query": "capsid", "limit": 1}) + bad = await c.call_tool("get_recording", {"recording_id": env["ids"][2]}) + assert bad.is_error + return c.session.protocol_version, [t.name for t in tools.tools], res.structured_content, res.content[0].text + + for mode, version in (("legacy", "2025-11-25"), ("auto", MODERN)): + got, names, found, text = anyio.run(session, mode) + assert got == version + assert names == list(mcp_tools.TOOLS) + assert found["results"][0]["recording_id"] == env["ids"][0] and json.loads(text) == found diff --git a/fastapi_backend/tests/api/test_oauth.py b/fastapi_backend/tests/api/test_oauth.py index a4f3d1579..75e6f57be 100644 --- a/fastapi_backend/tests/api/test_oauth.py +++ b/fastapi_backend/tests/api/test_oauth.py @@ -85,11 +85,12 @@ def test_discovery_says_where_everything_is(client, app): "resource": "https://lens.example.org/mcp", "authorization_servers": ["https://lens.example.org"], "bearer_methods_supported": ["header"], - "scopes_supported": ["read", "write"], + "scopes_supported": ["read"], # the MCP server only reads "resource_name": "Lens", } assert web.get("/.well-known/oauth-protected-resource/a b/mcp and sign in with OAuth +// on this origin; see lib/api/backend-proxy.ts. +export const dynamic = "force-dynamic"; +export const { GET, POST, DELETE, OPTIONS } = handlers; diff --git a/nextjs-frontend/proxy.ts b/nextjs-frontend/proxy.ts index ee79deecb..45c9b1fc0 100644 --- a/nextjs-frontend/proxy.ts +++ b/nextjs-frontend/proxy.ts @@ -32,10 +32,10 @@ export default auth((req) => { export const config = { // Skip Auth.js, the backend paths (proxied by route handlers, see lib/api/backend-proxy.ts; /.well-known is OAuth - // discovery), Next's files and assets. + // discovery, /mcp the MCP server), Next's files and assets. // Under /iiif only the backend's IIIF resources are skipped (collection, discovery, auth, /iiif//…); the app's own // IIIF pages (/iiif, /iiif/collections/…, /iiif/import, /iiif/metadata/…) need the session like any other page. matcher: [ - "/((?!api/|\\.well-known/|_next/|embed/|s/|iiif/(?:collection(?!s)|discovery|auth|\\d)|reports/|static/|fonts/|favicon\\.ico|icon\\.svg|robots\\.txt).*)", + "/((?!api/|\\.well-known/|mcp(?:/|$)|_next/|embed/|s/|iiif/(?:collection(?!s)|discovery|auth|\\d)|reports/|static/|fonts/|favicon\\.ico|icon\\.svg|robots\\.txt).*)", ], };