From 472d19af903d16dc7c4ab3cb5cc52f41defd756c Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 17 Oct 2026 02:33:11 +0000 Subject: [PATCH 1/2] docs(readme): back each built requirement with the checks that prove it The README's built list now links docs/requirements.md, where each requirement names the tests, clicks journeys and measured numbers (journey <= N clicks in the clicks baseline) that fail when it stops being true. Requirements nothing checks yet say so and link #486. In progress is refreshed from the open pull requests, and a Planned list links where each next step is tracked. The assistant's merged web search, code sandbox, versions and MCP server form join the built list. ci/check_requirements.py (make requirements, and a pre-commit hook that always runs) fails when a built item links no checks, a named test, journey, workflow step or file is gone, a measured number is no longer met, an In progress item is already released, or a Planned item links nowhere. ci/test_check_requirements.py tests it. AGENTS.md gains the rule that requirements move with the code. Closes #483 Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01MTK8aqzLBhdhfCmVf2LaWZ --- .pre-commit-config.yaml | 14 +- AGENTS.md | 3 + Makefile | 5 + README.md | 102 ++++++++++----- ci/check_requirements.py | 231 +++++++++++++++++++++++++++++++++ ci/test_check_requirements.py | 184 ++++++++++++++++++++++++++ docs/contributing.md | 8 ++ docs/requirements.md | 238 ++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 9 files changed, 754 insertions(+), 32 deletions(-) create mode 100755 ci/check_requirements.py create mode 100755 ci/test_check_requirements.py create mode 100644 docs/requirements.md diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 2f047acc0..0e315cb12 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -79,4 +79,16 @@ repos: language: system # Only run frontend client generation if frontend files have changed: files: openapi\.json$ - pass_filenames: false \ No newline at end of file + pass_filenames: false + - id: requirements + name: README requirements name checks that exist (docs/requirements.md) + entry: python3 ci/check_requirements.py + language: system + always_run: true # a test, journey or workflow step can go without the README changing + pass_filenames: false + - id: requirements-tests + name: tests of the requirements check + entry: python3 ci/test_check_requirements.py + language: system + files: ^ci/(check|test_check)_requirements\.py$ + pass_filenames: false diff --git a/AGENTS.md b/AGENTS.md index 20c6c0a2a..bb5ef55dd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -81,6 +81,9 @@ requirements and this file as you learn. Loosening a requirement, a test or a ru - **Generated files come from tooling** (`make openapi`), never from hand-merging. - **Docs move with the code.** Update `docs/` when behaviour or configuration changes; `mkdocs build --strict` stays clean. +- **Requirements move with the code.** A pull request that builds something adds it to the README's built list with + its checks in [docs/requirements.md](docs/requirements.md), and takes its own line out of In progress. `make + requirements` (also a pre-commit hook) fails when a named check is gone. - **Debug from evidence.** Read the logs (`docker compose logs`), reproduce, then fix. - **Quality over cost.** Don't cut work short to save tokens. diff --git a/Makefile b/Makefile index 95afbe062..7554352ba 100644 --- a/Makefile +++ b/Makefile @@ -57,6 +57,11 @@ docs-build: node_modules/.bin/nx ## Build the documentation site, failing on bro affected: node_modules/.bin/nx ## Lint, type check, test and build only the projects your changes affect, against main $(NX) affected -t lint typecheck test build +.PHONY: requirements +requirements: ## Check that every built requirement in the README names checks that exist (docs/requirements.md) + python3 ci/check_requirements.py + python3 ci/test_check_requirements.py + # Run it (Docker) PROD_COMPOSE=$(DOCKER_COMPOSE) -f docker-compose.prod.yml .PHONY: clicks diff --git a/README.md b/README.md index 3fc476a80..48f738527 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ in PowerShell; or skip the terminal and double-click the launcher for your syste the menu bar (built, early: not signed yet). [Deployment](docs/deployment.md) covers servers and Cloudron; there are also packages for [Synology](packaging/synology/README.md), [QNAP](packaging/qnap/README.md) and [Proxmox](proxmox/README.md). +The [checks](docs/requirements.md#install) say which of these CI proves. To work on the code instead, `make run` builds and runs everything from a checkout, `make setup-code` prints the first-admin setup code, and `make dev` runs it with hot reload; see [Get started](docs/get-started.md). @@ -56,7 +57,9 @@ what their person may read. ## What you can do with it -Everything below is built and merged. Work that is still in a pull request is listed under [In progress](#in-progress). +Everything below is built and merged, and each item links to the checks that prove it: tests, clicks journeys and +measured numbers ([requirements and their checks](docs/requirements.md)). Work that is still in a pull request is +listed under [In progress](#in-progress), and what comes next under [Planned](#planned). - **Connect your agents.** Lens is an MCP server (`/mcp`): agents search, read assets' text and notes, cite exact moments, find similar content, explore the graph with Cypher or SPARQL or browse it as folders and files, suggest @@ -68,21 +71,27 @@ Everything below is built and merged. Work that is still in a pull request is li scripts read Lens too. The web app runs on the same REST API, so an agent can do what a person can. ([MCP](docs/mcp.md), [one namespace's MCP server](docs/mcp.md#one-namespaces-mcp-server), [chat completions](docs/api.md#chat-completions-for-any-app), [API](docs/api.md), - [the graph as files](docs/file-tree.md), [authentication](docs/authentication.md)) + [the graph as files](docs/file-tree.md), [authentication](docs/authentication.md); + [checks](docs/requirements.md#connect-your-agents)) - **Ask, and get cited answers.** A chat assistant on every page, and by voice, searches your archive by words (and by - meaning, once turned on), reads it, cites it and acts on it with tools: changes wait for your approval. It reads - answers aloud with a speech provider, macOS `say` or StyleTTS2 on this machine (opt-in), and `/clear` and - `/compact` keep long conversations short. Each namespace can have its own assistant with its own name, instructions - and memory, answering in Telegram and Slack (built in, with voice notes and files), in Discord, Matrix, WhatsApp - and other chat rooms through Matterbridge, and by email at an address of its own that you email or forward things - to. ([Chat](docs/assistant.md), [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), - [speech providers](docs/speech-providers.md)) + meaning, once turned on), reads it, cites it and acts on it with tools: changes wait for your approval. Once an admin + turns them on, it also searches the web and reads pages, cited like the archive, and runs Python and bash in a sandbox + of each conversation's own. Regenerating an answer or editing a question makes a new version beside the old one, and + nothing is lost. It reads answers aloud with a speech provider, macOS `say` or StyleTTS2 on this machine (opt-in), and + `/clear` and `/compact` keep long conversations short. Each namespace can have its own assistant with its own name, + instructions and memory, answering in Telegram and Slack (built in, with voice notes and files), in Discord, Matrix, + WhatsApp and other chat rooms through Matterbridge, and by email at an address of its own that you email or forward + things to. ([Chat](docs/assistant.md), [web search](docs/assistant.md#web-search-and-reading-pages), + [code sandbox](docs/assistant.md#code-sandbox), [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), + [speech providers](docs/speech-providers.md); [checks](docs/requirements.md#ask-and-get-cited-answers)) - **Extend the assistant.** Add tools (a prompt, a web request, a graph drawn on the canvas, or sandboxed Python), skills, hooks and MCP servers, by code, on the canvas or just by asking. Remote MCP servers connect by address, and - ones that run as a program (`npx`, `uvx`) run in a sandboxed container of their own, off until you turn it on. + ones that run as a program (`npx`, `uvx`) run in a sandboxed container of their own, off until you turn it on. Add + one with a form or by pasting another app's config, its tokens sealed and each of its tools switched on or off. Install skills and plugins from marketplaces in the open plugin format, or from a repository or an archive. ([Extending the assistant](docs/assistant.md#extending-the-assistant-tools-skills-hooks-and-plugins), - [the MCP runner](docs/assistant.md#the-mcp-runner)) + [adding an MCP server](docs/assistant.md#adding-an-mcp-server), [the MCP runner](docs/assistant.md#the-mcp-runner); + [checks](docs/requirements.md#extend-the-assistant)) - **Bring in everything.** Watched folders on S3, Dropbox, Google Drive, OneDrive, SFTP, SMB, WebDAV or local disks (a local folder in four clicks); IMAP mail, Gmail and Google Workspace, iCal calendars and Google Calendar; a built-in MQTT broker, syslog and webhook streams; import webhooks for scanners and phone shortcuts. Google accounts connect @@ -92,30 +101,30 @@ Everything below is built and merged. Work that is still in a pull request is li capture web pages one at a time, from a link list or a bookmarks file, by API or by asking the assistant, from private networks you allow in Settings too; and import videos and podcasts by their address (off by default). ([Sensors](docs/sensors.md), [processing](docs/processing.md), [Apache Tika](docs/configuration.md#apache-tika), - [import webhooks](docs/api.md#import-webhooks)) + [import webhooks](docs/api.md#import-webhooks); [checks](docs/requirements.md#bring-in-everything)) - **Clip from the browser.** The Lens web clipper for Chrome, Edge, Firefox and Safari turns a page, a selection or a part you pick into Markdown, lets you look it over, and sends it to a namespace. New releases reach installed copies through the browser's store or a self-hosted update file. In Chrome and Edge, its side panel is a chat with the assistant, which can also use that browser on the sites you allow (open, read, click, type), asking before anything that sends, deletes, buys or posts. ([Web clipper](docs/web-clipper.md), - [the assistant in your browser](docs/browser-assistant.md)) + [the assistant in your browser](docs/browser-assistant.md); [checks](docs/requirements.md#clip-from-the-browser)) - **Understand it.** Transcription, people recognised by voice (long recordings go to a speech provider in pieces that fit its limits), text on screen and in scans, faces (off by default), NSFW pictures flagged by a model on your own - server (off by default, per namespace), entities and topics in a knowledge graph you can explore, query with Cypher - or SPARQL, and see as it was at any version. Every entity has its own page at a wiki address - (`/wiki//`), and entities, topics and people are edited and merged from their lists, the graph or - an asset, with Undo instead of a confirm. Find what is most like a passage, an asset, a topic or an entity, in a - Similar tab on every asset or by selecting text anywhere, and, with anytopdf, photos by what they show. + server (off by default, per namespace), entities and topics in a knowledge graph you can explore, query with Cypher or + SPARQL, and see as it was at any version. Every entity has its own page at a wiki address + (`/wiki//`), and entities, topics and people are edited and merged from their lists, the graph or an + asset, with Undo instead of a confirm. Find what is most like a passage, an asset, a topic or an entity, in a Similar + tab on every asset or by selecting text anywhere, and, with anytopdf, photos by what they show. ([Graph](docs/graph.md), [entity pages](docs/notes.md#entity-pages), - [similar content](docs/processing.md#similar-content), - [graph history](docs/graph-history.md), [topics](docs/topics.md), [video](docs/video.md), - [NSFW classification](docs/nsfw.md), [long recordings](docs/speech-providers.md#long-recordings)) + [similar content](docs/processing.md#similar-content), [graph history](docs/graph-history.md), + [topics](docs/topics.md), [video](docs/video.md), [NSFW classification](docs/nsfw.md), + [long recordings](docs/speech-providers.md#long-recordings); [checks](docs/requirements.md#understand-it)) - **Learn from it.** Make a two-host podcast episode about what you pick, every claim cited and checked against its source, read aloud in two voices. Keep notes beside every asset, entity and topic, written by you or the assistant, in a tree where notes are renamed, moved and pinned (pins are your own), with today's note one click away, a search by their words, and Undo and Recently deleted instead of a confirm. Export an asset or a collection as an evidence PDF with everything Lens found about it, or the parts you pick. ([Podcasts](docs/podcasts.md), [notes](docs/notes.md), - [evidence PDFs](docs/api.md#evidence-pdfs)) + [evidence PDFs](docs/api.md#evidence-pdfs); [checks](docs/requirements.md#learn-from-it)) - **Automate it.** Pipelines and workflows on a canvas, routines on a schedule, batch runs over a collection, and notifications to webhooks, Slack, Discord and Matterbridge. Everything that happens in Lens (new assets, emails, calendar events, files, what people say to Lens, sensor readings, notes, graph changes, runs, budget warnings) is kept @@ -128,20 +137,22 @@ Everything below is built and merged. Work that is still in a pull request is li told you in; agents can decline an approval but never give one. Each automation has limits and budgets, and agents make, try and run them over MCP and in the assistant, or draft one from a sentence; the studio to draw them in the web app is planned. ([Processing](docs/processing.md#workflows-and-the-canvas), [notifications](docs/notifications.md), - [automations](docs/automations.md), [approvals](docs/automations.md#approvals), [timeline](docs/timeline.md)) + [automations](docs/automations.md), [approvals](docs/automations.md#approvals), [timeline](docs/timeline.md); + [checks](docs/requirements.md#automate-it)) - **See what it costs.** An activity ledger of every model call, with reports, charts and budgets that cap spending. Setup asks for one overall budget after a cloud model, with a soft limit that warns and a hard limit that stops paid calls. A meter in the top bar shows spend against the budget, and the Costs page breaks it down by day, model and what cost most. Routine choices go to small decision models, and the LLM only when it adds value. ([Activity and costs](docs/activity.md), [budgets](docs/budgets.md#one-budget-for-everything), - [the meter](docs/budgets.md#the-meter-and-the-costs-page)) + [the meter](docs/budgets.md#the-meter-and-the-costs-page); [checks](docs/requirements.md#see-what-it-costs)) - **Keep it up to date and backed up.** Lens checks for a new release once a day and tells admins by email or a notification. One click in Settings › Updates (or one API call) upgrades it through a small agent on the machine, showing each step, and rolls back when the new version doesn't start; nothing installs unless an admin says so. The same agent changes whether other devices reach Lens and which edition runs, from Settings. Settings › Backups takes, - lists, downloads and restores backups of the database and Lens's files (a file that hasn't changed takes no new - room), takes them on any schedule, and can copy each one, encrypted, to a storage connection. - ([Updates and backups](docs/updates.md), [backups](docs/database.md#backups)) + lists, downloads and restores backups of the database and Lens's files (a file that hasn't changed takes no new room), + takes them on any schedule, and can copy each one, encrypted, to a storage connection. + ([Updates and backups](docs/updates.md), [backups](docs/database.md#backups); + [checks](docs/requirements.md#keep-it-up-to-date-and-backed-up)) - **Watch it run.** Structured logs from every process, read in Settings › Logs and optionally sent to a syslog server. OpenTelemetry traces and metrics, and a Prometheus `/metrics` endpoint, both off until you turn them on, and sending nothing anywhere else. On a Docker install, one switch in the web app (or one API call from an agent) starts @@ -149,7 +160,7 @@ Everything below is built and merged. Work that is still in a pull request is li The Analytics page shows admins active people, churn, retention and which features get used, worked out on this machine; the same daily events can also go to a Google Analytics 4 property of your own (off unless you turn it on). ([Logging](docs/logging.md), [telemetry](docs/telemetry.md), [Prometheus](docs/prometheus.md), - [usage analytics](docs/analytics.md)) + [usage analytics](docs/analytics.md); [checks](docs/requirements.md#watch-it-run)) - **Stay private and safe.** Passkeys instead of passwords, encryption at rest with a key per namespace, roles per namespace (viewer, editor, owner, or custom roles made of permissions), email invitations, removing someone for good, and an audit log. Prompt injection meets several layers: what the assistant reads (recordings, @@ -162,10 +173,11 @@ Everything below is built and merged. Work that is still in a pull request is li domains and, if you choose, only to devices holding a client certificate; or reach it from anywhere through a Cloudflare tunnel with nothing opened on your router. ([Encryption](docs/encryption.md), [access](docs/access.md), [prompt injection](docs/assistant.md#third-party-content-and-prompt-injection), [HTTPS](docs/https.md), - [remote access](docs/remote-access.md)) + [remote access](docs/remote-access.md); [checks](docs/requirements.md#stay-private-and-safe)) - **Use open standards.** RDF and Dublin Core for every record, OAuth for apps and OpenID Connect sign-in, MCP, MQTT, IIIF for publishing, OpenTelemetry and Prometheus for monitoring, and an optional Fedora mirror. - ([Linked data](docs/rdf.md), [IIIF](docs/iiif.md), [Fedora](docs/fedora.md)) + ([Linked data](docs/rdf.md), [IIIF](docs/iiif.md), [Fedora](docs/fedora.md); + [checks](docs/requirements.md#use-open-standards)) ## Light by default @@ -176,7 +188,7 @@ Picking a cloud model provider fills in its address and a model, so only its key in the web app, which works on a phone too: settings are in named groups, a search finds any one by name, and the options most people leave as they are stay folded. ([Components](docs/components.md), [speech providers](docs/speech-providers.md), [local models](docs/local-models.md), -[cloud model providers](docs/configuration.md#cloud-model-providers)) +[cloud model providers](docs/configuration.md#cloud-model-providers); [checks](docs/requirements.md#light-by-default)) ## In progress @@ -186,13 +198,41 @@ These are open pull requests, not yet part of Lens: ([#398](https://github.com/adeelahmad/lense/pull/398)). - **The assistant** answering questions about Lens itself, from its docs and this server ([#400](https://github.com/adeelahmad/lense/pull/400)). +- **Chat:** each answer recording what produced it ([#469](https://github.com/adeelahmad/lense/pull/469)), and a + conversation exported as Markdown, PDF or JSON ([#474](https://github.com/adeelahmad/lense/pull/474)). - **Notes:** download a note, or all of a namespace's notes, as Markdown ([#419](https://github.com/adeelahmad/lense/pull/419)). - **Sources:** RSS and Atom news feeds ([#453](https://github.com/adeelahmad/lense/pull/453)). +- **Pictures:** described on this machine with Florence-2 ([#477](https://github.com/adeelahmad/lense/pull/477)), and + low-quality images enhanced before they're read ([#479](https://github.com/adeelahmad/lense/pull/479)). +- **Entities:** names given their real types instead of Term ([#395](https://github.com/adeelahmad/lense/pull/395)). +- **Web clipper:** signing in, or using your API token, not only an import webhook's + ([#406](https://github.com/adeelahmad/lense/pull/406)). - **Automations:** more operators for conditions ([#434](https://github.com/adeelahmad/lense/pull/434)). - **Shares:** S3 and WebDAV reached through remote access ([#402](https://github.com/adeelahmad/lense/pull/402)), and files added over them imported ([#408](https://github.com/adeelahmad/lense/pull/408)). +## Planned + +Next, and not in a pull request yet; each links to where it is tracked: + +- **The automations studio:** drawing automations in the web app, joins, and the decide node, where a decision model + picks the branch ([automations](docs/automations.md)). +- **Timeline:** what each item caused, and Approve and Decline on waiting approvals ([timeline](docs/timeline.md)). +- **Notes:** nesting a note under its project or area without a click, pages in the graph, and wiki addresses for + topics, recordings and the other things that have pages ([notes](docs/notes.md)). +- **Local models:** GPU builds of llama.cpp fetched by Lens, and local embedding models + ([local models](docs/local-models.md)). +- **Speech providers:** their summaries, chapters and entities as optional imports, and per-minute prices in the + activity ledger ([speech providers](docs/speech-providers.md)). +- **Storage:** uploads and assets' media in the file store, and a storage key to download + ([storage](docs/storage.md)). +- **Logging:** sending straight to Loki or another HTTP endpoint, and client certificates for TLS + ([logging](docs/logging.md)). +- **Proof:** a scorecard of measured numbers per release ([#481](https://github.com/adeelahmad/lense/issues/481)), and + each release tested by upgrading an install of the last one, holding data + ([#482](https://github.com/adeelahmad/lense/issues/482)). + ## Stack | | | diff --git a/ci/check_requirements.py b/ci/check_requirements.py new file mode 100755 index 000000000..5be6d9291 --- /dev/null +++ b/ci/check_requirements.py @@ -0,0 +1,231 @@ +#!/usr/bin/env python3 +"""Check that the README's requirements still point at checks that exist. + +The README lists what Lens does (built: "What you can do with it", "Install" and "Light +by default"), what is in a pull request (In progress) and what comes next (Planned). +docs/requirements.md backs each built item with the checks that fail when it stops being +true. This script fails when the two drift apart: + +- a built item links no section of docs/requirements.md, or one that isn't there, and a + section there that no README item links; +- a row that names no check, or a check that is gone: a file, a test function or class + in a Python file, a quoted name (test title, journey id) in a script, a job or step + name in a workflow, a journey in a clicks baseline; +- a measured number (`journey` <= N clicks) that the clicks baseline no longer meets, or + a journey that fails there; +- a row with no check yet that links no issue; +- an In progress item that links no pull request, or one the changelog lists as released; +- a Planned item that links nowhere. + +Usage: python3 ci/check_requirements.py [repo_root]. Exits 1 and lists every problem. +""" + +from __future__ import annotations + +import json +import re +import sys +from pathlib import Path + +REPO = "https://github.com/adeelahmad/lense/" +REQUIREMENTS = "docs/requirements.md" +BUILT = ("What you can do with it", "Install", "Light by default") + +# A link to a file in the repository, as docs pages write them (mkdocs needs full URLs) +FILE_LINK = r"\[[^\]]*\]\(" + re.escape(REPO) + r"(?:blob|tree)/main/([^)#\s]+)[^)]*\)" +# Each piece of a row's checks, in order: a file, a name in it, a measured number +PIECE = re.compile(rf"{FILE_LINK}|`([^`]+)`|(?:≤|<=)\s*(\d+)\s+(click|key|swipe)s?\b") +ISSUE = re.compile(re.escape(REPO) + r"issues/\d+") +PULL = re.compile(re.escape(REPO) + r"pull/(\d+)") +SECTION_LINK = re.compile(re.escape(REQUIREMENTS) + r"#([a-z0-9-]+)") + + +def slug(title: str) -> str: + """The anchor GitHub and mkdocs give a heading.""" + return re.sub(r"[\s]+", "-", re.sub(r"[^\w\s-]", "", title.strip().lower())) + + +def sections(text: str) -> dict[str, str]: + """Each level-two heading's title and the text under it.""" + found: dict[str, str] = {} + title = None + for line in text.splitlines(): + if line.startswith("## "): + title = line[3:].strip() + found[title] = "" + elif title is not None: + found[title] += line + "\n" + return found + + +def bullets(text: str) -> list[str]: + """The top-level list items of a section, each with its continuation lines.""" + items: list[str] = [] + for line in text.splitlines(): + if line.startswith("- "): + items.append(line[2:]) + elif items and line.startswith(" "): + items[-1] += " " + line.strip() + elif line.strip() and items: + break + return items + + +def rows(text: str) -> list[tuple[str, str]]: + """Each table row's first cell (the requirement) and last cell (its checks).""" + found = [] + for line in text.splitlines(): + if not line.startswith("|") or re.match(r"^\|[\s:-]+\|", line): + continue + cells = [cell.strip() for cell in line.strip().strip("|").split(" | ")] + if cells[0] in ("Requirement", "") or len(cells) < 2: + continue + found.append((cells[0], cells[-1])) + return found + + +def has_name(path: Path, name: str) -> bool: + """Whether the file still holds the named test, journey, job or step.""" + text = path.read_text(encoding="utf-8") + word = re.escape(name) + if path.suffix == ".py": + return bool( + re.search(rf"^\s*(?:async\s+)?(?:def|class)\s+{word}\b", text, re.M) + ) + if path.suffix in (".yml", ".yaml"): + step = rf"^\s*(?:-\s+)?name:\s*[\"']?{word}[\"']?\s*$" + return bool( + re.search(step, text, re.M) or re.search(rf"^ {word}:\s*$", text, re.M) + ) + if path.suffix == ".json": + return any(item.get("id") == name for item in journeys(path)) + if path.suffix in (".js", ".mjs", ".ts", ".tsx", ".jsx"): + return any(f"{quote}{name}{quote}" in text for quote in "\"'`") + return name in text + + +def journeys(path: Path) -> list[dict]: + try: + data = json.loads(path.read_text(encoding="utf-8")) + except ValueError: + return [] + return data.get("journeys", []) if isinstance(data, dict) else [] + + +def measured(path: Path, name: str, most: int, unit: str) -> str | None: + """Why a journey in a clicks baseline doesn't meet its number, or None when it does.""" + journey = next((item for item in journeys(path) if item.get("id") == name), None) + if journey is None: + return f"`{name}` is not a journey in {path.name}" + if not journey.get("ok", False): + return f"`{name}` fails in {path.name}" + if journey.get(unit, 0) > most: + return f"`{name}` takes {journey.get(unit)} {unit} in {path.name}, not ≤ {most}" + return None + + +def check_row(root: Path, where: str, checks: str) -> list[str]: + problems = [] + if "no check yet" in checks.lower(): + if not ISSUE.search(checks): + problems.append(f"{where}: 'no check yet' links no issue") + return problems + path: Path | None = None + name: str | None = None + for piece in PIECE.finditer(checks): + link, token, most, unit = piece.groups() + if link: + path, name = root / link, None + if not path.exists(): + problems.append(f"{where}: {link} is gone") + path = None + elif token and path is not None and path.is_file(): + name = token + if not has_name(path, token): + problems.append( + f"{where}: `{token}` is gone from {path.relative_to(root)}" + ) + elif most: + if path is None or path.suffix != ".json" or name is None: + problems.append( + f"{where}: ≤ {most} {unit}s names no journey in a baseline" + ) + elif why := measured(path, name, int(most), unit + "s"): + problems.append(f"{where}: {why}") + if not re.search(FILE_LINK, checks): + problems.append(f"{where}: names no check (a file in the repository)") + return list(dict.fromkeys(problems)) # a failing journey, once + + +def check(root: Path) -> list[str]: + problems: list[str] = [] + readme = sections((root / "README.md").read_text(encoding="utf-8")) + reqs_path = root / REQUIREMENTS + if not reqs_path.exists(): + return [f"{REQUIREMENTS} is missing"] + reqs = sections(reqs_path.read_text(encoding="utf-8")) + backed = {slug(title): (title, text) for title, text in reqs.items() if rows(text)} + + linked: set[str] = set() + for title in BUILT: + if title not in readme: + problems.append(f"README: no '## {title}' section") + continue + items = bullets(readme[title]) if title == BUILT[0] else [readme[title]] + for item in items: + label = re.match(r"\*\*([^*]+)\*\*", item) + where = f"README › {title}" + (f" › {label.group(1)}" if label else "") + anchors = SECTION_LINK.findall(item) + if not anchors: + problems.append(f"{where}: links no section of {REQUIREMENTS}") + for anchor in anchors: + linked.add(anchor) + if anchor not in backed: + problems.append(f"{where}: {REQUIREMENTS}#{anchor} has no checks") + + for anchor, (title, text) in backed.items(): + if anchor not in linked: + problems.append( + f"{REQUIREMENTS} › {title}: no built item in the README links it" + ) + for requirement, checks in rows(text): + where = f"{REQUIREMENTS} › {title} › {requirement[:60]}" + problems += check_row(root, where, checks) + + changelog = (root / "CHANGELOG.md").read_text(encoding="utf-8") + for item in bullets(readme.get("In progress", "")): + pulls = PULL.findall(item) + if not pulls: + problems.append(f"README › In progress: links no pull request: {item[:60]}") + for number in pulls: + if f"/pull/{number})" in changelog: + problems.append( + f"README › In progress: #{number} is released (CHANGELOG.md); " + "list it as built, with its checks" + ) + if "Planned" not in readme: + problems.append("README: no '## Planned' section") + for item in bullets(readme.get("Planned", "")): + if "](" not in item: + problems.append(f"README › Planned: links nowhere: {item[:60]}") + return problems + + +def main() -> int: + root = Path( + sys.argv[1] if len(sys.argv) > 1 else Path(__file__).resolve().parent.parent + ) + problems = check(root) + for problem in problems: + print(problem) + if problems: + print( + f"\n{len(problems)} problem(s): see {REQUIREMENTS}, 'How the checks are kept'" + ) + return 1 + print("Every built requirement names checks that exist.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/ci/test_check_requirements.py b/ci/test_check_requirements.py new file mode 100755 index 000000000..fbfd080a8 --- /dev/null +++ b/ci/test_check_requirements.py @@ -0,0 +1,184 @@ +#!/usr/bin/env python3 +"""Tests for check_requirements.py: python3 ci/test_check_requirements.py""" + +from __future__ import annotations + +import json +import tempfile +import unittest +from pathlib import Path + +from check_requirements import check + +BLOB = "https://github.com/adeelahmad/lense/blob/main/" +PULL = "https://github.com/adeelahmad/lense/pull/" +ISSUE = "https://github.com/adeelahmad/lense/issues/9" + +README = """# Lens + +## Install + +One line. ([checks](docs/requirements.md#install)) + +## What you can do with it + +- **Search.** Find words. ([search](docs/search.md), + [checks](docs/requirements.md#search)) + +## Light by default + +Small. ([checks](docs/requirements.md#light-by-default)) + +## In progress + +- **Feeds** ([#12](PULL12)). + +## Planned + +- **Studio** ([automations](docs/automations.md)). +""" + +REQUIREMENTS = """# Requirements + +## Install + +| Requirement | Checks | +|---|---| +| One line | [install.yml](BLOBci/install.yml) `Install` | + +## Search + +| Requirement | Checks | +|---|---| +| Finds words | [test_search.py](BLOBtests/test_search.py) `test_finds`, `Words` | +| In clicks | [journeys](BLOBci/journeys.mjs) `search-word`; [baseline](BLOBci/baseline.json) `search-word` ≤ 2 clicks, ≤ 9 keys | +| By meaning | No check yet ([#9](ISSUE)) | + +## Light by default + +| Requirement | Checks | +|---|---| +| Small | [components](BLOBdocs/components.md) | +""" + + +class CheckRequirements(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.root = Path(self.tmp.name) + self.write("README.md", README.replace("PULL", PULL)) + self.write( + "docs/requirements.md", + REQUIREMENTS.replace("BLOB", BLOB).replace("ISSUE", ISSUE), + ) + self.write("docs/components.md", "# Components\n") + self.write( + "ci/install.yml", "jobs:\n install:\n steps:\n - name: Install\n" + ) + self.write( + "tests/test_search.py", + "def test_finds():\n pass\n\n\nclass Words:\n pass\n", + ) + self.write("ci/journeys.mjs", 'export default [{ id: "search-word" }];\n') + self.baseline(clicks=2) + self.write("CHANGELOG.md", f"# Changelog\n\n- feeds ([#11]({PULL}11))\n") + + def tearDown(self) -> None: + self.tmp.cleanup() + + def write(self, name: str, text: str) -> None: + path = self.root / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + def edit(self, name: str, old: str, new: str) -> None: + path = self.root / name + text = path.read_text(encoding="utf-8") + self.assertIn(old, text) + path.write_text(text.replace(old, new), encoding="utf-8") + + def baseline(self, clicks: int, ok: bool = True, keys: int = 9) -> None: + journey = {"id": "search-word", "clicks": clicks, "keys": keys, "ok": ok} + self.write("ci/baseline.json", json.dumps({"journeys": [journey]})) + + def problems(self) -> list[str]: + return check(self.root) + + def test_clean(self) -> None: + self.assertEqual(self.problems(), []) + + def test_a_test_that_is_gone(self) -> None: + self.edit("tests/test_search.py", "def test_finds", "def test_found") + [problem] = self.problems() + self.assertIn("`test_finds` is gone from tests/test_search.py", problem) + + def test_a_file_that_is_gone(self) -> None: + (self.root / "tests/test_search.py").unlink() + [problem] = self.problems() + self.assertIn("tests/test_search.py is gone", problem) + + def test_a_journey_and_a_workflow_step_that_are_gone(self) -> None: + self.edit("ci/journeys.mjs", '"search-word"', '"find-word"') + self.edit("ci/install.yml", "name: Install", "name: Set up") + problems = "\n".join(self.problems()) + self.assertIn("`search-word` is gone from ci/journeys.mjs", problems) + self.assertIn("`Install` is gone from ci/install.yml", problems) + + def test_a_measured_number_not_met(self) -> None: + self.baseline(clicks=3) + [problem] = self.problems() + self.assertIn("takes 3 clicks in baseline.json, not ≤ 2", problem) + self.baseline(clicks=2, keys=10) + [problem] = self.problems() + self.assertIn("takes 10 keys in baseline.json, not ≤ 9", problem) + self.baseline(clicks=1, ok=False) + [problem] = self.problems() + self.assertIn("`search-word` fails in baseline.json", problem) + + def test_no_check_yet_needs_an_issue(self) -> None: + self.edit("docs/requirements.md", f"([#9]({ISSUE}))", "") + [problem] = self.problems() + self.assertIn("'no check yet' links no issue", problem) + + def test_a_row_without_a_check(self) -> None: + self.edit( + "docs/requirements.md", + f"[components]({BLOB}docs/components.md)", + "the docs", + ) + [problem] = self.problems() + self.assertIn("names no check", problem) + + def test_a_built_item_without_checks(self) -> None: + self.edit("README.md", ",\n [checks](docs/requirements.md#search)", "") + problems = "\n".join(self.problems()) + self.assertIn( + "README › What you can do with it › Search.: links no section", problems + ) + self.assertIn( + "requirements.md › Search: no built item in the README links it", problems + ) + + def test_a_link_to_a_missing_section(self) -> None: + self.edit("README.md", "requirements.md#search", "requirements.md#find") + problems = "\n".join(self.problems()) + self.assertIn("docs/requirements.md#find has no checks", problems) + + def test_in_progress_already_released(self) -> None: + self.edit("README.md", f"[#12]({PULL}12)", f"[#11]({PULL}11)") + [problem] = self.problems() + self.assertIn("#11 is released", problem) + + def test_in_progress_without_a_pull_request(self) -> None: + self.edit("README.md", f" ([#12]({PULL}12))", "") + [problem] = self.problems() + self.assertIn("In progress: links no pull request", problem) + + def test_planned_links_somewhere(self) -> None: + self.edit("README.md", " ([automations](docs/automations.md))", "") + [problem] = self.problems() + self.assertIn("Planned: links nowhere", problem) + + +if __name__ == "__main__": + unittest.main() diff --git a/docs/contributing.md b/docs/contributing.md index 5daacb53b..cf5f6d6c2 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -117,6 +117,14 @@ pnpm test && pnpm lint && pnpm tsc && pnpm build The docs are these Markdown files, built with mkdocs-material: `make docs` serves them, and `make docs-build` builds the site the way CI does, failing on broken links. +## Requirements + +The README lists what Lens does (built), what is in a pull request (In progress) and what comes next (Planned), and +[Requirements and checks](requirements.md) backs each built item with the tests, clicks journeys and measured numbers +that fail when it stops being true. A pull request that builds something adds it to the README and a row there, and +takes its own line out of In progress. `make requirements` (also a pre-commit hook, so CI runs it on every pull +request) fails when a check a row names is gone or a measured number is no longer met. + ## Release The backend and frontend share a [semantic version](https://semver.org/). The changelog is written from the PR titles, diff --git a/docs/requirements.md b/docs/requirements.md new file mode 100644 index 000000000..8a0eeacdb --- /dev/null +++ b/docs/requirements.md @@ -0,0 +1,238 @@ +# Requirements and their checks + +Lens's requirements are the lists in the [README](https://github.com/adeelahmad/lense#what-you-can-do-with-it): what +is built, what is [in progress](https://github.com/adeelahmad/lense#in-progress) (an open pull request) and what is +[planned](https://github.com/adeelahmad/lense#planned). Each built item there links its section here, and each row of +a section names the checks that fail when that requirement stops being true: a test, a +[clicks journey](https://github.com/adeelahmad/lense/blob/main/ci/clicks/README.md) and the number it is held to, or +the CI workflow that builds or tests it. A requirement nothing checks yet says so and links the issue that adds one. + +## How the checks are kept + +`make requirements` runs +[`ci/check_requirements.py`](https://github.com/adeelahmad/lense/blob/main/ci/check_requirements.py), and pre-commit +runs it on every pull request, so drift shows up in CI. It fails when: + +- a built item in the README links no section here, or a section here is linked by no item; +- a check is gone: its file, a test function or class in a Python test, a quoted name (a test's title, a journey's + id) in a script, a job or step `name:` in a workflow, or a journey in a clicks baseline; +- a measured number is no longer met: `journey` ≤ N clicks (or keys) in + [`ci/clicks/baseline.json`](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) (a computer) or + [`baseline-phone.json`](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline-phone.json) (a phone), or + the journey fails there; +- a requirement with no check yet links no issue; +- an In progress item links no pull request, or one the changelog already lists as released; +- a Planned item links nowhere. + +The script checks that a check exists, not that it passes: CI runs the tests, the Clicks workflow the journeys. A +check named here runs in CI rather than skipping there (a test that needs a program the runners lack is not a check +yet). When a pull request builds something, it adds a line to the README's built list and a row here, and takes its +own line out of In progress. Renaming or removing a test that a row names means changing the row in the same pull +request, to another check that proves the same thing; loosening a requirement or its number waits for the owner +([AGENTS.md](https://github.com/adeelahmad/lense/blob/main/AGENTS.md)). + +## Install + +| Requirement | Checks | +|---|---| +| One line (`install.sh`) opens a web installer that checks the computer | [install.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/install.yml) `Install from the web installer`, `Start the installer, as install.sh does on a first install`; [test_lens_installer.py](https://github.com/adeelahmad/lense/blob/main/installer/test_lens_installer.py) `test_plan_lists_docker_when_it_is_missing`, `test_only_the_link_opens_it` | +| It installs with the defaults, using a model server it finds on the computer | [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `install`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `install` ≤ 3 clicks, ≤ 2 keys | +| It installs what's missing, Docker included, starts Lens and opens its setup page | [install.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/install.yml) `A new computer, without Docker`, `Answer it and install`, `Lens answers, waits for its first admin, and has the installer's settings`; [test_lens_installer.py](https://github.com/adeelahmad/lense/blob/main/installer/test_lens_installer.py) `test_install_runs_install_sh_with_the_answers` | +| Lens can run in a microsandbox microVM instead of Docker | [install.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/install.yml) `One-line install in a microsandbox`; [test_lens_installer.py](https://github.com/adeelahmad/lense/blob/main/installer/test_lens_installer.py) `test_lens_app_offers_microsandbox_first_where_it_runs` | +| Running the line again updates Lens in place and keeps its secrets | [install.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/install.yml) `One-line install`, `Running it again updates in place and keeps the secrets` | +| Windows installs with `install.ps1` | [install.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/install.yml) `install.ps1 parses in Windows PowerShell 5.1, what Windows has`; [test_install_windows.py](https://github.com/adeelahmad/lense/blob/main/installer/test_install_windows.py) `test_installs_and_hands_over_the_setup_address`, `test_a_failed_build_says_so` | +| Double-click launchers for each system | [launchers.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/launchers.yml) `Build the launchers`, `Check them` | +| Lens.app on a Mac with Apple silicon installs Lens and runs it without Docker | [desktop.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/desktop.yml) `Build Lens.app`, `The bundled msb runs`, `The app carries its helper and microsandbox` | +| Packages for Synology, QNAP and Cloudron | [synology.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/synology.yml) `Build`; [qnap.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/qnap.yml) `Build the package`; [cloudron.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/cloudron.yml) `build` | +| A Proxmox container script | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | +| The first admin signs up with a passkey from the setup link, which fills in the setup code | [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `first-admin`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `first-admin` ≤ 1 click; [test_passkeys.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_passkeys.py) `test_first_admin_with_a_passkey_and_no_password`; [setupForm.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/setupForm.test.tsx) `fills in the code from the setup link and starts at the email` | + +## Connect your agents + +| Requirement | Checks | +|---|---| +| An MCP server at `/mcp`: agents search, read assets' text and notes, and cite exact moments | [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_search_and_cite`, `test_reading_recordings`, `test_note_tools` | +| Agents find similar content | [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_search_and_cite`; [test_similar.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_similar.py) `test_like_a_text_by_its_key_words`, `test_like_a_resource_by_words_topics_and_entities` | +| Agents explore the graph with Cypher or SPARQL, or browse it as folders and files | [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_graph_tools`, `test_sparql`; [test_file_tree.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_file_tree.py) `test_agents_browse_it_over_mcp` | +| Agents suggest topics and graph changes, and edit and merge entities and topics | [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_topic_tools`, `test_editing_topics_and_entities` | +| Agents write notes and make podcasts | [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_note_tools`; [test_podcasts.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_podcasts.py) `test_agents_make_episodes` | +| Agents sign in with OAuth and see only what their person may read | [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_clients_are_sent_to_sign_in`, `test_collection_roles`; [test_oauth.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_oauth.py) `test_an_app_gets_access_and_acts_with_the_persons_roles` | +| The account menu copies the MCP address in one click; connecting an assistant takes a handful | [connect-assistant.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/connect-assistant.test.tsx) `copies the address from the account menu in one click`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `agent-connect`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `agent-connect` ≤ 5 clicks | +| A namespace's own MCP server at `/mcp/`, off until its owner turns it on | [test_mcp_namespace.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp_namespace.py) `test_off_until_an_owner_turns_it_on`, `test_it_sees_one_namespace` | +| OpenAI's chat completions API per namespace, cited from that namespace alone, off until turned on | [test_completions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_completions.py) `test_off_until_an_owner_turns_it_on`, `test_answers_from_the_namespace_alone_with_citations`, `test_streams_as_openai_does` | +| API keys that open one namespace and nothing else | [test_namespace_keys.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_namespace_keys.py) `test_a_key_for_one_namespace_opens_its_servers_alone`, `test_it_has_the_persons_role_there_and_no_more` | +| The graph's folders and files mounted read-only over S3, WebDAV, NFS and SMB | [test_s3.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_s3.py) `test_people_read_their_namespaces_over_s3`, `test_what_s3_refuses`; [test_webdav.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_webdav.py) `test_mount_and_read`, `test_what_webdav_refuses`; [test_nfs_share.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_nfs_share.py) `test_the_runner_follows_the_setting`; [test_smb_share.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_smb_share.py) `test_reads_the_tree_as_its_key_sees_it`, `test_nothing_can_be_written` | +| The web app runs on the same REST API, through a client generated from its OpenAPI schema | [ci.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/ci.yml) `OpenAPI schema is up to date` | + +## Ask, and get cited answers + +| Requirement | Checks | +|---|---| +| A chat assistant on every page | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_asking_from_a_page`; [page-chat.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/page-chat.test.tsx); [ask-place.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/ask-place.test.tsx) | +| By voice | [assistant-mode.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/assistant-mode.test.tsx) `starts a voice chat from the mic`; [test_voice.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_voice.py) `test_what_is_said_comes_back_as_text` | +| It searches by words, and by meaning once search by meaning is set up | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_tools_approvals_and_checking`; [test_semantic.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_semantic.py) `test_chat_retrieval_by_meaning`, `test_off_or_unconfigured` | +| Answers cite their sources, and only what the person may read | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_chat_cites_only_what_you_can_read`; [chat.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat.test.tsx) `splits [n], [n, m] and [n][m]` | +| It acts with tools, and changes wait for approval | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_tools_approvals_and_checking`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_the_assistant_lists_and_declines_but_never_approves` | +| Web search and reading pages, cited like the archive, off until an admin turns it on | [test_web_search.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_web_search.py) `test_an_answer_cites_the_web_and_the_conversation_keeps_it`, `test_reading_pages_and_pdfs`; [chat-web-citations.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-web-citations.test.tsx) | +| Python and bash in a sandbox of each conversation's own, off until an admin turns it on | [test_sandbox.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_sandbox.py) `test_python_and_bash_share_one_sandbox_per_conversation`, `test_the_sandbox_is_off_until_an_admin_turns_it_on_for_the_people_it_names` | +| Regenerating an answer or editing a question makes a new version beside the old one | [test_chat_versions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chat_versions.py) `test_regenerate_edit_switch_and_go_on`; [chat-versions.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-versions.test.tsx) `regenerating an answer`, `the version switch` | +| Answers read aloud by a speech provider, macOS `say` or StyleTTS2 (opt-in) | [test_voice.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_voice.py) `test_answers_are_read_aloud_by_a_speech_model`; [test_speech.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_speech.py) `test_macos_say_reads_aloud_on_a_mac`, `test_styletts2_reads_aloud_from_its_own_service`; [styletts2.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/styletts2.yml) `Read a sentence aloud` | +| `/clear` and `/compact` | [test_compaction.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_compaction.py) `test_clear_starts_afresh_and_compact_folds_everything_now`, `test_commands_in_the_app` | +| A namespace's own assistant, with its own name, instructions and memory | [test_ns_assistant.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_ns_assistant.py) `test_off_by_default_and_owners_turn_it_on`, `test_remembers_across_conversations_with_citations`; [ns-assistant.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/ns-assistant.test.tsx) `saving a name and forgetting a memory` | +| Telegram and Slack built in, with voice notes and files | [test_telegram.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_telegram.py) `test_it_answers_private_chats_and_what_is_said_to_it_in_groups`, `test_voice_notes_are_heard`, `test_files_are_held_for_the_assistant_to_import`; [test_slack.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_slack.py) `test_it_answers_direct_messages_mentions_and_the_slash_command`, `test_voice_clips_are_heard_and_files_held` | +| Other chat rooms through Matterbridge | [test_bridge.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_bridge.py) `test_it_answers_what_is_said_to_it_and_nothing_else`; [test_matterbridge_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_matterbridge_run.py) `test_chat_networks_are_set_in_the_app_and_become_matterbridges_config`, `test_the_bridge_talks_to_the_matterbridge_lens_runs`; [matterbridge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/matterbridge.yml) `Lens writes a config, and Matterbridge starts with it` | +| By email, at an address of its own | [test_mailbox.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mailbox.py) `test_mail_is_answered_by_email_in_its_thread`, `test_the_mailbox_is_set_up_and_checked_in_the_app`, `test_answers_list_the_excerpts_they_cite` | + +## Extend the assistant + +| Requirement | Checks | +|---|---| +| Tools: a prompt, a web request, a graph drawn on the canvas, or sandboxed Python | [test_extensions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_extensions.py) `test_tools_and_skills_in_a_conversation`, `test_tools_drawn_on_the_canvas`, `test_python_tools_run_apart_and_only_admins_write_them` | +| Skills, hooks and MCP servers, by code, on the canvas or by asking | [test_extensions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_extensions.py) `test_written_as_code_shared_and_versioned`, `test_change_tools_ask_first_and_hooks`, `test_made_in_chat_behind_an_approval`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `extension-add`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `extension-add` ≤ 3 clicks | +| Remote MCP servers connect by address | [test_mcp_servers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp_servers.py) `test_an_admin_adds_a_server_and_the_assistant_uses_it` | +| MCP servers that run as a program (`npx`, `uvx`) run in a sandboxed container of their own | [test_mcp_servers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp_servers.py) `test_a_server_that_runs_as_a_program_runs_in_the_runner`; [mcp-runner.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/mcp-runner.yml) `Runs an npm server and a PyPI server` | +| An MCP server added by form or a pasted config, its tokens sealed and each tool switched on or off | [test_mcp_servers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp_servers.py) `test_a_pasted_config_reads_as_servers_to_add`, `test_its_secrets_are_sealed_and_never_shown`, `test_each_tool_is_switched_and_says_whether_it_asks`; [mcp-server-ui.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/mcp-server-ui.test.tsx) `adding an MCP server` | +| Skills and plugins from marketplaces in the open plugin format, a repository or an archive | [test_marketplaces.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_marketplaces.py) `test_a_marketplace_an_admin_adds`, `test_imported_from_a_link_or_a_file` | + +## Bring in everything + +| Requirement | Checks | +|---|---| +| Watched folders on storage connections (S3, Dropbox, Google Drive, OneDrive, SFTP, SMB, WebDAV, local disks) | [test_sources.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_sources.py) `test_sources_and_watches_without_rclone`, `test_importing_chosen_files_of_a_source` | +| A local folder in four clicks | [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `source-folder`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `source-folder` ≤ 4 clicks; [watch-folder.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/watch-folder.test.tsx) | +| A watched folder on each kind of storage connection, run against rclone in CI | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | +| IMAP mail, Gmail and Google Workspace | [test_mail_calendar_sources.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mail_calendar_sources.py) `test_an_imap_account_as_a_source`, `test_an_imap_message_is_an_email_document_with_its_attachments`; [test_gmail_source.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_gmail_source.py) `test_a_workspace_mailbox_through_the_gmail_api`, `test_a_connected_google_account` | +| iCal calendars and Google Calendar | [test_mail_calendar_sources.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mail_calendar_sources.py) `test_a_calendar_feed_as_a_source`; [test_google_calendar_source.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_google_calendar_source.py) `test_a_shared_calendar_read_as_a_service_account`, `test_a_connected_google_account` | +| A built-in MQTT broker, syslog and webhook streams | [test_sensors.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_sensors.py) `test_hub_takes_devices_in_and_organises_them`, `test_syslog_senders_become_sensors`, `test_webhooks_and_one_list_of_sensors`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `sensor-webhook`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `sensor-webhook` ≤ 3 clicks | +| Import webhooks for scanners and phone shortcuts | [test_import_hooks.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_import_hooks.py) `test_pushing_files_urls_and_text`, `test_owners_manage_hooks`; [import-hooks.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/import-hooks.test.tsx) | +| Connect with Google; Dropbox, OneDrive and Google Drive sign in through rclone on the server | [test_gmail_source.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_gmail_source.py) `test_connecting_the_account_in_the_browser`; [test_drive_google.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_drive_google.py) `test_drive_connects_with_google_for_rclone`; [test_rclone_signin.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_rclone_signin.py) `test_drive_signs_in_through_rclone_with_no_google_app`, `test_a_browser_on_the_server_reaches_rclone_itself`, `test_onedrive_finds_the_accounts_own_drive` | +| Uploads of audio, PDFs, Office files, photos, JSON and archives | [test_uploads.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_uploads.py) `test_uploading_audio_in_chunks`, `test_json_uploaded_is_read_as_records`; [test_documents.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_documents.py) `test_a_pdf_becomes_a_document_with_its_pages_and_text`, `test_scans_and_images_are_read_by_ocr`; [test_conversion.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_conversion.py) `test_office_and_text_files_become_documents`; [test_anytopdf.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_anytopdf.py) `test_an_uploaded_zip_keeps_its_files`; [test_archives.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_archives.py) `test_reading_an_archive`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `import-file`, `import-paste` | +| An uploaded video, end to end | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | +| Apache Tika (opt-in) reads documents nothing else here can | [test_tika.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_tika.py) `test_off_by_default_nothing_changes`, `test_what_nothing_here_can_convert_tika_reads_as_text` | +| Web pages captured one at a time, from a link list or a bookmarks file, by API or by asking; private networks only when allowed | [test_webcapture.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_webcapture.py) `test_a_web_page_becomes_a_document`, `test_an_admin_allows_a_private_network_in_settings`, `test_private_addresses_are_refused`; [test_web_links.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_web_links.py) `test_one_or_several_addresses_as_a_list`, `test_a_bookmarks_file_becomes_pages_to_capture`; [test_setup_assistant.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_setup_assistant.py) `test_the_assistant_captures_web_pages` | +| Videos and podcasts imported by their address, off by default | [test_video_urls.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_video_urls.py) `test_off_by_default`, `test_a_video_by_its_address_is_downloaded_then_transcribed`; [import-web.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/import-web.test.tsx) `imports a video or podcast by its address, when asked to` | + +## Clip from the browser + +| Requirement | Checks | +|---|---| +| A page, a selection or a part you pick becomes Markdown, looked over and sent to a namespace | [clip.test.js](https://github.com/adeelahmad/lense/blob/main/extensions/browser/test/clip.test.js) `a full page becomes its article, with front matter`, `the selection`, `a picked element, with the selection template`; [clipper-edge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/clipper-edge.yml) `Smoke test in Edge` | +| Built for Chrome, Edge, Firefox and Safari | [browser-extension.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/browser-extension.yml) `Test and build the web clipper`, `Build every target`; [browser-extension-firefox.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/browser-extension-firefox.yml) `Lint and package the Firefox clipper`; [safari-clipper.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/safari-clipper.yml) `Build the Safari clipper`; [clipper-edge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/clipper-edge.yml) `Clip a page in Edge` | +| New releases reach installed copies through a store or a self-hosted update file | [release.test.js](https://github.com/adeelahmad/lense/blob/main/extensions/browser/test/release.test.js) `update manifests` | +| In Chrome and Edge, the side panel is a chat with the assistant | [assistant.test.js](https://github.com/adeelahmad/lense/blob/main/extensions/browser/test/assistant.test.js); [clipper-edge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/clipper-edge.yml) `Smoke test in Edge` | +| The assistant uses that browser on allowed sites, asking before anything that sends, deletes, buys or posts | [act.test.js](https://github.com/adeelahmad/lense/blob/main/extensions/browser/test/act.test.js); [test_browser.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_browser.py) `test_a_connected_side_panel_gives_the_assistant_the_browser`, `test_after_reading_a_page_the_browser_asks_before_what_the_person_didnt_write` | + +## Understand it + +| Requirement | Checks | +|---|---| +| Transcription | [test_speech.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_speech.py) `test_an_openai_compatible_server_transcribes`, `test_deepgram_transcribes_with_speakers_and_sentiment` | +| Transcription by a local engine (SenseVoice, Whisper), run in CI | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | +| People recognised by voice; long recordings sent to a speech provider in pieces that fit its limits | [test_voices.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_voices.py) `test_match_suggest_merge_undo_link`; [test_speech.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_speech.py) `test_a_long_recording_goes_in_pieces_and_comes_back_whole`, `test_a_piece_too_large_to_send_is_halved` | +| Text on screen and in scans | [test_video.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_video.py) `test_shots_text_search_chat_and_iiif`; [test_documents.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_documents.py) `test_scans_and_images_are_read_by_ocr`; [test_ocr.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_ocr.py) `test_a_video_read_by_doctr_and_without_it` | +| Faces, off by default | [test_video.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_video.py) `test_people_on_screen`, `test_shots_text_search_chat_and_iiif`; [test_video.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_video.py) `test_face_modes` | +| NSFW pictures flagged by a model on your own server, off by default, per namespace | [test_nsfw.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_nsfw.py) `test_off_by_default_nothing_is_fetched_or_classified`, `test_a_namespace_classifies_flags_blurs_and_hides`; [nsfw.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/nsfw.test.tsx) | +| Entities and topics in a knowledge graph to explore and query with Cypher or SPARQL | [test_graph.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_graph.py) `test_query`; [test_rdf.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_rdf.py) `test_sparql`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `graph-find` | +| The graph as it was at any version | [test_graph_history.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_graph_history.py) `test_the_explorer_and_cypher_as_of_a_version`, `test_the_canvas_as_of_a_version`; [test_graph_history.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_graph_history.py) `test_every_curation_is_an_event_and_every_version_replays` | +| Every entity's own page at `/wiki//` | [test_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_entities.py) `test_wiki_addresses`; [wiki-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/wiki-page.test.tsx); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `entity-find` | +| Entities, topics and people edited and merged from their lists, the graph or an asset, with Undo instead of a confirm | [test_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_entities.py) `test_suggestions_and_merge_undo`; [entities-curate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/entities-curate.test.tsx); [topics-curate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/topics-curate.test.tsx); [recording-entities-edit.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/recording-entities-edit.test.tsx); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `entity-merge`, `graph-merge`, `speaker-rename` | +| What is most like a passage, an asset, a topic or an entity | [test_similar.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_similar.py) `test_like_a_text_by_its_key_words`, `test_like_a_resource_by_words_topics_and_entities`, `test_like_a_topic_and_an_entity`; [similar.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/similar.test.tsx) | +| With anytopdf, photos found by what they show | [test_photos.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_photos.py) `test_photos_found_by_what_they_show`, `test_off_by_default` | + +## Learn from it + +| Requirement | Checks | +|---|---| +| A two-host podcast episode, every claim cited and checked against its source, in two voices | [test_podcasts.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_podcasts.py) `test_episode_is_made_by_a_job`, `test_fact_check_rewrites_or_drops`, `test_episode_gets_two_voices`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `podcast-create`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `podcast-create` ≤ 5 clicks | +| Notes beside every asset, entity and topic, written by you or the assistant | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_pages_of_things`, `test_links_and_backlinks`, `test_assistant_keeps_notes`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `note-create` | +| A tree of notes renamed, moved and pinned (pins are your own), today's note one click away, and a search | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_tree`, `test_pins`, `test_search`; [notes-tree.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-tree.test.tsx); [notes-today.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-today.test.tsx) `is one click from the Notes page` | +| Undo and Recently deleted instead of a confirm | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_recently_deleted`; [notes-delete.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-delete.test.tsx) | +| An asset or a collection exported as an evidence PDF, whole or the parts you pick | [test_evidence.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_evidence.py) `test_a_collection_as_one_evidence_pdf`, `test_everything_lens_found_goes_in`, `test_picking_the_parts`; [evidence-parts.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/evidence-parts.test.tsx) | + +## Automate it + +| Requirement | Checks | +|---|---| +| Pipelines and workflows on a canvas | [test_workflows.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_workflows.py) `test_workflows_run_in_pipelines_drawn_as_graphs`; [workflow-canvas.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-canvas.test.ts); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `pipeline-create`, `workflow-create` | +| Routines on a schedule | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_routines_are_checked_and_run_when_due`, `test_schedules`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `routine-create`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `routine-create` ≤ 2 clicks | +| Batch runs over a collection | [test_batches.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_batches.py) `test_collections_and_batches` | +| Notifications to webhooks, Slack, Discord and Matterbridge | [test_notifications.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notifications.py) `test_owners_manage_targets_and_a_test_is_signed`, `test_runs_that_end_are_sent_once_to_matterbridge` | +| Everything that happens is kept as W3C Activity Streams activities, read by agents through the API, MCP and the assistant | [test_events.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_events.py) `test_activities_have_one_shape`; [test_activities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_activities.py) `test_the_timeline_lists_new_assets_newest_first`, `test_agents_read_the_timeline_too` | +| The Timeline page and Latest on Home, marking what you've seen | [test_timeline_marks.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_timeline_marks.py) `test_marks_are_set_cleared_and_filter_the_timeline`; [timeline-ui.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/timeline-ui.test.tsx) `Home's Latest card lists what you haven't reviewed` | +| Automations act on the stream, on a schedule, at a time or when their own webhook is called | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_each_new_activity_runs_each_automation_it_matches_once`, `test_a_schedule_runs_its_automation_when_its_timer_fires`, `test_a_call_to_a_webhook_runs_its_automation`; [test_timer_triggers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_timer_triggers.py) `test_an_at_trigger_fires_once_then_reads_done` | +| They test conditions, then tell you, ask the model or the agent, use a tool, call a web address with sealed credentials, run Lens work, or change assets and notes | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`; [test_actions_messages.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_messages.py) `test_notify_tells_me_in_my_chat_app`; [test_actions_ai.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_ai.py) `test_ask_model_answers_in_text_or_as_the_fields_a_schema_names`, `test_ask_agent_works_on_what_happened_and_saves_the_conversation`, `test_call_tool_reads_as_the_person_and_changes_only_by_asking_them`; [test_actions_web.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_web.py) `test_credentials_in_headers_are_sealed_to_the_automation_and_the_host_and_never_shown`; [test_actions_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_run.py) `test_run_workflow_queues_it_on_the_asset_as_the_automation`, `test_run_pipeline_processes_the_asset_again`; [test_actions_lens.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_lens.py) `test_set_field_is_kept_in_the_assets_history_by_the_automation`, `test_write_note_writes_a_page_adds_to_a_things_page_or_notes_an_asset` | +| What reaches outside Lens, or changes more than what set it off, asks first: on the Approvals page or with a yes in chat; agents decline but never approve | [test_automation_spec.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_spec.py) `test_risk_follows_the_effect`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_approve_carries_the_action_out_and_the_run_records_it`, `test_agents_over_mcp_list_and_decline_but_never_approve`; [test_approvals_chat.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals_chat.py) `test_a_yes_in_my_chat_decides_an_action_inside_lens`; [approvals-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/approvals-page.test.tsx) `groups what waits, puts the first in focus, and Enter approves it` | +| Each automation has limits and budgets | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`, `test_a_run_over_budget_is_held_for_the_persons_pick` | +| Agents make, try and run automations over MCP and in the assistant, or draft one from a sentence | [test_automation_tools.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_automation_tools.py) `test_agents_save_try_and_run_automations_over_mcp`, `test_the_assistant_saves_at_once_and_asks_to_switch_on_or_run`; [test_automations.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_automations.py) `test_an_automation_is_drafted_from_a_sentence` | + +## See what it costs + +| Requirement | Checks | +|---|---| +| An activity ledger of every model call, with reports | [test_activity.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_activity.py) `test_a_routines_calls_count_for_it_its_run_and_the_jobs_it_queued`, `test_chats_and_people_see_only_their_own` | +| One overall budget, asked after a cloud model, with a soft limit that warns and a hard limit that stops paid calls | [test_budgets.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_budgets.py) `test_one_budget_for_everything_with_a_soft_and_a_hard_limit`; [test_setup.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_setup.py) `test_the_wizard_asks_only_what_has_no_answer` | +| A meter in the top bar; the Costs page by day, model and what cost most | [budget-meter.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/budget-meter.test.tsx) `the top bar's budget meter`; [test_budgets.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_budgets.py) `test_the_meter_follows_the_namespace_and_its_budget` | +| The Costs page's charts | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | +| Routine choices go to small decision models, the LLM only when it adds value | [test_decisions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_decisions.py) `test_the_decision_model_answers_when_it_has_a_key`, `test_the_language_model_decides_without_one_and_when_it_fails` | + +## Keep it up to date and backed up + +| Requirement | Checks | +|---|---| +| A daily check for a new release, told to admins by email or a notification | [test_updates.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_updates.py) `test_the_daily_check_runs_when_due_and_can_be_turned_off`, `test_a_new_release_is_announced_once` | +| One click (or one API call) upgrades through the machine's agent, showing each step | [test_updates.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_updates.py) `test_one_click_upgrade_hands_the_release_to_the_agent`; [settings-updates.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/settings-updates.test.tsx) `shows each step of the upgrade as the machine goes`; [install.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/install.yml) `An upgrade asked for in the web app is installed by the machine's agent` | +| A new version that doesn't start is rolled back | [test_upgrade_agent.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_upgrade_agent.py) `test_a_release_that_doesnt_start_is_rolled_back` | +| The same agent changes whether other devices reach Lens and which edition runs | [test_install_settings.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_install_settings.py) `test_the_agent_sets_up_the_version_running_with_the_change`, `test_the_agent_passes_the_change_to_install_sh`; [install-settings.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/install-settings.test.tsx) | +| Backups of the database and files: take, list, download and restore; unchanged files take no new room | [test_backups.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_backups.py) `test_take_download_and_restore`, `test_backups_keep_and_restore_the_files_too` | +| Backups on a schedule, each copied, encrypted, to a storage connection | [test_backups.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_backups.py) `test_backups_on_a_schedule`, `test_copies_are_sealed_with_secret_key` | +| A scheduled backup's copy reaching a storage connection, in CI | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | + +## Watch it run + +| Requirement | Checks | +|---|---| +| Structured logs, read in Settings › Logs, and optionally sent to a syslog server | [test_logs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_logs.py) `test_viewer_reads_the_file_newest_first_and_filters`, `test_syslog_off_by_default_then_sent_over_udp` | +| OpenTelemetry traces and metrics, off until turned on | [test_telemetry.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_telemetry.py) `test_off_by_default_sends_nothing`, `test_traces_and_metrics_go_to_the_endpoint` | +| A Prometheus `/metrics` endpoint, off until turned on | [test_prometheus.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_prometheus.py) `test_off_by_default`, `test_needs_its_token` | +| One switch starts Prometheus, node-exporter and Grafana next to Lens | [test_monitoring.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_monitoring.py) `test_turning_monitoring_on_and_off`, `test_the_agent_changes_compose_profiles` | +| The Analytics page: active people, churn, retention and feature use, worked out on this machine | [test_analytics.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_analytics.py) `test_metrics_active_churn_retention`; [analyticsPage.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/analyticsPage.test.tsx) | +| The same daily events to a Google Analytics 4 property of your own, off unless turned on | [test_analytics.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_analytics.py) `test_events_kept_locally_with_ga_off`, `test_daily_events_once_a_day_counts_only` | + +## Stay private and safe + +| Requirement | Checks | +|---|---| +| Passkeys instead of passwords | [test_passkeys.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_passkeys.py) `test_first_admin_with_a_passkey_and_no_password`, `test_sign_in_with_a_passkey` | +| Encryption at rest, with a key per namespace | [test_keyring.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_keyring.py) `test_each_namespace_gets_its_own_key_kept_only_wrapped`; [test_encryption.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_encryption.py) `test_a_new_archive_encrypts_and_still_plays_seeks_and_processes` | +| Roles per namespace: viewer, editor, owner, or custom roles made of permissions | [test_roles.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_roles.py) `test_builtin_roles_keep_their_meaning`, `test_custom_roles_allow_exactly_their_permissions`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `person-add` | +| Email invitations, and removing someone for good | [test_users.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_users.py) `test_invites_are_emailed_when_mail_is_set_up`, `test_admins_remove_people_for_good` | +| An audit log | [test_settings.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_settings.py) `test_settings_are_editable_and_secret`; [test_files.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_files.py) `test_change_a_file_and_the_audit_log` | +| Outside text reaches the model marked as data; text that reads like instructions is flagged, by patterns or an optional local classifier | [test_prompt_injection.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_prompt_injection.py) `test_what_tools_read_is_marked_as_data`, `test_a_result_that_reads_like_instructions_is_flagged`; [test_injection_model.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_injection_model.py) `test_until_the_model_is_here_the_patterns_stand_alone` | +| Once an answer has read outside text, sending anything out, or the browser acting on what the assistant wrote, waits for a yes | [test_prompt_injection.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_prompt_injection.py) `test_a_web_tool_asks_first_once_the_answer_read_outside_text`; [test_browser.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_browser.py) `test_after_reading_a_page_the_browser_asks_before_what_the_person_didnt_write` | +| Chat-room answers have the links they built broken up; agents over MCP get the archive as data | [test_prompt_injection.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_prompt_injection.py) `test_a_room_gets_the_links_the_answer_built_broken_up`; [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_what_the_archive_says_reaches_agents_as_data` | +| HTTPS set up in Settings, with certificates that renew, only for its own domains and optionally only for client certificates | [test_https.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_https.py) `test_https_is_set_up_in_the_app`, `test_caddys_configuration_follows_the_settings`, `test_strict_sni_client_certificates_and_tls_13` | +| Remote access through a Cloudflare tunnel | [test_tunnel.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_tunnel.py) `test_remote_access_is_set_up_in_the_app`, `test_a_quick_tunnel_runs_in_one_process_and_stops_when_turned_off`, `test_a_managed_tunnel_is_made_routed_and_given_a_dns_record` | +| The threat model's three files agree | [threat-model.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/threat-model.yml) `validate`, `Validate the threat model`; [validate_threat_model.py](https://github.com/adeelahmad/lense/blob/main/ci/validate_threat_model.py) | + +## Use open standards + +| Requirement | Checks | +|---|---| +| RDF and Dublin Core for every record | [test_rdf.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_rdf.py) `test_recording_uri_public_and_members`, `test_import_round_trip_and_dublin_core` | +| OAuth for apps and OpenID Connect sign-in | [test_oauth.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_oauth.py) `test_discovery_says_where_everything_is`, `test_an_app_gets_access_and_acts_with_the_persons_roles`; [test_external_login.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_external_login.py) `test_signing_in_connects_the_account_with_the_same_verified_email` | +| MQTT: a built-in broker, and a client for another one | [test_sensors.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_sensors.py) `test_broker_is_an_ordinary_local_broker`, `test_bridge_client_reads_another_broker` | +| IIIF for publishing | [test_iiif.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_iiif.py) `test_manifests_collections_search_state_discovery`, `test_authorization_flow` | +| OpenTelemetry and Prometheus for monitoring | [test_telemetry.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_telemetry.py) `test_traces_and_metrics_go_to_the_endpoint`; [test_prometheus.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_prometheus.py) `test_what_it_measures` | +| An optional Fedora mirror | [test_fedora.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_fedora.py) `test_off_by_default`, `test_sync_sends_changes_and_deletes` | + +## Light by default + +| Requirement | Checks | +|---|---| +| Heavy parts stay off or are fetched on first use: speech engines, a local LLM, document conversion, Apache Tika, the NSFW model, Fedora | [test_components.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_components.py) `test_engines_are_fetched_on_first_use`; [test_local_llm.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_local_llm.py) `test_settings_take_a_listed_model_or_any_gguf_on_hugging_face`; [test_convert.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_convert.py) `test_libreoffice_and_chromium_are_installed_the_first_time_theyre_needed`; [test_tika.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_tika.py) `test_off_by_default_nothing_changes`; [test_nsfw.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_nsfw.py) `test_off_by_default_nothing_is_fetched_or_classified`; [test_fedora.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_fedora.py) `test_off_by_default` | +| The MCP runner is off until turned on | [test_mcp_servers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp_servers.py) `test_a_server_that_runs_as_a_program_runs_in_the_runner` | +| Setup asks only what has no default: the model, and a budget for a cloud model | [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `setup-wizard`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `setup-wizard` ≤ 1 click, ≤ 0 keys; [test_setup.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_setup.py) `test_the_wizard_asks_only_what_has_no_answer`, `test_a_paid_model_from_the_environment_still_asks_for_a_budget` | +| Picking a cloud provider fills in its address and model; only the key is left to paste | [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `cloud-model`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `cloud-model` ≤ 3 clicks; [test_setup.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_setup.py) `test_offers_cloud_model_providers`; [settings-llm-presets.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/settings-llm-presets.test.tsx) `switches to a cloud provider with one click and a pasted key` | +| Settings are in named groups, a search finds any one by name, and rarely changed options stay folded | [settings-find.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/settings-find.test.tsx) `puts every section in exactly one group`, `finds a field by its label, and opens the section on it`; [settings-save.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/settings-save.test.tsx) `are folded under a line saying what's in them, the main one left in view` | +| The web app works on a phone: every journey is surveyed on one, and none may take more than its baseline | [clicks.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/clicks.yml) `Survey on a phone`; [baseline-phone.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline-phone.json); [phone-layout.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/phone-layout.test.tsx) | +| The default install runs on a Raspberry Pi (arm64) with a cloud model | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) | diff --git a/mkdocs.yml b/mkdocs.yml index 9d8d1d712..0f43c58e3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -119,6 +119,7 @@ nav: - Technology: technology-selection.md - API work the design needs: backend-gaps.md - Contributing: contributing.md + - Requirements and checks: requirements.md - Changelog: CHANGELOG.md plugins: From 8728ef68954f992de57938a4b123dd53f1fa8b9c Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sat, 17 Oct 2026 02:40:29 +0000 Subject: [PATCH 2/2] docs(readme): list the merged answer record as built #469 merged while this was open: its line moves from In progress to the built list, with its checks. The scorecard (#487) is merged too, so Planned keeps only what it doesn't measure yet. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01MTK8aqzLBhdhfCmVf2LaWZ --- README.md | 25 +++++++++++++------------ docs/requirements.md | 1 + 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 48f738527..83d7342d1 100644 --- a/README.md +++ b/README.md @@ -77,13 +77,15 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p meaning, once turned on), reads it, cites it and acts on it with tools: changes wait for your approval. Once an admin turns them on, it also searches the web and reads pages, cited like the archive, and runs Python and bash in a sandbox of each conversation's own. Regenerating an answer or editing a question makes a new version beside the old one, and - nothing is lost. It reads answers aloud with a speech provider, macOS `say` or StyleTTS2 on this machine (opt-in), and - `/clear` and `/compact` keep long conversations short. Each namespace can have its own assistant with its own name, - instructions and memory, answering in Telegram and Slack (built in, with voice notes and files), in Discord, Matrix, - WhatsApp and other chat rooms through Matterbridge, and by email at an address of its own that you email or forward - things to. ([Chat](docs/assistant.md), [web search](docs/assistant.md#web-search-and-reading-pages), - [code sandbox](docs/assistant.md#code-sandbox), [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), - [speech providers](docs/speech-providers.md); [checks](docs/requirements.md#ask-and-get-cited-answers)) + nothing is lost. Each answer keeps a record of what produced it (the model, its tools, how long it took and what it + cost), opened from its header. It reads answers aloud with a speech provider, macOS `say` or StyleTTS2 on this machine + (opt-in), and `/clear` and `/compact` keep long conversations short. Each namespace can have its own assistant with + its own name, instructions and memory, answering in Telegram and Slack (built in, with voice notes and files), in + Discord, Matrix, WhatsApp and other chat rooms through Matterbridge, and by email at an address of its own that you + email or forward things to. ([Chat](docs/assistant.md), [web search](docs/assistant.md#web-search-and-reading-pages), + [code sandbox](docs/assistant.md#code-sandbox), [answer records](docs/assistant.md#what-each-answer-records), + [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), [speech providers](docs/speech-providers.md); + [checks](docs/requirements.md#ask-and-get-cited-answers)) - **Extend the assistant.** Add tools (a prompt, a web request, a graph drawn on the canvas, or sandboxed Python), skills, hooks and MCP servers, by code, on the canvas or just by asking. Remote MCP servers connect by address, and ones that run as a program (`npx`, `uvx`) run in a sandboxed container of their own, off until you turn it on. Add @@ -198,8 +200,7 @@ These are open pull requests, not yet part of Lens: ([#398](https://github.com/adeelahmad/lense/pull/398)). - **The assistant** answering questions about Lens itself, from its docs and this server ([#400](https://github.com/adeelahmad/lense/pull/400)). -- **Chat:** each answer recording what produced it ([#469](https://github.com/adeelahmad/lense/pull/469)), and a - conversation exported as Markdown, PDF or JSON ([#474](https://github.com/adeelahmad/lense/pull/474)). +- **Chat:** a conversation exported as Markdown, PDF or JSON ([#474](https://github.com/adeelahmad/lense/pull/474)). - **Notes:** download a note, or all of a namespace's notes, as Markdown ([#419](https://github.com/adeelahmad/lense/pull/419)). - **Sources:** RSS and Atom news feeds ([#453](https://github.com/adeelahmad/lense/pull/453)). @@ -229,9 +230,9 @@ Next, and not in a pull request yet; each links to where it is tracked: ([storage](docs/storage.md)). - **Logging:** sending straight to Loki or another HTTP endpoint, and client certificates for TLS ([logging](docs/logging.md)). -- **Proof:** a scorecard of measured numbers per release ([#481](https://github.com/adeelahmad/lense/issues/481)), and - each release tested by upgrading an install of the last one, holding data - ([#482](https://github.com/adeelahmad/lense/issues/482)). +- **Proof:** each release tested by upgrading an install of the last one, holding data + ([#482](https://github.com/adeelahmad/lense/issues/482)), and the scorecard's install-to-first-answer and Raspberry Pi + numbers, not measured yet ([#481](https://github.com/adeelahmad/lense/issues/481)). ## Stack diff --git a/docs/requirements.md b/docs/requirements.md index 8a0eeacdb..d3670a383 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -76,6 +76,7 @@ request, to another check that proves the same thing; loosening a requirement or | Web search and reading pages, cited like the archive, off until an admin turns it on | [test_web_search.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_web_search.py) `test_an_answer_cites_the_web_and_the_conversation_keeps_it`, `test_reading_pages_and_pdfs`; [chat-web-citations.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-web-citations.test.tsx) | | Python and bash in a sandbox of each conversation's own, off until an admin turns it on | [test_sandbox.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_sandbox.py) `test_python_and_bash_share_one_sandbox_per_conversation`, `test_the_sandbox_is_off_until_an_admin_turns_it_on_for_the_people_it_names` | | Regenerating an answer or editing a question makes a new version beside the old one | [test_chat_versions.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chat_versions.py) `test_regenerate_edit_switch_and_go_on`; [chat-versions.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-versions.test.tsx) `regenerating an answer`, `the version switch` | +| Each answer keeps a record of what produced it: the model, its tools, how long it took and what it cost | [test_chat_records.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chat_records.py) `test_an_answer_records_what_produced_it`, `test_a_regenerated_answer_has_its_own_record`; [chat-details.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-details.test.tsx) `says what produced it: model and host, scope, tools, use, tokens, cost, time and how it ended` | | Answers read aloud by a speech provider, macOS `say` or StyleTTS2 (opt-in) | [test_voice.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_voice.py) `test_answers_are_read_aloud_by_a_speech_model`; [test_speech.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_speech.py) `test_macos_say_reads_aloud_on_a_mac`, `test_styletts2_reads_aloud_from_its_own_service`; [styletts2.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/styletts2.yml) `Read a sentence aloud` | | `/clear` and `/compact` | [test_compaction.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_compaction.py) `test_clear_starts_afresh_and_compact_folds_everything_now`, `test_commands_in_the_app` | | A namespace's own assistant, with its own name, instructions and memory | [test_ns_assistant.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_ns_assistant.py) `test_off_by_default_and_owners_turn_it_on`, `test_remembers_across_conversations_with_citations`; [ns-assistant.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/ns-assistant.test.tsx) `saving a name and forgetting a memory` |