From 0139f886f1503b17df2b4de27c887cd6fcd0cfb3 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 00:44:03 +0000 Subject: [PATCH] feat(settings): change network reach and the edition from the web app MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Settings › Access & embedding gets "Reachable from other devices on this network", and Settings › Components the edition (Full or Lean). Changing either says what will happen and applies with one button: the machine's upgrade agent runs install.sh again for the version running with LENS_PUBLIC or LENS_TARGET, and Lens restarts, the way Settings › HTTPS opens the HTTPS ports with LENS_HTTPS. Without an agent the section shows the install line to run. Before, both needed the install line again. Choices made: - Current values come from the environment Lens was started with. docker-compose.prod.yml passes LENS_TARGET (${LENS_TARGET:-full}, the build target's own default) and LENS_BIND (${LENS_BIND:-127.0.0.1}, the web port's) to the API and workers; a microsandbox already exports .env. install.sh needs no change for it. Started any other way, both read as null and the UI says it can't tell, never a guess. A loopback LENS_BIND is "this machine only", any other address "reachable". - updates.reconfigure takes https, public and target and writes only the lines that are set: an HTTPS request is byte for byte what it was. lens-upgrade.sh reads public= and target= with val, keeps only 0|1 and full|lean, passes LENS_TARGET (else .env's) and LENS_PUBLIC (empty when not asked, which is install.sh's own fallback to LENS_BIND). - GET/PUT /api/v1/admin/install on the updates router (AdminReader, AdminWriter, audited as lens.install). PUT leaves alone what is so already, and with nothing left to change restarts nothing (200); without an agent it answers 409 with the line to run; `manual` has the line for each choice. - MCP tools install_settings and set_install_settings, next to monitoring and set_monitoring, the comparable machine tools. The chat assistant's ops tools expose no machine changes, so none were added there. - UI: no confirm dialog; switching back cancels; keeping Lens to this machine while the page is open at a network address warns that this browser may lose it there. - Stored data: the run record gains optional public and target fields, read with .get and simply absent from older rows, so no upgrade step. - threat-model.md records the new admin surface (the variants, the inbox lines and the endpoints); the YAML and JSON follow it. Refine later (docs/updates.md): Windows installs get the Linux/Mac line; installs not started by install.sh, compose or a microsandbox show both as not known. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_014Y4mUBxDN6pTR2fykqrfYV --- CHANGELOG.md | 7 + docker-compose.prod.yml | 4 + docs/configuration.md | 4 +- docs/get-started.md | 3 +- docs/mcp.md | 5 +- docs/updates.md | 39 ++- fastapi_backend/app/api/mcp_tools.py | 47 ++++ fastapi_backend/app/api/v1/routes/updates.py | 41 ++- fastapi_backend/app/domain/updates.py | 97 ++++++- fastapi_backend/app/schemas/updates.py | 40 ++- .../tests/api/test_install_settings.py | 187 +++++++++++++ fastapi_backend/tests/api/test_mcp.py | 28 ++ install.sh | 6 +- .../__tests__/install-settings.test.tsx | 226 +++++++++++++++ nextjs-frontend/app/openapi-client/index.ts | 11 + nextjs-frontend/app/openapi-client/sdk.gen.ts | 42 +++ .../app/openapi-client/types.gen.ts | 135 +++++++++ .../components/settings/bodies.tsx | 3 + .../components/settings/install-settings.tsx | 260 ++++++++++++++++++ nextjs-frontend/components/settings/model.ts | 4 +- nextjs-frontend/openapi.json | 223 +++++++++++++++ packaging/host/lens-upgrade.sh | 18 +- threat-model.json | 14 +- threat-model.md | 9 +- threat-model.yaml | 33 ++- 25 files changed, 1451 insertions(+), 35 deletions(-) create mode 100644 fastapi_backend/tests/api/test_install_settings.py create mode 100644 nextjs-frontend/__tests__/install-settings.test.tsx create mode 100644 nextjs-frontend/components/settings/install-settings.tsx diff --git a/CHANGELOG.md b/CHANGELOG.md index d15df6a0f..1867c4a6d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version takes their roles and their private chats, notes and views, and keeps what they shared under their name; their email can be invited again ([Authentication](docs/authentication.md#removing-someone)). The assistant gets `list_people`, `invite_person` and `change_person`, which always wait for an admin's approval. +- **Reachable from the network, and the edition, changed in Settings.** Settings › Access & embedding has **Reachable + from other devices on this network**, and Settings › Components the edition: Full (LibreOffice and Chromium, to read + Office files, web pages and emails) or Lean. Changing either says what will happen and applies with one button: the + machine's upgrade agent runs the install again for the version running, and Lens restarts. Before, the install line + had to run again with `LENS_PUBLIC` or `LENS_TARGET`; without an agent, Settings shows that line. Agents use + `GET`/`PUT /api/v1/admin/install` and the MCP tools `install_settings` and `set_install_settings` + ([Updates](docs/updates.md#setting-the-machine-up-again)). - **NSFW classification, with a model on your server.** Turn it on in Settings → NSFW for every namespace, or on a namespace's page for that one (off by default). The new `nsfw` step scores a video's sampled frames, a document's pages or an image with an Apache-2.0 model (fetched on first use, about 105 MB with ONNX Runtime) and flags the diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml index 97284fdf7..bff3a6f61 100644 --- a/docker-compose.prod.yml +++ b/docker-compose.prod.yml @@ -76,6 +76,10 @@ x-backend: &backend # upgrading from the web app (docs/updates.md): the folder install.sh's upgrade agent watches, and the install folder LENS_CONTROL_DIR: /control LENS_HOST_DIR: ${LENS_DIR:-} + # how this stack runs, shown (and changed through that agent) in Settings: the edition built above, and where the + # web app's port is published (127.0.0.1: this machine only) + LENS_TARGET: ${LENS_TARGET:-full} + LENS_BIND: ${LENS_BIND:-127.0.0.1} volumes: - archive-data:/data - ./.lens-control:/control diff --git a/docs/configuration.md b/docs/configuration.md index b3c8f5ee1..bc5e5bc45 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -34,6 +34,7 @@ with AES-GCM and are write-only: the API reports whether one is set, never its v | `LENS_SETUP_SETTINGS` | | settings to start with, as JSON `{"section": {"key": value}}` (or that JSON base64url-encoded): saved once at startup as if made in the app, so unlike the variables above they stay editable there. The [web installer](get-started.md#the-web-installer) writes its answers here; a `budget` section (`hard_usd`, `soft_usd`, `period`, `on_over`, `price_in`, `price_out`) sets [the overall budget](budgets.md#one-budget-for-everything) | | `LENS_SETUP_WIZARD` | | `off`: never show the setup wizard | | `LENS_TELEMETRY` / `LENS_TELEMETRY_ENDPOINT` / `LENS_TELEMETRY_HEADERS` | off | opt-in OpenTelemetry traces and metrics, sent only to this OTLP/HTTP endpoint ([Telemetry](telemetry.md)); `LENS_TELEMETRY=off` keeps it off whatever Settings say | +| `LENS_TARGET` / `LENS_BIND` | | how install.sh set the machine up: the edition (`full` or `lean`) and where the web app's port is published (`0.0.0.0`: the network, `127.0.0.1`: this machine only). `docker-compose.prod.yml` and a microsandbox pass them on from `.env`, and Settings show them; unset, Lens says it doesn't know. They're changed in Settings, which has the install run again ([Updates](updates.md#setting-the-machine-up-again)) | | `LENS_LOG_LEVEL` / `LENS_LOG_FORMAT` | info / text | the log level and console format for the API and every worker ([Logging](logging.md)); win over Settings → Logs | | `FRONTEND_URL` | `http://localhost:3000` | the web app's address as people use it: links in emails, and where apps send people to sign in ([OAuth](authentication.md#oauth)) | | `CORS_ORIGINS` | `["http://localhost:3000"]` | origins allowed to call the API from a browser | @@ -105,7 +106,8 @@ Start from `fastapi_backend/archive.example.yaml`, which documents every key. Th Admins can change, in Settings: transcription, speaker separation, speech providers, voice IDs, analysis, the LLM provider and key, a local model, the AI assistant, search (word search and search by meaning), reports and graph, -video, OCR, faces and objects, [NSFW classification](nsfw.md), workers, sign-in, components, access and embedding (embed frame ancestors, transcript +video, OCR, faces and objects, [NSFW classification](nsfw.md), workers, sign-in, components (and the edition, full or +lean), access and embedding (whether other devices on the network reach Lens, embed frame ancestors, transcript upload limit, allowed hosts, trusted proxies, session length), remote access, email, chat rooms, notifications, telemetry, the Fedora repository, sensors, file shares, uploads, storage, documents, API keys, and IIIF and metadata. The API refuses an allowed-host list that leaves out the address you are using. diff --git a/docs/get-started.md b/docs/get-started.md index 4ae5aa407..29bd12fbc 100644 --- a/docs/get-started.md +++ b/docs/get-started.md @@ -14,7 +14,8 @@ code already filled in. On a server without a desktop, Lens is reachable from th server's address. Run the same line again to update: the secrets, the database and the archive are kept. Settings go before `sh`: `LENS_DIR`, `LENS_PORT`, `LENS_PUBLIC` (`1`: from the network, `0`: this machine only), `LENS_TARGET` (`lean` for the smaller image, kept when the line runs again) and `LENS_REF` (a branch or tag) change the defaults; -the top of `install.sh` lists them. If Lens doesn't answer in time (`LENS_WAIT`, in seconds: 10 minutes with Docker, +the top of `install.sh` lists them. Whether the network reaches it and the edition can be changed later in Settings +(Access & embedding, Components), with no terminal ([Updates](updates.md#setting-the-machine-up-again)). If Lens doesn't answer in time (`LENS_WAIT`, in seconds: 10 minutes with Docker, 30 in a microsandbox), the installer stops waiting and shows the last lines Lens logged and why. Where it can (Linux with KVM, or a Mac with Apple silicon), a first install asks whether to run Lens in Docker (the diff --git a/docs/mcp.md b/docs/mcp.md index 965bb4c99..8cf0b644f 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -75,6 +75,8 @@ front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](aut | `read_file` | a text file of that tree (an asset's `.txt` text or `.json` metadata), a page at a time; other files' download link | | `file_storage` | where Lens keeps its own files (notes' images and attachments) and the connections it could use; `check` tests it. Admins only | | `set_file_storage` | keep new files on this machine or a storage connection (folder, rclone crypt); checked before it's saved. Admins only, with a write-scope token | +| `install_settings` | whether other devices on the network reach Lens and its edition (full or lean), whether the machine's upgrade agent can change them, and the install line to run when it can't. Admins only | +| `set_install_settings` | open Lens to the network or keep it to this machine, or change the edition: the machine sets itself up again and Lens restarts ([Updates](updates.md#setting-the-machine-up-again)). Admins only, with a write-scope token | | `propose_graph_change` | ask for two entities to be merged or linked; it waits in Proposed changes unless `apply` makes it at once (it can be undone). Needs a write-scope token and editor access | Results are JSON (as `structuredContent`, and the same as text). Links open the recording's page in the web app at @@ -86,7 +88,8 @@ expect (`results` with `id`, `title`, `url`; `fetch` by that `id`). Every tool reads, except `suggest_topic`, which adds suggestions people accept or dismiss, `propose_graph_change`, which records an undoable change (proposed by default), `write_note`, which adds a note, `create_podcast`, which queues an episode, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and -editors), and `set_file_storage`, which changes where new files are kept (admins). +editors), `set_file_storage`, which changes where new files are kept (admins), and `set_monitoring` and +`set_install_settings`, which have the machine change how Lens runs (admins). Importing, editing and curating stay in the web app and the API. ### What the archive says is data diff --git a/docs/updates.md b/docs/updates.md index 3580ac346..ae967015f 100644 --- a/docs/updates.md +++ b/docs/updates.md @@ -2,7 +2,8 @@ Lens tells admins when a new release is out, backs the database up, and installs the release from the web app, so upgrading never needs a terminal. Status: **built**: the check, backups, one-click upgrades with their progress and a -rollback when the new version doesn't start, in Settings → Updates and a banner for admins. +rollback when the new version doesn't start, in Settings → Updates and a banner for admins, and +[setting the machine up again](#setting-the-machine-up-again) from Settings. ## The check @@ -70,6 +71,42 @@ with `LENS_UPGRADE_AGENT=0`), the API gives the one line to run on the machine i `updates.auto_install: true` installs a new release on its own as soon as the check sees it, once per release. It is off by default: what Lens runs changes only when an admin says so. +## Setting the machine up again + +Some choices are made when Lens is installed, as settings on the install line. Three of them can be changed later from +the web app, with no terminal: the agent runs install.sh again for the version running (a request with an empty +`ref`), changing only what the request says, and Lens restarts. + +| In Settings | On the install line | In the request | +| --- | --- | --- | +| **Access & embedding › Reachable from other devices on this network** | `LENS_PUBLIC=1` (or `0`: this machine only) | `public=1` or `0` | +| **Components › Edition**: Full (LibreOffice and Chromium, to read Office files, web pages and emails) or Lean | `LENS_TARGET=full` or `lean` | `target=full` or `lean` | +| **HTTPS › Open the HTTPS ports** ([HTTPS](https.md)) | `LENS_HTTPS=1` | `https=1` or `0` | + +Changing one says what will happen (Lens restarts; opening it to the network lets other devices reach its sign-in +page; with Docker a new edition is built, which can take a few minutes), then one button applies it. Without an +agent, the same place shows the install line to run on the machine. install.sh keeps each choice in `.env` +(`LENS_BIND`, `LENS_TARGET`, `LENS_HTTPS`), so upgrades keep them; what a request leaves out stays as `.env` has it. +The agent takes nothing but `0` or `1`, and `full` or `lean`. + +Lens reads how it runs from what it was started with: `docker-compose.prod.yml` and a microsandbox pass on +`LENS_TARGET` and `LENS_BIND` (`0.0.0.0` or the machine's own address: the network; `127.0.0.1`: this machine only). +Started some other way, it says it doesn't know, rather than guessing. + +For agents: `GET /api/v1/admin/install` says `public` and `target` (or null), whether there's an agent, the install +line for each choice (`manual`) and how the last change went (`reconfigure`). `PUT /api/v1/admin/install` with +`{"public": true}`, `{"target": "lean"}` or both asks for the change (202, audited as `lens.install`); what is so +already is left alone, and with nothing left to change nothing restarts (200). Without an agent it answers 409 with +the line to run. The MCP tools `install_settings` and `set_install_settings` do the same (admins, and a write-scope +token to change them). + +### Refine later + +- Windows installs have no upgrade agent, and the line Lens shows to run there is the one for Linux and Macs + (`curl … | sh`), for upgrades, the HTTPS ports and these choices alike. +- Installs not started by install.sh, `docker-compose.prod.yml` or a microsandbox (Cloudron, the NAS packages, + Proxmox) show whether the network reaches Lens and its edition as not known. + ## Backups * `GET /api/v1/admin/backups`: the backups in `/backups`, newest first. diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py index f90e878d1..45101d7f3 100644 --- a/fastapi_backend/app/api/mcp_tools.py +++ b/fastapi_backend/app/api/mcp_tools.py @@ -49,6 +49,7 @@ sources, store, untrusted, + updates, ) from app.domain import entities as ents from app.domain import speakers as spk @@ -1738,3 +1739,49 @@ def set_monitoring(ctx: Context, on: bool) -> dict[str, Any]: raise ToolError(str(e)) from None auth.audit(ctx.db, ctx.user.as_audit(), "lens.monitoring", None, {"on": bool(on), "via": "mcp"}) return got + + +# ---------- how install.sh set this machine up: reachable on the network, and the edition (admins) ---------- +@tool( + "install_settings", + "How Lens is installed", + "Whether other machines on the network reach the web app (public true) or only this one, and the edition (target: " + "full, with LibreOffice and Chromium to read Office files, web pages and emails, or lean); null where Lens doesn't " + "know. Also whether this machine's upgrade agent can change them (agent), the install line to run on the machine " + "for each choice when it can't (manual), and how the last change went (reconfigure). Admins only.", +) +def install_settings(ctx: Context) -> dict[str, Any]: + if not ctx.user.admin: + raise ToolError("only admins see or change how Lens is installed") + return updates.install_view(ctx.db, ctx.cfg) | { + "settings": f"{ctx.web}/settings/access", + "edition_settings": f"{ctx.web}/settings/components", + } + + +@tool( + "set_install_settings", + "Change how Lens is installed", + "Make the web app reachable from other machines on the network (public true) or this machine only (false), or " + "change the edition (target full or lean). This machine's upgrade agent runs the install again for the version " + "running, and Lens restarts, away while it does (longer for a new edition with Docker, which builds it); " + "install_settings follows along. What is so already is left alone. Making it reachable from the network lets " + "anyone there reach the sign-in page. Admins only, with a token that has the write scope.", + Arg("public", "boolean", "reachable from other machines on the network (true) or this machine only (false)"), + Arg("target", "string", "the edition: full or lean", enum=updates.TARGETS), + writes=True, +) +def set_install_settings(ctx: Context, public: bool | None = None, target: str | None = None) -> dict[str, Any]: + if not ctx.user.admin: + raise ToolError("only admins see or change how Lens is installed") + if not ctx.user.can_write: + raise ToolError("this token is read-only: ask for a token with the write scope to change how Lens is installed") + if public is None and target is None: + raise ToolError("say what to change: public, target or both") + try: + asked = updates.change_install(ctx.db, ctx.cfg, public, target, by=ctx.user.email) + except updates.UpdateError as e: + raise ToolError(str(e)) from None + if asked: + auth.audit(ctx.db, ctx.user.as_audit(), "lens.install", None, asked | {"via": "mcp"}) + return updates.install_view(ctx.db, ctx.cfg) | {"changing": asked} diff --git a/fastapi_backend/app/api/v1/routes/updates.py b/fastapi_backend/app/api/v1/routes/updates.py index 0823d8b52..ac64c85e5 100644 --- a/fastapi_backend/app/api/v1/routes/updates.py +++ b/fastapi_backend/app/api/v1/routes/updates.py @@ -1,17 +1,27 @@ -"""New releases, upgrading from the web app, and the database backups (admins only; docs/updates.md).""" +"""New releases, upgrading from the web app, how the machine is set up (reachable on the network, the edition), and +the database backups (admins only; docs/updates.md).""" from __future__ import annotations import contextlib -from fastapi import APIRouter, HTTPException, Request +from fastapi import APIRouter, HTTPException, Request, Response from fastapi.responses import FileResponse from app.api.deps import AdminReader, AdminWriter, Cfg, Db, domain_errors from app.domain import auth, backups, monitoring, updates from app.schemas.backups import BackupSchedule, RestoreWork from app.schemas.common import Ok -from app.schemas.updates import Backup, Backups, Monitoring, MonitoringRequest, UpdateStatus, UpgradeRequest +from app.schemas.updates import ( + Backup, + Backups, + InstallChange, + InstallStatus, + Monitoring, + MonitoringRequest, + UpdateStatus, + UpgradeRequest, +) router = APIRouter(tags=["updates"]) @@ -46,6 +56,31 @@ def start_upgrade(user: AdminWriter, db: Db, cfg: Cfg, body: UpgradeRequest | No return UpdateStatus.model_validate(updates.view(db, cfg)) +@router.get("/admin/install") +def get_install(user: AdminReader, db: Db, cfg: Cfg) -> InstallStatus: + """How install.sh set this machine up: whether other machines on the network reach the web app, and the edition + (full or lean); what can change them here, and how the last change went.""" + return InstallStatus.model_validate(updates.install_view(db, cfg)) + + +@router.put("/admin/install", status_code=202) +def set_install(body: InstallChange, user: AdminWriter, response: Response, db: Db, cfg: Cfg) -> InstallStatus: + """Make the web app reachable from other machines on the network (public) or this machine only, or change the + edition (target: full, with LibreOffice and Chromium, or lean): this machine's upgrade agent runs install.sh again + for the version running with LENS_PUBLIC and LENS_TARGET, and Lens restarts; GET /admin/install follows along. + What is so already is left alone, and with nothing left to change nothing restarts (200). Without an agent, 409 + says the line to run on the machine instead (`manual` has each one).""" + try: + asked = updates.change_install(db, cfg, body.public, body.target, by=user.email) + except updates.UpdateError as e: + raise HTTPException(409, str(e)) from None + if asked: + auth.audit(db, user.as_audit(), "lens.install", None, asked) + else: + response.status_code = 200 + return InstallStatus.model_validate(updates.install_view(db, cfg)) + + @router.get("/admin/monitoring") def get_monitoring(user: AdminReader, db: Db) -> Monitoring: """Whether Prometheus, node-exporter and Grafana run next to Lens (the monitoring profile), and how the machine's diff --git a/fastapi_backend/app/domain/updates.py b/fastapi_backend/app/domain/updates.py index bf9ccc17b..391fbc14f 100644 --- a/fastapi_backend/app/domain/updates.py +++ b/fastapi_backend/app/domain/updates.py @@ -13,12 +13,18 @@ (upgrade-status, upgrade.log), which the web app shows. With no agent (an install that didn't use install.sh, or a machine where it couldn't set one up) the web app shows the one line to run instead. +**Setting the machine up again.** The same agent runs install.sh again for the version running, changing only what +the request says (reconfigure): the HTTPS ports open or closed (Settings › HTTPS), the web app reachable from other +machines on the network or from this one only (Settings › Access & embedding), and the edition, full or lean +(Settings › Components). How the machine runs now is what Lens was started with (installed(): LENS_BIND, LENS_TARGET). + updates.auto_install installs a new release by itself as soon as it is seen; off unless an admin turns it on. """ from __future__ import annotations import datetime as dt +import ipaddress import logging import os import pathlib @@ -42,6 +48,7 @@ LOG_LINES = 60 ACTIVE = ("backing-up", "queued", "running", "rolling-back") STEPS = ("download", "build", "start", "check") # what the agent reports it is on (after the backup) +TARGETS = ("full", "lean") # the editions install.sh builds (LENS_TARGET) # How long a request may sit in the inbox before the web app says nobody picked it up PICKUP_SECONDS = 120 @@ -250,15 +257,26 @@ def upgrade(db, cfg, ref=None, by=None, backup=True, wait=False): return run_state(db) -def reconfigure(db, cfg, https, by=None, wait=False): - """Have this machine run install.sh again for the version it runs, with the HTTPS ports opened (https=True) or - closed (docs/https.md). Lens restarts; nothing else changes. Returns the run.""" +def reconfigure(db, cfg, https=None, by=None, wait=False, *, public=None, target=None): + """Have this machine run install.sh again for the version it runs, changing only what is given: the HTTPS ports + opened (https=True) or closed (docs/https.md), the web app reachable from other machines on the network + (public=True) or from this one only, and the edition (target: full or lean). Lens restarts; nothing else changes. + Returns the run.""" + if https is None and public is None and target is None: + raise UpdateError("nothing to change: give https, public or target") + if target is not None and target not in TARGETS: + raise UpdateError(f"the edition is full or lean, not {target!r}") if not agent(): raise UpdateError("this machine has no upgrade agent; run the line shown instead") run = run_state(db) or {} if run.get("state") in ACTIVE and not run.get("error"): raise UpdateError("an upgrade is already under way; try again when it's done") - run = {"id": secrets.token_hex(8), "ref": "", "https": bool(https), "by": by, "state": "queued", "asked_at": _ts(), "error": None} + run = {"id": secrets.token_hex(8), "ref": "", "by": by, "state": "queued", "asked_at": _ts(), "error": None} + for k, v in (("https", https), ("public", public)): + if v is not None: + run[k] = bool(v) + if target is not None: + run["target"] = target _put(db, upgrade=run) t = threading.Thread(target=_hand_over, args=(db, run, False), daemon=True, name="reconfigure") t.start() @@ -267,10 +285,72 @@ def reconfigure(db, cfg, https, by=None, wait=False): return run_state(db) +def manual_reconfigure(cfg, https=None, public=None, target=None): + """The install line that sets this machine up again by hand with these changes (LENS_HTTPS, LENS_PUBLIC, + LENS_TARGET), on the machine Lens runs on.""" + if target is not None and target not in TARGETS: # it goes into a shell line + raise UpdateError(f"the edition is full or lean, not {target!r}") + given = (("LENS_HTTPS", https), ("LENS_PUBLIC", public), ("LENS_TARGET", target)) + env = [f"{k}={v if isinstance(v, str) else int(bool(v))}" for k, v in given if v is not None] + line = manual(cfg) + return f"{line[: -len(' sh')]} {' '.join(env)} sh" if env else line + + def manual_https(cfg, https): """The line that opens (or closes) the HTTPS ports by hand, on the machine Lens runs on.""" - line = manual(cfg) - return f"{line[: -len(' sh')]} LENS_HTTPS={1 if https else 0} sh" + return manual_reconfigure(cfg, https=https) + + +# ---------- how install.sh set this machine up: the edition, and who reaches the web app ---------- +def installed(): + """The edition (LENS_TARGET: full or lean) and whether the web app is reachable from other machines on the network + (LENS_BIND: 0.0.0.0, or this machine's address on it; 127.0.0.1 is this machine only), as Lens was started with + them: docker-compose.prod.yml passes both from .env, and a microsandbox reads .env itself. None where Lens wasn't + told (it wasn't started that way), never a guess.""" + target = (os.environ.get("LENS_TARGET") or "").strip().lower() + bind = (os.environ.get("LENS_BIND") or "").strip().strip("[]") + public = None + if bind == "localhost": + public = False + elif bind: + try: + public = not ipaddress.ip_address(bind).is_loopback + except ValueError: + pass + return {"public": public, "target": target if target in TARGETS else None} + + +def install_view(db, cfg): + """installed(), what can change it here (the agent, else the line to run for each choice), and the last run that + changed it.""" + run = run_state(db) + return { + **installed(), + "agent": agent(), + "manual": { + "public": manual_reconfigure(cfg, public=True), + "private": manual_reconfigure(cfg, public=False), + "full": manual_reconfigure(cfg, target="full"), + "lean": manual_reconfigure(cfg, target="lean"), + }, + "reconfigure": run if run and (run.get("public") is not None or run.get("target") is not None) else None, + } + + +def change_install(db, cfg, public=None, target=None, by=None): + """Have the machine set itself up again with what differs from how it runs now (public, target): Lens restarts. + Returns what was asked for, empty when it's all so already (nothing restarts). UpdateError when it can't: no agent + (the message has the line to run instead), or a run under way.""" + if target is not None and target not in TARGETS: + raise UpdateError(f"the edition is full or lean, not {target!r}") + now = installed() + ask = {k: v for k, v in (("public", public), ("target", target)) if v is not None and v != now[k]} + if not ask: + return {} + if not agent(): + raise UpdateError(f"this machine has no upgrade agent: run this on it instead: {manual_reconfigure(cfg, **ask)}") + reconfigure(db, cfg, by=by, **ask) + return ask def _hand_over(db, run, backup): @@ -282,7 +362,10 @@ def _hand_over(db, run, backup): run["backup"] = dst.name inbox = control_dir() / "inbox" part = inbox / f".upgrade.{run['id']}" - extra = f"https={1 if run['https'] else 0}\n" if run.get("https") is not None else "" + # what this run changes: https=, public= (1 or 0) and target= (full or lean), which the agent checks again + extra = "".join( + f"{k}={v if isinstance(v, str) else int(bool(v))}\n" for k in ("https", "public", "target") if (v := run.get(k)) is not None + ) part.write_text( f"id={run['id']}\nref={run['ref']}\n{extra}backup={run.get('backup') or ''}\nby={run.get('by') or ''}\nat={_ts()}\n", encoding="utf-8", diff --git a/fastapi_backend/app/schemas/updates.py b/fastapi_backend/app/schemas/updates.py index c6806d4c1..55e8ca06b 100644 --- a/fastapi_backend/app/schemas/updates.py +++ b/fastapi_backend/app/schemas/updates.py @@ -4,7 +4,7 @@ from typing import Literal -from pydantic import Field +from pydantic import Field, model_validator from app.schemas.backups import BackupSchedule, CopyWork, RestoreWork from app.schemas.common import RequestModel, ResponseModel @@ -22,6 +22,10 @@ class UpgradeRun(ResponseModel): id: str ref: str = Field(description="the release tag (or branch) being installed; empty: the one running, set up again") https: bool | None = Field(None, description="the HTTPS ports opened (true) or closed (false) by this run (docs/https.md)") + public: bool | None = Field( + None, description="the web app made reachable from other machines on the network (true) or this one only (false) by this run" + ) + target: Literal["full", "lean"] | None = Field(None, description="the edition this run changed to") state: Literal["backing-up", "queued", "running", "rolling-back", "done", "failed", "rolled-back"] = Field( description="rolling-back / rolled-back: the new version didn't start, so the agent went back to the one before" ) @@ -53,6 +57,40 @@ class UpdateStatus(ResponseModel): upgrade: UpgradeRun | None = Field(None, description="the last upgrade asked for") +class InstallLines(ResponseModel): + public: str = Field(description="the install line that makes the web app reachable from other machines (LENS_PUBLIC=1)") + private: str = Field(description="the install line that keeps it to this machine (LENS_PUBLIC=0)") + full: str = Field(description="the install line for the full edition (LENS_TARGET=full)") + lean: str = Field(description="the install line for the lean edition (LENS_TARGET=lean)") + + +class InstallStatus(ResponseModel): + public: bool | None = Field( + None, + description="the web app is reachable from other machines on the network (LENS_BIND), or from this one only; " + "none: Lens wasn't started by install.sh or docker-compose.prod.yml, so it doesn't know", + ) + target: Literal["full", "lean"] | None = Field( + None, + description="the edition (LENS_TARGET): full has LibreOffice and Chromium, to read Office files, web pages and " + "emails; lean doesn't. None: not known here", + ) + agent: str | None = Field(None, description="what changes them on this machine (systemd, launchd); none: run a `manual` line there") + manual: InstallLines = Field(description="the line to run on the machine Lens runs on for each choice, when there's no agent") + reconfigure: UpgradeRun | None = Field(None, description="the last run that changed either (as in /admin/updates)") + + +class InstallChange(RequestModel): + public: bool | None = Field(None, description="reachable from other machines on the network (true) or this machine only (false)") + target: Literal["full", "lean"] | None = Field(None, description="the edition: full or lean") + + @model_validator(mode="after") + def _something(self) -> InstallChange: + if self.public is None and self.target is None: + raise ValueError("say what to change: public, target or both") + return self + + class UpgradeRequest(RequestModel): ref: str | None = Field(None, max_length=100, description="a release tag; default: the latest release") backup: bool = Field(True, description="back the database up first") diff --git a/fastapi_backend/tests/api/test_install_settings.py b/fastapi_backend/tests/api/test_install_settings.py new file mode 100644 index 000000000..fbf5337b9 --- /dev/null +++ b/fastapi_backend/tests/api/test_install_settings.py @@ -0,0 +1,187 @@ +"""Changing how install.sh set the machine up, from the web app: whether other machines on the network reach the web +app, and the edition (full or lean). The machine's upgrade agent runs install.sh again for the version running with +LENS_PUBLIC or LENS_TARGET, as Settings › HTTPS does with LENS_HTTPS (docs/updates.md).""" + +from __future__ import annotations + +import pathlib +import shutil +import subprocess +import time + +import pytest + +from app.domain import updates +from tests.helpers import login, make_user + +AGENT = pathlib.Path(__file__).resolve().parents[3] / "packaging" / "host" / "lens-upgrade.sh" + + +@pytest.fixture +def control(tmp_path, monkeypatch): + """The control folder install.sh shares with the containers, with an upgrade agent watching it, and Lens started + the way docker-compose.prod.yml starts it: this machine only, the full edition.""" + d = tmp_path / "control" + (d / "inbox").mkdir(parents=True) + (d / "agent").write_text("kind=systemd\n") + monkeypatch.setenv("LENS_CONTROL_DIR", str(d)) + monkeypatch.setenv("LENS_BIND", "127.0.0.1") + monkeypatch.setenv("LENS_TARGET", "full") + return d + + +def admin(client, db): + make_user(db, "root@x.io", "root password 1", admin=True) + return login(client, "root@x.io", "root password 1") + + +def handed_over(control): + """The request in the agent's inbox, once the web app has handed it over (after the response).""" + f = control / "inbox" / "upgrade" + for _ in range(100): + if f.exists(): + break + time.sleep(0.02) + return dict(line.split("=", 1) for line in f.read_text().splitlines()) + + +def audit(db): + return [r["detail"] for r in db.rows("SELECT detail FROM audit_log WHERE action = 'lens.install'")] + + +@pytest.mark.parametrize( + ("bind", "public"), + [ + ("0.0.0.0", True), + ("::", True), + ("192.168.1.20", True), # this machine's address on the network + ("127.0.0.1", False), + ("::1", False), + ("[::1]", False), + ("localhost", False), + ("", None), # not started by install.sh or docker-compose.prod.yml: not known + ("lens.lan", None), + ], +) +def test_who_reaches_the_web_app_is_read_from_where_it_listens(monkeypatch, bind, public): + monkeypatch.setenv("LENS_BIND", bind) + monkeypatch.delenv("LENS_TARGET", raising=False) + assert updates.installed() == {"public": public, "target": None} + + +def test_how_it_runs_is_shown_with_the_line_for_each_choice(client, db, monkeypatch): + monkeypatch.delenv("LENS_CONTROL_DIR", raising=False) + monkeypatch.delenv("LENS_BIND", raising=False) + monkeypatch.delenv("LENS_TARGET", raising=False) + h = admin(client, db) + s = client.get("/api/v1/admin/install", headers=h).json() + assert (s["public"], s["target"], s["agent"], s["reconfigure"]) == (None, None, None, None) # unknown, not guessed + m = s["manual"] + assert m["public"].endswith("| LENS_REF=main LENS_PUBLIC=1 sh") and m["private"].endswith("LENS_PUBLIC=0 sh") + assert m["full"].endswith("LENS_TARGET=full sh") and m["lean"].endswith("LENS_TARGET=lean sh") + monkeypatch.setenv("LENS_BIND", "0.0.0.0") + monkeypatch.setenv("LENS_TARGET", "lean") + s = client.get("/api/v1/admin/install", headers=h).json() + assert (s["public"], s["target"]) == (True, "lean") + monkeypatch.setenv("LENS_TARGET", "medium") + assert client.get("/api/v1/admin/install", headers=h).json()["target"] is None + + make_user(db, "ed@x.io", "editor password 1", roles={"pods": "editor"}) + he = login(client, "ed@x.io", "editor password 1") + assert client.get("/api/v1/admin/install", headers=he).status_code == 403 + assert client.put("/api/v1/admin/install", headers=he, json={"public": True}).status_code == 403 + + +def test_the_agent_sets_up_the_version_running_with_the_change(client, db, control): + h = admin(client, db) + s = client.get("/api/v1/admin/install", headers=h).json() + assert (s["public"], s["target"], s["agent"]) == (False, "full", "systemd") + + r = client.put("/api/v1/admin/install", headers=h, json={"public": True}) + assert r.status_code == 202, r.text + req = handed_over(control) + # the version running (empty ref), set up again with LENS_PUBLIC=1 and nothing else + assert (req["ref"], req["public"], req["by"]) == ("", "1", "root@x.io") + assert "target" not in req and "https" not in req + run = client.get("/api/v1/admin/install", headers=h).json()["reconfigure"] + assert (run["public"], run["target"], run["state"], run["id"]) == (True, None, "queued", req["id"]) + assert audit(db) == [{"public": True}] + # it isn't HTTPS's run, nor HTTPS's an install change + assert client.get("/api/v1/settings/https/status", headers=h).json()["reconfigure"] is None + + # one run at a time + r = client.put("/api/v1/admin/install", headers=h, json={"target": "lean"}) + assert r.status_code == 409 and "under way" in r.json()["detail"] + + # the agent says how it went; then the edition, and both at once + (control / "inbox" / "upgrade").unlink() + (control / "upgrade-status").write_text(f"id={req['id']}\nref=main\nstate=done\ncode=0\n") + assert client.get("/api/v1/admin/install", headers=h).json()["reconfigure"]["state"] == "done" + r = client.put("/api/v1/admin/install", headers=h, json={"target": "lean", "public": True}) + assert r.status_code == 202 + req = handed_over(control) + assert (req["ref"], req["public"], req["target"]) == ("", "1", "lean") # Lens still runs as it was started + assert sorted(audit(db), key=len) == [{"public": True}, {"public": True, "target": "lean"}] + + +def test_what_is_so_already_restarts_nothing(client, db, control): + h = admin(client, db) + r = client.put("/api/v1/admin/install", headers=h, json={"public": False, "target": "full"}) + assert r.status_code == 200 and r.json()["reconfigure"] is None + time.sleep(0.1) + assert not (control / "inbox" / "upgrade").exists() and audit(db) == [] + r = client.put("/api/v1/admin/install", headers=h, json={"public": False, "target": "lean"}) # only what differs + assert r.status_code == 202 + req = handed_over(control) + assert req["target"] == "lean" and "public" not in req + + +def test_what_it_takes(client, db, control): + h = admin(client, db) + for body in ({}, {"public": None}, {"target": "medium"}, {"https": True}, {"public": "maybe"}): + assert client.put("/api/v1/admin/install", headers=h, json=body).status_code == 422, body + assert not (control / "inbox" / "upgrade").exists() + with pytest.raises(updates.UpdateError): + updates.reconfigure(db, {}, by="x") # nothing to change + with pytest.raises(updates.UpdateError): + updates.manual_reconfigure({"updates": {}}, target="lean; rm -rf /") + + +def test_no_agent_gives_the_line_to_run(client, db, control, monkeypatch): + monkeypatch.delenv("LENS_CONTROL_DIR") + h = admin(client, db) + r = client.put("/api/v1/admin/install", headers=h, json={"target": "lean"}) + assert r.status_code == 409 + assert "no upgrade agent" in r.json()["detail"] and r.json()["detail"].endswith("LENS_REF=main LENS_TARGET=lean sh") + assert audit(db) == [] + + +@pytest.mark.skipif(not shutil.which("git"), reason="needs git") +def test_the_agent_passes_the_change_to_install_sh(tmp_path): + """packaging/host/lens-upgrade.sh, with an install.sh that writes down what it was given.""" + d = tmp_path / "lens" + d.mkdir() + (d / "install.sh").write_text( + 'echo "ref=$LENS_REF https=$LENS_HTTPS public=${LENS_PUBLIC-unset} target=$LENS_TARGET" > "$LENS_DIR/ran"\n' + ) + (d / ".env").write_text("LENS_HTTPS=0\nLENS_TARGET=full\nLENS_BIND=127.0.0.1\n") + git = ["git", "-C", str(d), "-c", "user.name=t", "-c", "user.email=t@x.io"] + subprocess.run([*git, "init", "-q", "-b", "main"], check=True) + subprocess.run([*git, "add", "."], check=True) + subprocess.run([*git, "commit", "-q", "-m", "x"], check=True) + (d / ".lens-control" / "inbox").mkdir(parents=True) + + def run(request): + (d / ".lens-control" / "inbox" / "upgrade").write_text(request) + subprocess.run(["sh", str(AGENT), str(d)], check=True, capture_output=True) + return (d / "ran").read_text().strip(), (d / ".lens-control" / "upgrade-status").read_text() + + ran, st = run("id=b1\nref=\npublic=1\ntarget=lean\n") + assert ran == "ref=main https=0 public=1 target=lean" and "state=done" in st + # what the request doesn't change is as .env has it: an empty LENS_PUBLIC is install.sh's own fallback to LENS_BIND + ran, _ = run("id=b2\nref=\nhttps=1\n") + assert ran == "ref=main https=1 public= target=full" + ran, _ = run("id=b3\nref=\npublic=0\n") + assert ran == "ref=main https=0 public=0 target=full" + ran, _ = run("id=b4\nref=\npublic=yes; rm -rf /\ntarget=../../x\n") # nothing but 0 or 1, full or lean + assert ran == "ref=main https=0 public= target=full" diff --git a/fastapi_backend/tests/api/test_mcp.py b/fastapi_backend/tests/api/test_mcp.py index 8f853d9a3..0e34bac9e 100644 --- a/fastapi_backend/tests/api/test_mcp.py +++ b/fastapi_backend/tests/api/test_mcp.py @@ -81,6 +81,7 @@ def test_clients_are_sent_to_sign_in(client, app, env): "mark_nsfw", "set_file_storage", "set_monitoring", + "set_install_settings", ] assert next(t for t in tools if t["name"] == "search")["inputSchema"]["required"] == ["query"] # API tokens and sessions work too @@ -472,6 +473,33 @@ def test_monitoring_tools(client, db, env, monkeypatch): assert "no upgrade agent" in tool_error(client, ha, "set_monitoring", on=True) +def test_install_tools(client, db, env, tmp_path, monkeypatch): + monkeypatch.delenv("LENS_CONTROL_DIR", raising=False) + monkeypatch.setenv("LENS_BIND", "127.0.0.1") + monkeypatch.setenv("LENS_TARGET", "full") + assert "only admins" in tool_error(client, env["h"], "install_settings") + assert "only admins" in tool_error(client, env["h"], "set_install_settings", public=True) + make_user(db, "root@x.io", "root password 1", admin=True) + ha = login(client, "root@x.io", "root password 1") + got = tool(client, ha, "install_settings") + assert (got["public"], got["target"], got["agent"]) == (False, "full", None) and got["settings"].endswith("/settings/access") + assert got["manual"]["public"].endswith("LENS_PUBLIC=1 sh") + err = tool_error(client, ha, "set_install_settings", public=True) + assert "no upgrade agent" in err and err.endswith("LENS_PUBLIC=1 sh") # the line to run instead + assert "say what to change" in tool_error(client, ha, "set_install_settings") + assert "full, lean" in tool_error(client, ha, "set_install_settings", target="medium") + + d = tmp_path / "control" + (d / "inbox").mkdir(parents=True) + (d / "agent").write_text("kind=systemd\n") + monkeypatch.setenv("LENS_CONTROL_DIR", str(d)) + assert tool(client, ha, "set_install_settings", public=False)["changing"] == {} # so already: nothing restarts + got = tool(client, ha, "set_install_settings", target="lean") + assert got["changing"] == {"target": "lean"} and got["reconfigure"]["target"] == "lean" + detail = db.values("SELECT VALUE detail FROM audit_log WHERE action = 'lens.install'") + assert detail == [{"target": "lean", "via": "mcp"}] + + def test_what_the_archive_says_reaches_agents_as_data(client, db, cfg, env): said = f"Mallory: {injections.PLAIN}\nMallory: {injections.CHATML}\nMallory: {injections.SMUGGLED}" rid = ingest.import_text(db, cfg, "pods", said, title="Mallory's talk") diff --git a/install.sh b/install.sh index e7eb8ae92..fd1e169a9 100755 --- a/install.sh +++ b/install.sh @@ -11,10 +11,10 @@ # LENS_DIR where Lens goes (default: ~/lens) # LENS_REF the branch or tag to run (default: main) # LENS_PORT the web app's port (default: 3000, or the next free one) -# LENS_PUBLIC 1: reachable from other machines on the network, 0: this machine only (default: 1 on a server -# without a desktop, else 0) +# LENS_PUBLIC 1: reachable from other machines on the network, 0: this machine only (default: as the last run +# left it, else 1 on a server without a desktop, else 0). Also in Settings › Access & embedding # LENS_TARGET full (reads Office files, web pages and emails too) or lean (smaller) (default: as the last run -# left it, else full) +# left it, else full). Also in Settings › Components # LENS_BACKEND docker or microsandbox (one microVM, no Docker: docs/microsandbox.md). Asked on a first install # when microsandbox can run here (Linux with KVM, or a Mac with Apple silicon); default: docker # LENS_DNS_FALLBACK in a microsandbox: the public resolvers it looks names up with when this machine's don't diff --git a/nextjs-frontend/__tests__/install-settings.test.tsx b/nextjs-frontend/__tests__/install-settings.test.tsx new file mode 100644 index 000000000..c9e790bb6 --- /dev/null +++ b/nextjs-frontend/__tests__/install-settings.test.tsx @@ -0,0 +1,226 @@ +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import "@testing-library/jest-dom"; +import type { ReactNode } from "react"; + +import { Updates } from "@/app/openapi-client"; +import type { InstallStatus } from "@/app/openapi-client/types.gen"; +import { + EditionChoice, + going, + manualLine, + NetworkReach, + onTheNetwork, + runWhat, +} from "@/components/settings/install-settings"; +import type { SettingsView } from "@/components/settings/model"; +import { SettingsSection } from "@/components/settings/section"; +import { TooltipProvider } from "@/components/ui/tooltip"; + +jest.mock("@/app/openapi-client", () => ({ + Updates: { getInstall: jest.fn(), setInstall: jest.fn() }, + Admin: { updateSettings: jest.fn() }, + Metadata: {}, +})); +jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) })); +jest.mock("next/navigation", () => ({ useRouter: () => ({ push: jest.fn() }) })); +jest.mock("@/lib/hooks/session", () => ({ useArchive: () => ({ namespaces: [] }) })); +jest.mock("@/components/ui/toast", () => ({ useToast: () => jest.fn() })); + +const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } }); +const m = (f: unknown) => f as jest.Mock; +const LINE = "curl -fsSL https://raw.githubusercontent.com/adeelahmad/lense/main/install.sh | LENS_REF=main"; +const STATUS: InstallStatus = { + public: false, + target: "full", + agent: "systemd", + manual: { + public: `${LINE} LENS_PUBLIC=1 sh`, + private: `${LINE} LENS_PUBLIC=0 sh`, + full: `${LINE} LENS_TARGET=full sh`, + lean: `${LINE} LENS_TARGET=lean sh`, + }, + reconfigure: null, +}; + +function show(ui: ReactNode) { + const qc = new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } } }); + return render( + + {ui} + , + ); +} + +const reach = () => screen.findByRole("switch", { name: "Reachable from other devices on this network" }); + +describe("Settings › Access & embedding: reachable from the network", () => { + beforeEach(() => { + m(Updates.getInstall) + .mockReset() + .mockImplementation(() => ok(STATUS)); + m(Updates.setInstall).mockReset(); + }); + + it("says what opening it does, and opens it with one button through the upgrade agent", async () => { + m(Updates.setInstall).mockImplementation(() => + ok({ ...STATUS, reconfigure: { id: "r1", ref: "", public: true, state: "queued", log: [] } }), + ); + show(); + expect(await reach()).not.toBeChecked(); + expect(screen.getByText(/Only this machine opens Lens/)).toBeInTheDocument(); + expect(screen.queryByRole("button")).not.toBeInTheDocument(); // nothing to apply until it's changed + fireEvent.click(await reach()); + expect(screen.getByText("Lens restarts to listen on the network.")).toBeInTheDocument(); + fireEvent.click(screen.getByRole("button", { name: "Open to the network and restart" })); + expect(await screen.findByText("Opening Lens to the network.")).toBeInTheDocument(); + expect(screen.getByText("Waiting for the machine to pick it up.")).toBeInTheDocument(); + expect(m(Updates.setInstall).mock.calls[0][0]).toMatchObject({ body: { public: true } }); + expect(await reach()).toBeDisabled(); // one change at a time + }); + + it("switching it back leaves nothing to apply", async () => { + show(); + fireEvent.click(await reach()); + fireEvent.click(await reach()); + expect(screen.queryByText(/Lens restarts/)).not.toBeInTheDocument(); + expect(Updates.setInstall).not.toHaveBeenCalled(); + }); + + it("gives the line to run when the machine has no upgrade agent", async () => { + m(Updates.getInstall).mockImplementation(() => ok({ ...STATUS, public: true, agent: null })); + show(); + expect(await reach()).toBeChecked(); + fireEvent.click(await reach()); + expect(screen.getByText("Lens restarts to listen on this machine only.")).toBeInTheDocument(); + expect(screen.getByText(`${LINE} LENS_PUBLIC=0 sh`)).toBeInTheDocument(); + expect(screen.queryByRole("button")).not.toBeInTheDocument(); + }); + + it("says when it can't tell, rather than guessing", async () => { + m(Updates.getInstall).mockImplementation(() => ok({ ...STATUS, public: null, target: null })); + show( + <> + + + , + ); + expect(await screen.findAllByText(/Lens can’t tell/)).toHaveLength(2); + expect(screen.getByRole("radio", { name: /Full/ })).toHaveAttribute("aria-checked", "false"); + expect(screen.getByRole("radio", { name: /Lean/ })).toHaveAttribute("aria-checked", "false"); + }); + + it("shows why the machine couldn't change it", async () => { + m(Updates.getInstall).mockImplementation(() => + ok({ + ...STATUS, + reconfigure: { + id: "r1", + ref: "", + public: true, + state: "failed", + error: "install.sh stopped (exit 1)", + log: ["x"], + }, + }), + ); + show(); + expect(await screen.findByText("The machine couldn’t change it.")).toBeInTheDocument(); + expect(screen.getByText("install.sh stopped (exit 1)")).toBeInTheDocument(); + expect(await reach()).toBeEnabled(); + }); + + it("shows the switch in the Access & embedding section", async () => { + const view = { + server: { + values: { + allowed_hosts: ["localhost"], + embed_frame_ancestors: [], + trusted_proxies: [], + session_hours: 12, + secure_cookies: false, + max_upload_mb: 500, + }, + }, + } as unknown as SettingsView; + show( undefined} />); + expect(await reach()).toBeInTheDocument(); + }); +}); + +describe("Settings › Components: the edition", () => { + beforeEach(() => { + m(Updates.getInstall) + .mockReset() + .mockImplementation(() => ok(STATUS)); + m(Updates.setInstall).mockReset(); + }); + + it("switches to lean with one button, after saying what it means", async () => { + m(Updates.setInstall).mockImplementation(() => + ok({ ...STATUS, reconfigure: { id: "r2", ref: "", target: "lean", state: "running", log: ["Building…"] } }), + ); + show(); + expect(await screen.findByRole("radio", { name: /Full/ })).toHaveAttribute("aria-checked", "true"); + fireEvent.click(screen.getByRole("radio", { name: /Lean/ })); + expect(screen.getByText("Lens restarts as the lean edition.")).toBeInTheDocument(); + expect(screen.getByText(/other documents then need anytopdf or Tika/)).toBeInTheDocument(); + fireEvent.click(screen.getByRole("button", { name: "Switch to lean and restart" })); + expect(await screen.findByText("Switching to the lean edition.")).toBeInTheDocument(); + expect(screen.getByText("Building…")).toBeInTheDocument(); + await waitFor(() => expect(Updates.setInstall).toHaveBeenCalledTimes(1)); + expect(m(Updates.setInstall).mock.calls[0][0]).toMatchObject({ body: { target: "lean" } }); + }); + + it("gives the line to run without an agent", async () => { + m(Updates.getInstall).mockImplementation(() => ok({ ...STATUS, target: "lean", agent: null })); + show(); + fireEvent.click(await screen.findByRole("radio", { name: /Full/ })); + expect(screen.getByText("Lens restarts as the full edition.")).toBeInTheDocument(); + expect(screen.getByText(`${LINE} LENS_TARGET=full sh`)).toBeInTheDocument(); + }); + + it("shows the API's reason when the machine is busy", async () => { + m(Updates.setInstall).mockImplementation(() => + Promise.resolve({ + error: { detail: "an upgrade is already under way; try again when it's done" }, + response: { ok: false, status: 409 }, + }), + ); + show(); + fireEvent.click(await screen.findByRole("radio", { name: /Lean/ })); + fireEvent.click(screen.getByRole("button", { name: "Switch to lean and restart" })); + expect(await screen.findByText(/already under way/)).toBeInTheDocument(); + }); +}); + +describe("the helpers", () => { + it("know a change that's still going", () => { + expect(going({ id: "a", ref: "", state: "queued" })).toBe(true); + expect(going({ id: "a", ref: "", state: "running" })).toBe(true); + expect(going({ id: "a", ref: "", state: "queued", error: "not picked up" })).toBe(false); + expect(going({ id: "a", ref: "", state: "done" })).toBe(false); + expect(going(null)).toBe(false); + }); + + it("pick the line for each change", () => { + expect(manualLine(STATUS, { public: true })).toMatch(/LENS_PUBLIC=1 sh$/); + expect(manualLine(STATUS, { public: false })).toMatch(/LENS_PUBLIC=0 sh$/); + expect(manualLine(STATUS, { target: "lean" })).toMatch(/LENS_TARGET=lean sh$/); + }); + + it("tell an address on the network from this machine's own", () => { + expect(onTheNetwork("192.168.1.20")).toBe(true); + expect(onTheNetwork("lens.lan")).toBe(true); + expect(onTheNetwork("localhost")).toBe(false); + expect(onTheNetwork("127.0.0.1")).toBe(false); + expect(onTheNetwork("[::1]")).toBe(false); + }); + + it("say what a run changes", () => { + expect(runWhat({ public: false })).toBe("Keeping Lens to this machine"); + expect(runWhat({ public: true, target: "full" })).toBe( + "Opening Lens to the network and switching to the full edition", + ); + }); +}); diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index 84f5d7bd8..8052739bb 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -1062,6 +1062,9 @@ export type { GetHealthData, GetHealthResponse, GetHealthResponses, + GetInstallData, + GetInstallResponse, + GetInstallResponses, GetJobData, GetJobError, GetJobErrors, @@ -1438,11 +1441,14 @@ export type { IndexSemanticErrors, IndexSemanticResponse, IndexSemanticResponses, + InstallChange, + InstallLines, InstallMarketplacePluginData, InstallMarketplacePluginError, InstallMarketplacePluginErrors, InstallMarketplacePluginResponse, InstallMarketplacePluginResponses, + InstallStatus, IpGroup, IpGroupCreate, IpGroups, @@ -2646,6 +2652,11 @@ export type { SetCollectionMemberErrors, SetCollectionMemberResponse, SetCollectionMemberResponses, + SetInstallData, + SetInstallError, + SetInstallErrors, + SetInstallResponse, + SetInstallResponses, SetMemberData, SetMemberError, SetMemberErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index f1a99b4d7..120c5e0da 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -514,6 +514,8 @@ import type { GetGraphResponses, GetHealthData, GetHealthResponses, + GetInstallData, + GetInstallResponses, GetJobData, GetJobErrors, GetJobLogData, @@ -1296,6 +1298,9 @@ import type { SetCollectionMemberData, SetCollectionMemberErrors, SetCollectionMemberResponses, + SetInstallData, + SetInstallErrors, + SetInstallResponses, SetMemberData, SetMemberErrors, SetMemberResponses, @@ -3365,6 +3370,43 @@ export class Updates { }); } + /** + * Get Install + * + * How install.sh set this machine up: whether other machines on the network reach the web app, and the edition + * (full or lean); what can change them here, and how the last change went. + */ + public static getInstall( + options?: Options, + ): RequestResult { + return (options?.client ?? client).get({ + url: "/api/v1/admin/install", + ...options, + }); + } + + /** + * Set Install + * + * Make the web app reachable from other machines on the network (public) or this machine only, or change the + * edition (target: full, with LibreOffice and Chromium, or lean): this machine's upgrade agent runs install.sh again + * for the version running with LENS_PUBLIC and LENS_TARGET, and Lens restarts; GET /admin/install follows along. + * What is so already is left alone, and with nothing left to change nothing restarts (200). Without an agent, 409 + * says the line to run on the machine instead (`manual` has each one). + */ + public static setInstall( + options: Options, + ): RequestResult { + return (options.client ?? client).put({ + url: "/api/v1/admin/install", + ...options, + headers: { + "Content-Type": "application/json", + ...options.headers, + }, + }); + } + /** * Get Monitoring * diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 19d230a3b..915607802 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -6880,6 +6880,88 @@ export type IndexQueued = { [key: string]: unknown; }; +/** + * InstallChange + */ +export type InstallChange = { + /** + * Public + * + * reachable from other machines on the network (true) or this machine only (false) + */ + public?: boolean | null; + /** + * Target + * + * the edition: full or lean + */ + target?: "full" | "lean" | null; +}; + +/** + * InstallLines + */ +export type InstallLines = { + /** + * Public + * + * the install line that makes the web app reachable from other machines (LENS_PUBLIC=1) + */ + public: string; + /** + * Private + * + * the install line that keeps it to this machine (LENS_PUBLIC=0) + */ + private: string; + /** + * Full + * + * the install line for the full edition (LENS_TARGET=full) + */ + full: string; + /** + * Lean + * + * the install line for the lean edition (LENS_TARGET=lean) + */ + lean: string; + [key: string]: unknown; +}; + +/** + * InstallStatus + */ +export type InstallStatus = { + /** + * Public + * + * the web app is reachable from other machines on the network (LENS_BIND), or from this one only; none: Lens wasn't started by install.sh or docker-compose.prod.yml, so it doesn't know + */ + public?: boolean | null; + /** + * Target + * + * the edition (LENS_TARGET): full has LibreOffice and Chromium, to read Office files, web pages and emails; lean doesn't. None: not known here + */ + target?: "full" | "lean" | null; + /** + * Agent + * + * what changes them on this machine (systemd, launchd); none: run a `manual` line there + */ + agent?: string | null; + /** + * the line to run on the machine Lens runs on for each choice, when there's no agent + */ + manual: InstallLines; + /** + * the last run that changed either (as in /admin/updates) + */ + reconfigure?: UpgradeRun | null; + [key: string]: unknown; +}; + /** * IpGroup * @@ -18561,6 +18643,18 @@ export type UpgradeRun = { * the HTTPS ports opened (true) or closed (false) by this run (docs/https.md) */ https?: boolean | null; + /** + * Public + * + * the web app made reachable from other machines on the network (true) or this one only (false) by this run + */ + public?: boolean | null; + /** + * Target + * + * the edition this run changed to + */ + target?: "full" | "lean" | null; /** * State * @@ -22661,6 +22755,47 @@ export type StartUpgradeResponses = { export type StartUpgradeResponse = StartUpgradeResponses[keyof StartUpgradeResponses]; +export type GetInstallData = { + body?: never; + path?: never; + query?: never; + url: "/api/v1/admin/install"; +}; + +export type GetInstallResponses = { + /** + * Successful Response + */ + 200: InstallStatus; +}; + +export type GetInstallResponse = GetInstallResponses[keyof GetInstallResponses]; + +export type SetInstallData = { + body: InstallChange; + path?: never; + query?: never; + url: "/api/v1/admin/install"; +}; + +export type SetInstallErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type SetInstallError = SetInstallErrors[keyof SetInstallErrors]; + +export type SetInstallResponses = { + /** + * Successful Response + */ + 202: InstallStatus; +}; + +export type SetInstallResponse = SetInstallResponses[keyof SetInstallResponses]; + export type GetMonitoringData = { body?: never; path?: never; diff --git a/nextjs-frontend/components/settings/bodies.tsx b/nextjs-frontend/components/settings/bodies.tsx index 5b771c7a8..56339af61 100644 --- a/nextjs-frontend/components/settings/bodies.tsx +++ b/nextjs-frontend/components/settings/bodies.tsx @@ -10,6 +10,7 @@ import type { HttpsStatus, MatterbridgeStatus } from "@/app/openapi-client/types import { MonitoringSwitch } from "@/components/admin/monitoring-switch"; import { BackupsPanel } from "@/components/settings/backups-panel"; import { ComponentsStatus } from "@/components/settings/components-status"; +import { EditionChoice, NetworkReach } from "@/components/settings/install-settings"; import { LogViewer } from "@/components/settings/log-viewer"; import { AllTokens } from "@/components/account/all-tokens"; import { ACCESS } from "@/components/iiif/metadata-model"; @@ -351,6 +352,7 @@ export function SectionBody({ ctx }: { ctx: BodyCtx }) { case "components": return ( <> + @@ -359,6 +361,7 @@ export function SectionBody({ ctx }: { ctx: BodyCtx }) { case "access": return ( <> + diff --git a/nextjs-frontend/components/settings/install-settings.tsx b/nextjs-frontend/components/settings/install-settings.tsx new file mode 100644 index 000000000..567763d25 --- /dev/null +++ b/nextjs-frontend/components/settings/install-settings.tsx @@ -0,0 +1,260 @@ +"use client"; + +import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; +import { useState, type ReactNode } from "react"; + +import { Updates } from "@/app/openapi-client"; +import type { InstallStatus, UpgradeRun } from "@/app/openapi-client/types.gen"; +import { ChoiceCards } from "@/components/settings/controls"; +import { Banner } from "@/components/ui/banner"; +import { Button } from "@/components/ui/button"; +import { Switch } from "@/components/ui/field"; +import { Skeleton } from "@/components/ui/states"; +import { data, useApiClient } from "@/lib/api/browser"; + +// How install.sh set this machine up, changed from Settings: the machine's upgrade agent runs install.sh again for the +// version running with the change, and Lens restarts (docs/updates.md). + +export type Edition = "full" | "lean"; +export type InstallChange = { public: boolean } | { target: Edition }; + +const KEY = ["install"]; +const REACH = "Reachable from other devices on this network"; +const UNKNOWN = "Lens can’t tell: it wasn’t started by install.sh or Docker Compose."; +const EDITIONS: { value: Edition; label: string; hint: string }[] = [ + { value: "full", label: "Full", hint: "Reads Office files, web pages and emails too, with LibreOffice and Chromium" }, + { value: "lean", label: "Lean", hint: "Smaller, without LibreOffice and Chromium, for a small machine" }, +]; + +/** A change the machine is still making: waiting for its upgrade agent, or install.sh running. */ +export function going(run: UpgradeRun | null | undefined): boolean { + return Boolean(run && !run.error && (run.state === "queued" || run.state === "running")); +} + +/** The install line that makes a change by hand, for a machine with no upgrade agent. */ +export function manualLine(s: Pick, change: InstallChange): string { + if ("public" in change) return change.public ? s.manual.public : s.manual.private; + return s.manual[change.target]; +} + +/** Whether this page was opened at an address other than this machine's own name, so maybe over the network. */ +export function onTheNetwork(hostname: string): boolean { + const h = hostname.replace(/^\[|\]$/g, "").toLowerCase(); + return !(h === "" || h === "localhost" || h.endsWith(".localhost") || h === "::1" || /^127\./.test(h)); +} + +/** What a run changes, in a few words. */ +export function runWhat(run: Pick): string { + const parts: string[] = []; + if (run.public != null) parts.push(run.public ? "opening Lens to the network" : "keeping Lens to this machine"); + if (run.target) parts.push(`switching to the ${run.target} edition`); + const what = parts.join(" and "); + return what.charAt(0).toUpperCase() + what.slice(1); +} + +/** Where the last run is: waiting for the machine, setting Lens up again, or why it stopped. */ +function RunState({ run }: { run: UpgradeRun }) { + const log = (run.log ?? []).length > 0 && ( +
+      {(run.log ?? []).join("\n")}
+    
+ ); + if (going(run)) + return ( + + {run.state === "queued" + ? "Waiting for the machine to pick it up." + : "Lens is setting itself up again and restarts; this page follows when it’s back."} + {log} + + ); + if (run.state === "failed" || run.error) + return ( + + {run.error} + {log} + + ); + return null; +} + +/** The status, followed while the machine changes it (Lens is away while it restarts: the polling carries on). */ +export function useInstall() { + const client = useApiClient(); + return useQuery({ + queryKey: KEY, + queryFn: () => data(Updates.getInstall({ client })), + refetchInterval: (q) => (going(q.state.data?.reconfigure) ? 3000 : false), + }); +} + +/** What a change does, then the one button that asks the machine for it; with no agent, the line to run there. */ +function Apply({ + s, + change, + title, + action, + onDone, + children, +}: { + s: InstallStatus; + change: InstallChange; + title: string; + action: string; + onDone: () => void; + children: ReactNode; +}) { + const client = useApiClient(); + const qc = useQueryClient(); + const apply = useMutation({ + mutationFn: () => data(Updates.setInstall({ client, body: change })), + onSuccess: (got) => { + qc.setQueryData(KEY, got); + onDone(); + }, + }); + return ( + + {children} + {s.agent ? ( + + + + ) : ( + <> + + This machine has no upgrade agent to do it (Lens wasn’t installed with install.sh, or it couldn’t set one + up). Run this on the machine Lens runs on: + +
+            {manualLine(s, change)}
+          
+ + )} + {apply.isError && {apply.error.message}} +
+ ); +} + +/** Settings › Access & embedding: whether other devices on the network reach the web app, or this machine only. */ +export function NetworkReach() { + const q = useInstall(); + const [want, setWant] = useState(null); + if (q.isPending) return ; + if (!q.data) return {q.error?.message}; + const s = q.data; + const run = s.reconfigure?.public != null ? s.reconfigure : null; + const busy = going(s.reconfigure); + const host = typeof window === "undefined" ? "" : window.location.hostname; + const now = + s.public == null + ? UNKNOWN + : s.public + ? "Other devices here open Lens at this machine’s address. Passkeys need https:// there (Settings › HTTPS); until then people sign in on them with a sign-in link." + : "Only this machine opens Lens. HTTPS and Remote access, when on, reach it either way."; + return ( +
+
+
+ {REACH} + {now} +
+ + setWant(v === s.public ? null : v)} + /> + +
+ {want !== null && + !busy && + (want ? ( + setWant(null)} + > + Other devices on this network can then open it, and its sign-in page. Sensors, file shares and Grafana, when + they’re on, listen there too. + + ) : ( + setWant(null)} + > + Other devices on this network can’t open it then. + {onTheNetwork(host) && ( + <> + {" "} + You opened this page at {host}: if that’s this machine’s address on the + network, this browser can’t reach Lens there after the restart. + + )} + + ))} + {run && } +
+ ); +} + +/** Settings › Components: the edition, full (LibreOffice and Chromium) or lean. */ +export function EditionChoice() { + const q = useInstall(); + const [want, setWant] = useState(null); + if (q.isPending) return ; + if (!q.data) return {q.error?.message}; + const s = q.data; + const run = s.reconfigure?.target ? s.reconfigure : null; + const busy = going(s.reconfigure); + return ( +
+ Edition + setWant(v === s.target ? null : (v as Edition))} + disabled={busy} + columns={2} + size="sm" + className="max-sm:!grid-cols-1" + /> + {s.target == null && {UNKNOWN}} + {want !== null && + !busy && + (want === "lean" ? ( + setWant(null)} + > + It’s set up again without LibreOffice and Chromium, which can take a few minutes. PDFs and images are read + as before; other documents then need anytopdf or Tika (Settings › Documents). + + ) : ( + setWant(null)} + > + It’s set up again with LibreOffice and Chromium, to read Office files, web pages and emails too. That takes + more disk, and can take a few minutes. + + ))} + {run && } +
+ ); +} diff --git a/nextjs-frontend/components/settings/model.ts b/nextjs-frontend/components/settings/model.ts index 120fa1a1a..033605db4 100644 --- a/nextjs-frontend/components/settings/model.ts +++ b/nextjs-frontend/components/settings/model.ts @@ -220,14 +220,14 @@ export const SECTIONS: SectionSpec[] = [ label: "Components", backend: ["components"], description: - "The engines and models Lens fetches for itself, sized to each worker’s machine. Steps that need one wait while it arrives.", + "The edition, full or lean, and the engines and models Lens fetches for itself, sized to each worker’s machine. Steps that need one wait while it arrives.", }, { id: "access", label: "Access & embedding", backend: ["server"], description: - "Who can reach the server, how it tells visitors’ addresses, which sites may embed the player, and how long sessions last.", + "Who can reach the server, from this machine or the network, how it tells visitors’ addresses, which sites may embed the player, and how long sessions last.", }, { id: "remote-access", diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index b9978a9cd..7299631df 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -4064,6 +4064,68 @@ } } }, + "/api/v1/admin/install": { + "get": { + "tags": [ + "updates" + ], + "summary": "Get Install", + "description": "How install.sh set this machine up: whether other machines on the network reach the web app, and the edition\n(full or lean); what can change them here, and how the last change went.", + "operationId": "get_install", + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/InstallStatus" + } + } + } + } + } + }, + "put": { + "tags": [ + "updates" + ], + "summary": "Set Install", + "description": "Make the web app reachable from other machines on the network (public) or this machine only, or change the\nedition (target: full, with LibreOffice and Chromium, or lean): this machine's upgrade agent runs install.sh again\nfor the version running with LENS_PUBLIC and LENS_TARGET, and Lens restarts; GET /admin/install follows along.\nWhat is so already is left alone, and with nothing left to change nothing restarts (200). Without an agent, 409\nsays the line to run on the machine instead (`manual` has each one).", + "operationId": "set_install", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/InstallChange" + } + } + }, + "required": true + }, + "responses": { + "202": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/InstallStatus" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/admin/monitoring": { "get": { "tags": [ @@ -38988,6 +39050,139 @@ ], "title": "IndexQueued" }, + "InstallChange": { + "properties": { + "public": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Public", + "description": "reachable from other machines on the network (true) or this machine only (false)" + }, + "target": { + "anyOf": [ + { + "type": "string", + "enum": [ + "full", + "lean" + ] + }, + { + "type": "null" + } + ], + "title": "Target", + "description": "the edition: full or lean" + } + }, + "additionalProperties": false, + "type": "object", + "title": "InstallChange" + }, + "InstallLines": { + "properties": { + "public": { + "type": "string", + "title": "Public", + "description": "the install line that makes the web app reachable from other machines (LENS_PUBLIC=1)" + }, + "private": { + "type": "string", + "title": "Private", + "description": "the install line that keeps it to this machine (LENS_PUBLIC=0)" + }, + "full": { + "type": "string", + "title": "Full", + "description": "the install line for the full edition (LENS_TARGET=full)" + }, + "lean": { + "type": "string", + "title": "Lean", + "description": "the install line for the lean edition (LENS_TARGET=lean)" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "public", + "private", + "full", + "lean" + ], + "title": "InstallLines" + }, + "InstallStatus": { + "properties": { + "public": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Public", + "description": "the web app is reachable from other machines on the network (LENS_BIND), or from this one only; none: Lens wasn't started by install.sh or docker-compose.prod.yml, so it doesn't know" + }, + "target": { + "anyOf": [ + { + "type": "string", + "enum": [ + "full", + "lean" + ] + }, + { + "type": "null" + } + ], + "title": "Target", + "description": "the edition (LENS_TARGET): full has LibreOffice and Chromium, to read Office files, web pages and emails; lean doesn't. None: not known here" + }, + "agent": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Agent", + "description": "what changes them on this machine (systemd, launchd); none: run a `manual` line there" + }, + "manual": { + "$ref": "#/components/schemas/InstallLines", + "description": "the line to run on the machine Lens runs on for each choice, when there's no agent" + }, + "reconfigure": { + "anyOf": [ + { + "$ref": "#/components/schemas/UpgradeRun" + }, + { + "type": "null" + } + ], + "description": "the last run that changed either (as in /admin/updates)" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "manual" + ], + "title": "InstallStatus" + }, "IpGroup": { "properties": { "id": { @@ -59102,6 +59297,34 @@ "title": "Https", "description": "the HTTPS ports opened (true) or closed (false) by this run (docs/https.md)" }, + "public": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Public", + "description": "the web app made reachable from other machines on the network (true) or this one only (false) by this run" + }, + "target": { + "anyOf": [ + { + "type": "string", + "enum": [ + "full", + "lean" + ] + }, + { + "type": "null" + } + ], + "title": "Target", + "description": "the edition this run changed to" + }, "state": { "type": "string", "enum": [ diff --git a/packaging/host/lens-upgrade.sh b/packaging/host/lens-upgrade.sh index ad9161746..3f1220bb7 100755 --- a/packaging/host/lens-upgrade.sh +++ b/packaging/host/lens-upgrade.sh @@ -8,8 +8,9 @@ # # It takes a release tag (vX.Y.Z), or the branch this install already follows; nothing else, whoever wrote the # request. An empty ref sets up the version running again, as Settings › HTTPS does with https=1 or 0 to open or close -# the HTTPS ports (docs/https.md). If the new version doesn't start, it goes back to the code that ran before and -# starts that (rolled-back). Run by hand: sh packaging/host/lens-upgrade.sh +# the HTTPS ports (docs/https.md), and Settings with public=1 or 0 (reachable from other machines on the network, or +# this one only) and target=full or lean (the edition). If the new version doesn't start, it goes back to the code that +# ran before and starts that (rolled-back). Run by hand: sh packaging/host/lens-upgrade.sh # # A request with action=services turns the monitoring containers on or off instead (docs/prometheus.md): it adds or # removes `monitoring` in COMPOSE_PROFILES in .env and starts or stops Prometheus, node-exporter and Grafana, and says @@ -91,6 +92,8 @@ backup="$(val backup "$REQ")" action="$(val action "$REQ")" monitoring="$(val monitoring "$REQ")" https="$(val https "$REQ")" +public="$(val public "$REQ")" +target="$(val target "$REQ")" rm -f "$REQ" now() { date -u +%Y-%m-%dT%H:%M:%SZ; } env_of() { [ -f "$DIR/.env" ] && sed -n "s/^$1=//p" "$DIR/.env" | head -1; } @@ -119,6 +122,8 @@ if [ -z "$ref" ]; then # the version running: this install's branch, or the rele [ -n "$ref" ] || ref="$(git -C "$DIR" describe --tags --exact-match 2>/dev/null || true)" fi case "$https" in 0 | 1 | "") ;; *) https="" ;; esac +case "$public" in 0 | 1 | "") ;; *) public="" ;; esac +case "$target" in full | lean | "") ;; *) target="" ;; esac if ! printf '%s' "$ref" | grep -Eq '^v[0-9]+\.[0-9]+\.[0-9]+$' && [ "$ref" != "$branch" ]; then echo "Not upgrading to '$ref': only a release tag (vX.Y.Z) or this install's branch ($branch)." >>"$CTL/upgrade.log" status failed "finished_at=$(now)" "code=2" "error=only a release tag (vX.Y.Z) or this install's branch can be installed" @@ -126,16 +131,17 @@ if ! printf '%s' "$ref" | grep -Eq '^v[0-9]+\.[0-9]+\.[0-9]+$' && [ "$ref" != "$ fi status running "step=download" -# install.sh with the settings this install was made with, its lines in the log; the step it is on goes into the -# status as it goes. Its exit code, after. +# install.sh with the settings this install was made with, but for what the request changes (an empty LENS_PUBLIC +# leaves the address as .env has it), its lines in the log; the step it is on goes into the status as it goes. Its +# exit code, after. install() { # extra environment, as NAME=value words rm -f "$CTL/.code" { ( cd "$DIR" && env LENS_DIR="$DIR" LENS_REF="$ref" LENS_NO_OPEN=1 LENS_YES=1 LENS_WIZARD=0 LENS_AGENT_RUN=1 \ - LENS_REPO="$(env_of LENS_REPO)" LENS_TARGET="$(env_of LENS_TARGET)" LENS_BACKEND="$(env_of LENS_BACKEND)" \ - LENS_HTTPS="${https:-$(env_of LENS_HTTPS)}" \ + LENS_REPO="$(env_of LENS_REPO)" LENS_TARGET="${target:-$(env_of LENS_TARGET)}" LENS_BACKEND="$(env_of LENS_BACKEND)" \ + LENS_HTTPS="${https:-$(env_of LENS_HTTPS)}" LENS_PUBLIC="$public" \ GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=safe.directory GIT_CONFIG_VALUE_0="$DIR" "$@" \ sh "$DIR/install.sh" ) &1 || echo "$?" >"$CTL/.code" diff --git a/threat-model.json b/threat-model.json index 8a718c0cd..735be68f9 100644 --- a/threat-model.json +++ b/threat-model.json @@ -51,6 +51,7 @@ "/iiif/auth/token, /probe, /logout", "workers and DB peers", "host control inbox and the monitoring scrape token", + "admin machine setup requests", "microsandbox program requests" ], "touches": [ @@ -502,13 +503,22 @@ }, { "entry_point": "host control inbox and the monitoring scrape token", - "parameter": "inbox/upgrade request (release ref, or action=services with monitoring=on|off); /metrics scrape token and .lens-control/prometheus-token", + "parameter": "inbox/upgrade request (release ref, empty for the version running, with https=0|1, public=0|1 and target=full|lean; or action=services with monitoring=on|off); /metrics scrape token and .lens-control/prometheus-token", "attacker_controllable": "yes", "component": "authz", - "caller_must_enforce": "the agent installs only a release tag or the install's branch and only toggles the monitoring profile; the token is compared in constant time and grants metrics only", + "caller_must_enforce": "the agent installs only a release tag or the install's branch, changes only the HTTPS ports, the web app's network reach and the edition (other values dropped), and only toggles the monitoring profile, so a writer can at most restart Lens with those choices or open it to the network; the token is compared in constant time and grants metrics only", "provenance": "documented", "source": "docs/updates.md, docs/prometheus.md" }, + { + "entry_point": "admin machine setup requests", + "parameter": "PUT /api/v1/admin/install {public, target}, POST /api/v1/settings/https/ports {open}, MCP set_install_settings", + "attacker_controllable": "no", + "component": "authz", + "caller_must_enforce": "admin is trusted; audited (lens.install, https.ports); reaches the machine only as an inbox request the agent checks again", + "provenance": "documented", + "source": "docs/updates.md \"Setting the machine up again\"" + }, { "entry_point": "microsandbox program requests", "parameter": "ask/NAME.want in /data/.lens/programs: a program's name", diff --git a/threat-model.md b/threat-model.md index 836074a99..c40770f6e 100644 --- a/threat-model.md +++ b/threat-model.md @@ -242,7 +242,9 @@ machine requires HTTPS, and a changed DB credential when SurrealDB runs as its o | `sensors.enabled`, `sensors.mqtt_anonymous` | off (`bind` `0.0.0.0`); off | opens MQTT and syslog listeners; anonymous MQTT publish | Yes; anonymous is the admin's choice | *(documented, docs/sensors.md "Sensors are off until an admin turns them on")* | | `shares.enabled` | off (`bind` `0.0.0.0`) | opens the read-only S3 and WebDAV servers for the asset tree | Yes | *(documented, docs/file-tree.md "Off until an admin turns it on")* | | `tunnel.mode`, `local_llm.enabled`, `telemetry` | off | outbound tunnel (Cloudflare terminates TLS, Q30); local model server; OTLP export | Yes | *(documented, docs/remote-access.md "It's off until an admin turns it on")* | -| `https.enabled` (and install.sh's `LENS_HTTPS=1`) | off | Caddy, fetched at run time, terminates TLS on ports 443/80 in the worker and proxies to the web app; the DNS provider's credentials are written to `data_dir/https/caddy.json` (mode 600), not a command line | Yes | *(documented, docs/https.md "Off by default")* | +| `https.enabled` (and install.sh's `LENS_HTTPS=1`, which Settings › HTTPS can also ask the upgrade agent for) | off | Caddy, fetched at run time, terminates TLS on ports 443/80 in the worker and proxies to the web app; the DNS provider's credentials are written to `data_dir/https/caddy.json` (mode 600), not a command line | Yes | *(documented, docs/https.md "Off by default")* | +| Reachable from the network: install.sh's `LENS_PUBLIC` (kept as `LENS_BIND`), also switched from Settings › Access & embedding, `PUT /api/v1/admin/install` or MCP `set_install_settings` | `1` (`0.0.0.0`) on a server without a desktop, else `0` (`127.0.0.1`) | publishes the web app's port to the network over plain HTTP, and the sensor, file-share and Grafana ports with it unless `LENS_SENSOR_BIND` or `LENS_SHARE_BIND` say otherwise. Admins (an agent with an admin's write-scope token too) can open it without a shell: the host upgrade agent runs install.sh again; audited as `lens.install` | Supported on a trusted LAN only until HTTPS is set up, as the headless-host row | *(documented, docs/updates.md "Setting the machine up again")* | +| Edition: install.sh's `LENS_TARGET`, also switched from Settings › Components or `PUT /api/v1/admin/install` | `full` | full ships LibreOffice and Chromium (Ingest's child processes); lean leaves them out | Yes, either | *(documented, docs/updates.md "Setting the machine up again")* | | `components.auto` | on | unpinned-hash pip installs at runtime | Yes | *(maintainer, 2026-10)* | | `encryption.files` | on for new archives | file encryption at rest | Yes | *(documented, docs/encryption.md "A new archive turns this on at its first start")* | | `ai.extensions`, `ai.disabled_tools` | on, none | turn assistant extensions or named tools off | Yes | *(documented, docs/assistant.md "Extending the assistant")* | @@ -292,7 +294,8 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | S3 9555 and WebDAV 9556 (file shares) | signed S3 requests (SigV4 header or presigned link); WebDAV with an access key as Basic sign-in | yes, once enabled | x-credential, resource-name | admin enables; each request carries a person's own access key; read-only; failed WebDAV sign-ins throttled per address | *(documented, docs/file-tree.md "Off until an admin turns it on")* | | `.env`, `archive.yaml`, `LENS_*`, `ARCHIVE_SECRET_KEY`, `RCLONE_BINARY` | process configuration | no (operator) | x-env-config | keep unreadable to others | *(maintainer, 2026-10)* | | workers and DB peers | DB credential, shared database rows | no (trusted peer) | x-peer | run peers on trusted hosts | *(inferred, Q9)* | -| `.lens-control/inbox/upgrade` (host upgrade agent), `/metrics`, `.lens-control/prometheus-token` | `ref` (release tag or this install's branch), `action=services` with `monitoring=on`/`off`; the scrape token | the inbox is writable by the containers (and local users); `/metrics` by anyone reaching the web app | x-env-config, credential | the agent installs only a release tag or the install's branch and only toggles the `monitoring` profile, nothing else from the file; `/metrics` answers only the scrape token (constant-time compare); the token file is world-readable on the host and grants metrics only, which carry no personal content | *(documented, docs/updates.md, docs/prometheus.md "Privacy")* | +| `.lens-control/inbox/upgrade` (host upgrade agent), `/metrics`, `.lens-control/prometheus-token` | `ref` (release tag or this install's branch; empty: the version running), with `https=`, `public=` (`0`/`1`) and `target=` (`full`/`lean`); `action=services` with `monitoring=on`/`off`; the scrape token | the inbox is writable by the containers (and local users); `/metrics` by anyone reaching the web app | x-env-config, credential | the agent installs only a release tag or the install's branch, changes only the HTTPS ports, the web app's network reach and the edition (any other value is dropped) and only toggles the `monitoring` profile, nothing else from the file, so a writer of the inbox can at most restart Lens with those choices or open it to the network; `/metrics` answers only the scrape token (constant-time compare); the token file is world-readable on the host and grants metrics only, which carry no personal content | *(documented, docs/updates.md, docs/prometheus.md "Privacy")* | +| `PUT /api/v1/admin/install`, `POST /api/v1/settings/https/ports`, MCP `set_install_settings` | `public`, `target`, `open` | admin only (MCP: with the write scope) | x-env-config | admin is trusted; audited (`lens.install`, `https.ports`); the change reaches the machine only as an inbox request, which the agent checks again | *(documented, docs/updates.md "Setting the machine up again")* | | `ask/NAME.want` in a microsandbox's `/data/.lens/programs` (LibreOffice and Chromium on first use) | a program's name | writable by the Lens user inside the sandbox | x-env-config | the start script, as root, installs only `libreoffice` or `chromium`, each a fixed list of signed Debian packages, and answers in `got/`, which only root writes; it reads nothing from the file but its name, and only deletes any other name or link | *(documented, docs/microsandbox.md "LibreOffice and Chromium")* | **Stateful postconditions.** A refused or failed one-time secret check consumes nothing; a used one is deleted before @@ -497,7 +500,7 @@ Paths in the Voided by column are relative to `fastapi_backend/app/`. Searches c | P-IPGROUP | An address opens an IP group only when the server can vouch for it, through `server.trusted_proxies` or `LENS_TRUSTED_PROXY_HOSTS`. Both compose files trust the web app container so IP groups see real addresses; elsewhere a reverse proxy must set `X-Forwarded-For` (other variants: Q2). | integrity-bypass | security-critical | *(documented, docs/access.md "An address the server can't vouch for opens nothing")* | Voided by: Default trusted proxies — `domain/store.py:190`; DNS-named hosts — `api/deps.py:320-331`; /8 and /16 ranges — `domain/ipgroups.py:24`; no `X-Forwarded-For` proxy outside compose (§1.13). | | P-HOST | The `Host` header must be a served host. This stops DNS rebinding. | bad-data-accepted | correctness-only | *(documented, docs/configuration.md "stops DNS rebinding")* | Voided by: `*` in `server.allowed_hosts` — `core/middleware.py:70`; `ARCHIVE_ALLOWED_HOSTS` — `domain/settings.py:129`. | | P-FRAME | The API sends a strict CSP. Only the player (`/embed/`, `/s/`) can be framed, and only by `server.embed_frame_ancestors`. | integrity-bypass | security-critical | *(documented, docs/configuration.md "can be framed")* | Voided by: `server.embed_frame_ancestors` — `core/middleware.py:72` (default `'self'`, `domain/store.py:185`). | -| P-HARDENED | In the compose files the database, API and Fedora ports bind to 127.0.0.1, containers run non-root with `no-new-privileges`, and the web app sends `nosniff`, a referrer policy, a permissions policy and a frame policy (HSTS through Cloudflare). In `docker-compose.prod.yml` the DB password is random and the web app listens on this machine unless `LENS_BIND` is set. | x-missing-hardening | correctness-only | *(documented, docs/deployment.md "What's hardened already")* | Voided by: `LENS_BIND=0.0.0.0` — `docker-compose.prod.yml:111`; dev web port — `docker-compose.yml:95` (§1.3). | +| P-HARDENED | In the compose files the database, API and Fedora ports bind to 127.0.0.1, containers run non-root with `no-new-privileges`, and the web app sends `nosniff`, a referrer policy, a permissions policy and a frame policy (HSTS through Cloudflare). In `docker-compose.prod.yml` the DB password is random and the web app listens on this machine unless `LENS_BIND` is set. | x-missing-hardening | correctness-only | *(documented, docs/deployment.md "What's hardened already")* | Voided by: `LENS_BIND=0.0.0.0` — `docker-compose.prod.yml:153`, which an admin can also ask for from Settings (`PUT /api/v1/admin/install`); dev web port — `docker-compose.yml:95` (§1.3). | | P-SECRETS | Stored secrets (source credentials, notification secrets, model API keys, the Hugging Face token, iCal addresses) are encrypted and write-only. The API reports whether one is set, never its value. A key from the environment is never shown. They do not leak through the API, logs, exports, or a redirect to another server. | info-leak | security-critical | *(documented, SECURITY.md "stored credentials for sources and model servers leaking")* | Voided by: `secret.key` or `ARCHIVE_SECRET_KEY` holders — `domain/settings.py:196-199` (Q14). An admin moving a model URL sends the key there — `domain/llm.py:30-32` (trusted). LLM errors name the URL as typed — `domain/llm.py:41` (Q18). | | P-SSRF-GUARD | URLs a non-admin sets (web import and extension `http` tools by editors; notification targets by owners) reach public addresses only, or networks the operator or an admin listed (an admin, in Settings, never loopback or link-local ones). Web paths use ports 80 and 443, follow redirects only to http and https, re-check every redirect and sub-request, and connect to the checked address. | info-leak | security-critical | *(maintainer, 2026-10)* | Voided by: Operator networks — `domain/store.py:475-476`, `domain/netguard.py:40-41`; admin web networks — `domain/settings.py` `_web_networks`; admin notification networks — `domain/notify.py:286-288`; Chromium without sandbox — `domain/convert.py:273`. Search: `grep -rlE 'public_ip\|netguard\.(resolve\|Guard)\|web_networks' fastapi_backend/app` — `netguard.py`, `webcapture.py`, `feeds.py`, `extensions.py`, `notify.py`, `convert.py`, `store.py`, `routes/imports.py`; none disables the check. | | P-ADMIN-GUARD | Guards Lens documents on admin-set operands hold. iCal feeds are fetched like web captures: public addresses only (and `documents.web_networks`), ports 80 and 443, and the password never goes on to a server the calendar redirects to *(documented, docs/processing.md "fetched the way web pages are captured")*. The telemetry endpoint cannot carry a user name or password or be a link-local address *(documented, domain/settings.py validation message "can't be a link-local address")*. | info-leak | security-critical | *(documented, docs/processing.md "fetched the way web pages are captured")* | Voided by: `documents.web_networks` widens the iCal guard — `domain/feeds.py:377`; telemetry checks — `domain/settings.py:751`, `domain/settings.py:757`. | diff --git a/threat-model.yaml b/threat-model.yaml index f900c7e45..f6631f901 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -2,7 +2,7 @@ # Authority order: prose > yaml > json. Regenerate whenever the prose changes. schema: threat-model-sidecar/v2 project: lens -prose_version: "threat-model.md@sha256:a2a97e50fe7cd260ac694254bdc37899a89fc1b0214f0b1c53d3a940281bd71a" +prose_version: "threat-model.md@sha256:72d7a29115424e7e973fe0865fd4a0e43fefab11c4433c242a1f62643898cd65" model_status: under-review triage_policy: strict confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8} @@ -441,12 +441,12 @@ entry_points: - id: "host control inbox and the monitoring scrape token" component: authz parameters: - - name: "inbox/upgrade request: release ref, or action=services with monitoring=on|off" + - name: "inbox/upgrade request: release ref (empty: the version running) with https=0|1, public=0|1 and target=full|lean, or action=services with monitoring=on|off" attacker_controllable: true x-condition: "writable by the containers and local users" control_kinds: [x-env-config] obligation_id: agent-allow-list - caller_must_enforce: "the agent installs only a release tag or the install's branch, and only toggles the monitoring profile" + caller_must_enforce: "the agent installs only a release tag or the install's branch, changes only the HTTPS ports, the web app's network reach and the edition (other values dropped), and only toggles the monitoring profile; a writer can at most restart Lens with those choices or open it to the network" provenance: {kind: documented, source: "docs/updates.md"} - name: "/metrics scrape token, .lens-control/prometheus-token" attacker_controllable: true @@ -455,6 +455,16 @@ entry_points: obligation_id: metrics-token caller_must_enforce: "token compared in constant time; metrics carry no personal content" provenance: {kind: documented, source: "docs/prometheus.md Privacy"} + - id: "admin machine setup requests" + component: authz + parameters: + - name: "PUT /api/v1/admin/install {public, target}, POST /api/v1/settings/https/ports {open}, MCP set_install_settings" + attacker_controllable: false + x-condition: "admin only (MCP: with the write scope)" + control_kinds: [x-env-config] + obligation_id: admin-is-trusted + caller_must_enforce: "admin is trusted; audited (lens.install, https.ports); reaches the machine only as an inbox request the agent checks again" + provenance: {kind: documented, source: 'docs/updates.md "Setting the machine up again"'} - id: "microsandbox program requests" component: authz parameters: @@ -1106,6 +1116,21 @@ build_flags: affects_properties: - {property_id: P-CHILD-SECRETS, effect: preserves} provenance: {kind: documented, source: 'docs/https.md "Off by default"'} + - name: "Reachable from the network: install.sh LENS_PUBLIC (kept as LENS_BIND), also from Settings › Access & embedding, PUT /api/v1/admin/install or MCP set_install_settings" + default: "1 (0.0.0.0) on a server without a desktop, else 0 (127.0.0.1)" + security_relevant: true + maintainer_stance: supported + x-scope: "publishes the web app's port to the network over plain HTTP, with the sensor, file-share and Grafana ports unless LENS_SENSOR_BIND or LENS_SHARE_BIND say otherwise; admins can open it without a shell through the host upgrade agent (audited as lens.install); supported on a trusted LAN only until HTTPS is set up" + affects_properties: + - {property_id: P-HARDENED, effect: narrows} + provenance: {kind: documented, source: 'docs/updates.md "Setting the machine up again"'} + - name: "Edition: install.sh LENS_TARGET, also from Settings › Components or PUT /api/v1/admin/install" + default: "full" + security_relevant: true + maintainer_stance: supported + x-scope: "full ships LibreOffice and Chromium (Ingest's child processes); lean leaves them out" + affects_properties: [] + provenance: {kind: documented, source: 'docs/updates.md "Setting the machine up again"'} - name: "tunnel.mode, local_llm.enabled, telemetry" default: "off" security_relevant: true @@ -1332,7 +1357,7 @@ properties_claimed: x-tier-question: Q37 conditions: "in the compose files the database, API and Fedora ports bind to 127.0.0.1, containers run non-root with no-new-privileges, and the web app sends nosniff, a referrer policy, a permissions policy and a frame policy (HSTS through Cloudflare); in docker-compose.prod.yml the DB password is random and the web app listens on this machine unless LENS_BIND is set" violation_symptoms: [x-missing-hardening] - x-voided-by: "LENS_BIND=0.0.0.0 (docker-compose.prod.yml:111); dev web port (docker-compose.yml:95, §1.3)" + x-voided-by: "LENS_BIND=0.0.0.0 (docker-compose.prod.yml:153), which an admin can also ask for from Settings (PUT /api/v1/admin/install); dev web port (docker-compose.yml:95, §1.3)" provenance: {kind: documented, source: 'docs/deployment.md "What''s hardened already"'} - id: P-SECRETS kind: confidentiality