Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
takes their roles and their private chats, notes and views, and keeps what they shared under their name; their email
can be invited again ([Authentication](docs/authentication.md#removing-someone)). The assistant gets `list_people`,
`invite_person` and `change_person`, which always wait for an admin's approval.
- **Reachable from the network, and the edition, changed in Settings.** Settings › Access & embedding has **Reachable
from other devices on this network**, and Settings › Components the edition: Full (LibreOffice and Chromium, to read
Office files, web pages and emails) or Lean. Changing either says what will happen and applies with one button: the
machine's upgrade agent runs the install again for the version running, and Lens restarts. Before, the install line
had to run again with `LENS_PUBLIC` or `LENS_TARGET`; without an agent, Settings shows that line. Agents use
`GET`/`PUT /api/v1/admin/install` and the MCP tools `install_settings` and `set_install_settings`
([Updates](docs/updates.md#setting-the-machine-up-again)).
- **NSFW classification, with a model on your server.** Turn it on in Settings → NSFW for every namespace, or on a
namespace's page for that one (off by default). The new `nsfw` step scores a video's sampled frames, a document's
pages or an image with an Apache-2.0 model (fetched on first use, about 105 MB with ONNX Runtime) and flags the
Expand Down
4 changes: 4 additions & 0 deletions docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ x-backend: &backend
# upgrading from the web app (docs/updates.md): the folder install.sh's upgrade agent watches, and the install folder
LENS_CONTROL_DIR: /control
LENS_HOST_DIR: ${LENS_DIR:-}
# how this stack runs, shown (and changed through that agent) in Settings: the edition built above, and where the
# web app's port is published (127.0.0.1: this machine only)
LENS_TARGET: ${LENS_TARGET:-full}
LENS_BIND: ${LENS_BIND:-127.0.0.1}
volumes:
- archive-data:/data
- ./.lens-control:/control
Expand Down
4 changes: 3 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ with AES-GCM and are write-only: the API reports whether one is set, never its v
| `LENS_SETUP_SETTINGS` | | settings to start with, as JSON `{"section": {"key": value}}` (or that JSON base64url-encoded): saved once at startup as if made in the app, so unlike the variables above they stay editable there. The [web installer](get-started.md#the-web-installer) writes its answers here; a `budget` section (`hard_usd`, `soft_usd`, `period`, `on_over`, `price_in`, `price_out`) sets [the overall budget](budgets.md#one-budget-for-everything) |
| `LENS_SETUP_WIZARD` | | `off`: never show the setup wizard |
| `LENS_TELEMETRY` / `LENS_TELEMETRY_ENDPOINT` / `LENS_TELEMETRY_HEADERS` | off | opt-in OpenTelemetry traces and metrics, sent only to this OTLP/HTTP endpoint ([Telemetry](telemetry.md)); `LENS_TELEMETRY=off` keeps it off whatever Settings say |
| `LENS_TARGET` / `LENS_BIND` | | how install.sh set the machine up: the edition (`full` or `lean`) and where the web app's port is published (`0.0.0.0`: the network, `127.0.0.1`: this machine only). `docker-compose.prod.yml` and a microsandbox pass them on from `.env`, and Settings show them; unset, Lens says it doesn't know. They're changed in Settings, which has the install run again ([Updates](updates.md#setting-the-machine-up-again)) |
| `LENS_LOG_LEVEL` / `LENS_LOG_FORMAT` | info / text | the log level and console format for the API and every worker ([Logging](logging.md)); win over Settings → Logs |
| `FRONTEND_URL` | `http://localhost:3000` | the web app's address as people use it: links in emails, and where apps send people to sign in ([OAuth](authentication.md#oauth)) |
| `CORS_ORIGINS` | `["http://localhost:3000"]` | origins allowed to call the API from a browser |
Expand Down Expand Up @@ -105,7 +106,8 @@ Start from `fastapi_backend/archive.example.yaml`, which documents every key. Th

Admins can change, in Settings: transcription, speaker separation, speech providers, voice IDs, analysis, the LLM
provider and key, a local model, the AI assistant, search (word search and search by meaning), reports and graph,
video, OCR, faces and objects, [NSFW classification](nsfw.md), workers, sign-in, components, access and embedding (embed frame ancestors, transcript
video, OCR, faces and objects, [NSFW classification](nsfw.md), workers, sign-in, components (and the edition, full or
lean), access and embedding (whether other devices on the network reach Lens, embed frame ancestors, transcript
upload limit, allowed hosts, trusted proxies, session length), remote access, email, chat rooms, notifications,
telemetry, the Fedora repository, sensors, file shares, uploads, storage, documents, API keys, and IIIF and metadata. The API
refuses an allowed-host list that leaves out the address you are using.
Expand Down
3 changes: 2 additions & 1 deletion docs/get-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ code already filled in. On a server without a desktop, Lens is reachable from th
server's address. Run the same line again to update: the secrets, the database and the archive are kept. Settings go
before `sh`: `LENS_DIR`, `LENS_PORT`, `LENS_PUBLIC` (`1`: from the network, `0`: this machine only), `LENS_TARGET`
(`lean` for the smaller image, kept when the line runs again) and `LENS_REF` (a branch or tag) change the defaults;
the top of `install.sh` lists them. If Lens doesn't answer in time (`LENS_WAIT`, in seconds: 10 minutes with Docker,
the top of `install.sh` lists them. Whether the network reaches it and the edition can be changed later in Settings
(Access & embedding, Components), with no terminal ([Updates](updates.md#setting-the-machine-up-again)). If Lens doesn't answer in time (`LENS_WAIT`, in seconds: 10 minutes with Docker,
30 in a microsandbox), the installer stops waiting and shows the last lines Lens logged and why.

Where it can (Linux with KVM, or a Mac with Apple silicon), a first install asks whether to run Lens in Docker (the
Expand Down
5 changes: 4 additions & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](aut
| `read_file` | a text file of that tree (an asset's `.txt` text or `.json` metadata), a page at a time; other files' download link |
| `file_storage` | where Lens keeps its own files (notes' images and attachments) and the connections it could use; `check` tests it. Admins only |
| `set_file_storage` | keep new files on this machine or a storage connection (folder, rclone crypt); checked before it's saved. Admins only, with a write-scope token |
| `install_settings` | whether other devices on the network reach Lens and its edition (full or lean), whether the machine's upgrade agent can change them, and the install line to run when it can't. Admins only |
| `set_install_settings` | open Lens to the network or keep it to this machine, or change the edition: the machine sets itself up again and Lens restarts ([Updates](updates.md#setting-the-machine-up-again)). Admins only, with a write-scope token |
| `propose_graph_change` | ask for two entities to be merged or linked; it waits in Proposed changes unless `apply` makes it at once (it can be undone). Needs a write-scope token and editor access |

Results are JSON (as `structuredContent`, and the same as text). Links open the recording's page in the web app at
Expand All @@ -86,7 +88,8 @@ expect (`results` with `id`, `title`, `url`; `fetch` by that `id`).
Every tool reads, except `suggest_topic`, which adds suggestions people accept or dismiss, `propose_graph_change`,
which records an undoable change (proposed by default), `write_note`, which adds a note, `create_podcast`, which
queues an episode, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
editors), and `set_file_storage`, which changes where new files are kept (admins).
editors), `set_file_storage`, which changes where new files are kept (admins), and `set_monitoring` and
`set_install_settings`, which have the machine change how Lens runs (admins).
Importing, editing and curating stay in the web app and the API.

### What the archive says is data
Expand Down
39 changes: 38 additions & 1 deletion docs/updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

Lens tells admins when a new release is out, backs the database up, and installs the release from the web app, so
upgrading never needs a terminal. Status: **built**: the check, backups, one-click upgrades with their progress and a
rollback when the new version doesn't start, in Settings → Updates and a banner for admins.
rollback when the new version doesn't start, in Settings → Updates and a banner for admins, and
[setting the machine up again](#setting-the-machine-up-again) from Settings.

## The check

Expand Down Expand Up @@ -70,6 +71,42 @@ with `LENS_UPGRADE_AGENT=0`), the API gives the one line to run on the machine i
`updates.auto_install: true` installs a new release on its own as soon as the check sees it, once per release. It is
off by default: what Lens runs changes only when an admin says so.

## Setting the machine up again

Some choices are made when Lens is installed, as settings on the install line. Three of them can be changed later from
the web app, with no terminal: the agent runs install.sh again for the version running (a request with an empty
`ref`), changing only what the request says, and Lens restarts.

| In Settings | On the install line | In the request |
| --- | --- | --- |
| **Access & embedding › Reachable from other devices on this network** | `LENS_PUBLIC=1` (or `0`: this machine only) | `public=1` or `0` |
| **Components › Edition**: Full (LibreOffice and Chromium, to read Office files, web pages and emails) or Lean | `LENS_TARGET=full` or `lean` | `target=full` or `lean` |
| **HTTPS › Open the HTTPS ports** ([HTTPS](https.md)) | `LENS_HTTPS=1` | `https=1` or `0` |

Changing one says what will happen (Lens restarts; opening it to the network lets other devices reach its sign-in
page; with Docker a new edition is built, which can take a few minutes), then one button applies it. Without an
agent, the same place shows the install line to run on the machine. install.sh keeps each choice in `.env`
(`LENS_BIND`, `LENS_TARGET`, `LENS_HTTPS`), so upgrades keep them; what a request leaves out stays as `.env` has it.
The agent takes nothing but `0` or `1`, and `full` or `lean`.

Lens reads how it runs from what it was started with: `docker-compose.prod.yml` and a microsandbox pass on
`LENS_TARGET` and `LENS_BIND` (`0.0.0.0` or the machine's own address: the network; `127.0.0.1`: this machine only).
Started some other way, it says it doesn't know, rather than guessing.

For agents: `GET /api/v1/admin/install` says `public` and `target` (or null), whether there's an agent, the install
line for each choice (`manual`) and how the last change went (`reconfigure`). `PUT /api/v1/admin/install` with
`{"public": true}`, `{"target": "lean"}` or both asks for the change (202, audited as `lens.install`); what is so
already is left alone, and with nothing left to change nothing restarts (200). Without an agent it answers 409 with
the line to run. The MCP tools `install_settings` and `set_install_settings` do the same (admins, and a write-scope
token to change them).

### Refine later

- Windows installs have no upgrade agent, and the line Lens shows to run there is the one for Linux and Macs
(`curl … | sh`), for upgrades, the HTTPS ports and these choices alike.
- Installs not started by install.sh, `docker-compose.prod.yml` or a microsandbox (Cloudron, the NAS packages,
Proxmox) show whether the network reaches Lens and its edition as not known.

## Backups

* `GET /api/v1/admin/backups`: the backups in `<data_dir>/backups`, newest first.
Expand Down
47 changes: 47 additions & 0 deletions fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
sources,
store,
untrusted,
updates,
)
from app.domain import entities as ents
from app.domain import speakers as spk
Expand Down Expand Up @@ -1738,3 +1739,49 @@ def set_monitoring(ctx: Context, on: bool) -> dict[str, Any]:
raise ToolError(str(e)) from None
auth.audit(ctx.db, ctx.user.as_audit(), "lens.monitoring", None, {"on": bool(on), "via": "mcp"})
return got


# ---------- how install.sh set this machine up: reachable on the network, and the edition (admins) ----------
@tool(
"install_settings",
"How Lens is installed",
"Whether other machines on the network reach the web app (public true) or only this one, and the edition (target: "
"full, with LibreOffice and Chromium to read Office files, web pages and emails, or lean); null where Lens doesn't "
"know. Also whether this machine's upgrade agent can change them (agent), the install line to run on the machine "
"for each choice when it can't (manual), and how the last change went (reconfigure). Admins only.",
)
def install_settings(ctx: Context) -> dict[str, Any]:
if not ctx.user.admin:
raise ToolError("only admins see or change how Lens is installed")
return updates.install_view(ctx.db, ctx.cfg) | {
"settings": f"{ctx.web}/settings/access",
"edition_settings": f"{ctx.web}/settings/components",
}


@tool(
"set_install_settings",
"Change how Lens is installed",
"Make the web app reachable from other machines on the network (public true) or this machine only (false), or "
"change the edition (target full or lean). This machine's upgrade agent runs the install again for the version "
"running, and Lens restarts, away while it does (longer for a new edition with Docker, which builds it); "
"install_settings follows along. What is so already is left alone. Making it reachable from the network lets "
"anyone there reach the sign-in page. Admins only, with a token that has the write scope.",
Arg("public", "boolean", "reachable from other machines on the network (true) or this machine only (false)"),
Arg("target", "string", "the edition: full or lean", enum=updates.TARGETS),
writes=True,
)
def set_install_settings(ctx: Context, public: bool | None = None, target: str | None = None) -> dict[str, Any]:
if not ctx.user.admin:
raise ToolError("only admins see or change how Lens is installed")
if not ctx.user.can_write:
raise ToolError("this token is read-only: ask for a token with the write scope to change how Lens is installed")
if public is None and target is None:
raise ToolError("say what to change: public, target or both")
try:
asked = updates.change_install(ctx.db, ctx.cfg, public, target, by=ctx.user.email)
except updates.UpdateError as e:
raise ToolError(str(e)) from None
if asked:
auth.audit(ctx.db, ctx.user.as_audit(), "lens.install", None, asked | {"via": "mcp"})
return updates.install_view(ctx.db, ctx.cfg) | {"changing": asked}
41 changes: 38 additions & 3 deletions fastapi_backend/app/api/v1/routes/updates.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,27 @@
"""New releases, upgrading from the web app, and the database backups (admins only; docs/updates.md)."""
"""New releases, upgrading from the web app, how the machine is set up (reachable on the network, the edition), and
the database backups (admins only; docs/updates.md)."""

from __future__ import annotations

import contextlib

from fastapi import APIRouter, HTTPException, Request
from fastapi import APIRouter, HTTPException, Request, Response
from fastapi.responses import FileResponse

from app.api.deps import AdminReader, AdminWriter, Cfg, Db, domain_errors
from app.domain import auth, backups, monitoring, updates
from app.schemas.backups import BackupSchedule, RestoreWork
from app.schemas.common import Ok
from app.schemas.updates import Backup, Backups, Monitoring, MonitoringRequest, UpdateStatus, UpgradeRequest
from app.schemas.updates import (
Backup,
Backups,
InstallChange,
InstallStatus,
Monitoring,
MonitoringRequest,
UpdateStatus,
UpgradeRequest,
)

router = APIRouter(tags=["updates"])

Expand Down Expand Up @@ -46,6 +56,31 @@ def start_upgrade(user: AdminWriter, db: Db, cfg: Cfg, body: UpgradeRequest | No
return UpdateStatus.model_validate(updates.view(db, cfg))


@router.get("/admin/install")
def get_install(user: AdminReader, db: Db, cfg: Cfg) -> InstallStatus:
"""How install.sh set this machine up: whether other machines on the network reach the web app, and the edition
(full or lean); what can change them here, and how the last change went."""
return InstallStatus.model_validate(updates.install_view(db, cfg))


@router.put("/admin/install", status_code=202)
def set_install(body: InstallChange, user: AdminWriter, response: Response, db: Db, cfg: Cfg) -> InstallStatus:
"""Make the web app reachable from other machines on the network (public) or this machine only, or change the
edition (target: full, with LibreOffice and Chromium, or lean): this machine's upgrade agent runs install.sh again
for the version running with LENS_PUBLIC and LENS_TARGET, and Lens restarts; GET /admin/install follows along.
What is so already is left alone, and with nothing left to change nothing restarts (200). Without an agent, 409
says the line to run on the machine instead (`manual` has each one)."""
try:
asked = updates.change_install(db, cfg, body.public, body.target, by=user.email)
except updates.UpdateError as e:
raise HTTPException(409, str(e)) from None
if asked:
auth.audit(db, user.as_audit(), "lens.install", None, asked)
else:
response.status_code = 200
return InstallStatus.model_validate(updates.install_view(db, cfg))


@router.get("/admin/monitoring")
def get_monitoring(user: AdminReader, db: Db) -> Monitoring:
"""Whether Prometheus, node-exporter and Grafana run next to Lens (the monitoring profile), and how the machine's
Expand Down
Loading
Loading