Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ a web page needs one, and the Components list says so.
| `components.auto` | `true` | fetch what's needed without asking; off, the workers only report what's missing |
| `components.ahead` | `false` | fetch everything the settings could need straight away, instead of on first use |
| `components.also` | `[]` | optional components to fetch too, e.g. `["msg"]` |
| `components.hf_token` | none | a Hugging Face token, kept encrypted, for gated models: pyannote's speaker diarization, and GGUF models whose page asks you to accept terms. `HF_TOKEN` in the environment wins over it |

**Settings → Components** shows each worker's machine (processors, memory, GPU, free disk), each component and where
every worker is with it, and the transcription engine that suits the machine. `GET /api/v1/components` returns the
Expand Down
2 changes: 1 addition & 1 deletion docs/local-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The page shows each step (fetching llama.cpp, downloading with a percentage, sta
lines when something goes wrong. Downloaded models can be deleted there to free the disk.

Any other GGUF chat model on Hugging Face works too: set the model to `hf:<owner>/<repo>/<file>.gguf`. Gated models
need `HF_TOKEN` in the environment.
need a Hugging Face token: add it in Settings → Components (`components.hf_token`).

## Settings

Expand Down
3 changes: 2 additions & 1 deletion docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ Speaker ids belong to one namespace. Per recording, `diarize.engine: auto` works

- If the file is genuinely two-channel (left and right carry different voices), each channel is one speaker. That separation is exact and free.
- Otherwise, voices are clustered from embeddings. Short backchannels take the label of the nearest long turn.
- Or set the engine to `pyannote`.
- Or set the engine to `pyannote`. Its model is gated: accept its terms on Hugging Face and add a token in Settings →
Components ([Components](components.md#settings)).

Each voice gets a voiceprint (SpeechBrain ECAPA, up to `sample_seconds` of that speaker's clearest audio). The voiceprint is matched one-to-one against the namespace registry:

Expand Down
7 changes: 6 additions & 1 deletion fastapi_backend/app/domain/components.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,11 @@ def activate(cfg):
importlib.invalidate_caches()


def hf_token(cfg):
"""The Hugging Face token gated models are downloaded with: HF_TOKEN, else the one in Settings → Components."""
return os.environ.get("HF_TOKEN") or (cfg.get("components") or {}).get("hf_token") or None


def importable(*modules):
for m in modules:
try:
Expand Down Expand Up @@ -615,7 +620,7 @@ def fetch(self, cfg, m, say):

if say:
say(f"downloading the Laya model {self._model(cfg)}")
snapshot_download(self._model(cfg))
snapshot_download(self._model(cfg), token=hf_token(cfg))
_mark(cfg, self._tag(cfg))

def detail(self, cfg):
Expand Down
10 changes: 5 additions & 5 deletions fastapi_backend/app/domain/local_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
import urllib.request
import zipfile

from . import machine, settings, store
from . import components, machine, settings, store

R = store.R
ROW = R("app_service", "local_llm")
Expand Down Expand Up @@ -202,9 +202,9 @@ def remove(cfg, ref):


# ---------- downloading ----------
def _open(url, headers=None, timeout=60):
def _open(url, headers=None, timeout=60, token=None):
"""`token`: a Hugging Face token, sent to huggingface.co only."""
req = urllib.request.Request(url, headers={"User-Agent": "lens", **(headers or {})})
token = os.environ.get("HF_TOKEN")
if token and urllib.parse.urlsplit(url).hostname == "huggingface.co":
req.add_header("Authorization", f"Bearer {token}")
return urllib.request.urlopen(req, timeout=timeout)
Expand All @@ -224,14 +224,14 @@ def download(cfg, m, progress=None, stop=None):
if m["size"] and m["size"] - have > free:
raise LocalModelError(f"not enough disk for {m['label']}: needs {m['size'] / GB:.1f} GB, {free / GB:.1f} GB free")
try:
r = _open(url, {"Range": f"bytes={have}-"} if have else None)
r = _open(url, {"Range": f"bytes={have}-"} if have else None, token=components.hf_token(cfg))
except urllib.error.HTTPError as e:
if e.code == 416: # the part is already whole
r = None
else:
raise LocalModelError(
f"Hugging Face said {e.code} for {m['repo']}/{m['file']}"
+ (" (a gated model: set HF_TOKEN)" if e.code in (401, 403) else "")
+ (" (a gated model: add a Hugging Face token in Settings → Components)" if e.code in (401, 403) else "")
) from None
except (urllib.error.URLError, TimeoutError, OSError) as e:
raise LocalModelError(f"can't reach Hugging Face: {getattr(e, 'reason', e)}") from None
Expand Down
9 changes: 9 additions & 0 deletions fastapi_backend/app/domain/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@
"tunnel": ("token", "api_token"),
"https": ("dns_credentials", "eab_hmac_key"),
"documents": ("anytopdf_token",),
"components": ("hf_token",),
}
ENUMS = {
("transcribe", "engine"): {"sensevoice", "whisper", "mlx-whisper", "openai", "elevenlabs", "assemblyai", "deepgram"},
Expand Down Expand Up @@ -219,6 +220,7 @@
("fedora", "url"): "LENS_FEDORA_URL",
("fedora", "user"): "LENS_FEDORA_USER",
("fedora", "password"): "LENS_FEDORA_PASSWORD",
("components", "hf_token"): "HF_TOKEN",
}


Expand Down Expand Up @@ -922,6 +924,12 @@ def _component_setting(key, value):
if not (isinstance(value, list) and all(isinstance(v, str) and v in known for v in value)):
raise ValueError("components.also lists components to fetch: " + ", ".join(sorted(known)))
return list(dict.fromkeys(value))
if key == "hf_token":
if value in (None, ""):
return None
if not (isinstance(value, str) and re.fullmatch(r"[A-Za-z0-9_\-]{8,500}", value.strip())):
raise ValueError("components.hf_token is a Hugging Face access token, such as hf_…")
return value.strip()
raise ValueError(f"unknown setting components.{key}")


Expand Down Expand Up @@ -1393,6 +1401,7 @@ def save(db, base, section, changes, user=None):
raise ValueError(f"matterbridge.{k} is one line of text")
if (section, k) in (
("documents", "anytopdf_token"),
("components", "hf_token"),
("telemetry", "prometheus_token"),
("telegram", "token"),
) or section == "slack":
Expand Down
4 changes: 2 additions & 2 deletions fastapi_backend/app/domain/speakers.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

from collections import Counter

from . import ingest, store
from . import components, ingest, store

GENERIC = re.compile(r"^(?:SPEAKER_?\d+|spk_?\d+|S\d+|CH\d+|Speaker \d+|unknown)$", re.I)
_CACHE = {}
Expand Down Expand Up @@ -131,7 +131,7 @@ def pyannote_turns(cfg, path):
from pyannote.audio import Pipeline
except ImportError as e:
raise RuntimeError("uv sync --extra pyannote") from e
tok = os.environ.get(d["pyannote"].get("token_env") or "HF_TOKEN")
tok = os.environ.get(d["pyannote"].get("token_env") or "HF_TOKEN") or components.hf_token(cfg)
try:
pipe = Pipeline.from_pretrained(d["pyannote"]["model"], token=tok)
except TypeError:
Expand Down
3 changes: 2 additions & 1 deletion fastapi_backend/app/domain/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -469,7 +469,8 @@ def labels():
},
# what Lens fetches for itself (components.py): auto fetches what the settings need, on first use unless ahead;
# also names optional ones (fetched straight away)
"components": {"auto": True, "ahead": False, "also": []},
# hf_token: a Hugging Face token, for gated models (pyannote, some GGUF models); kept encrypted, HF_TOKEN wins
"components": {"auto": True, "ahead": False, "also": [], "hf_token": None},
# routine decisions the assistant takes instead of asking (decide.py): engine auto uses the decision model when it
# has a key, else the language model. act_above: the confidence it acts on; below it, it asks. Engine laya runs a
# Laya model (laya_model) on this Mac, or asks a Laya server (laya_url, `lens decide-server`).
Expand Down
28 changes: 28 additions & 0 deletions fastapi_backend/tests/api/test_components.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,3 +253,31 @@ def test_admins_see_components_and_ask_for_a_check(client, db, cfg):
assert client.put("/api/v1/settings/components", headers=h, json={"also": ["ffmpeg"]}).status_code == 400
hv = login(client, "vi@x.io", "viewer password 1")
assert client.get("/api/v1/components", headers=hv).status_code == 403


def test_a_hugging_face_token_in_settings_downloads_gated_models(app, client, db, monkeypatch):
"""components.hf_token is kept encrypted, sent to huggingface.co only, and HF_TOKEN in the environment wins."""
from app.domain import local_llm

monkeypatch.delenv("HF_TOKEN", raising=False)
make_user(db, "root@x.io", "root password 1", admin=True)
h = login(client, "root@x.io", "root password 1")
assert client.put("/api/v1/settings/components", json={"hf_token": "hf token"}, headers=h).status_code == 400
r = client.put("/api/v1/settings/components", json={"hf_token": " hf_abcdefghijklmnop "}, headers=h)
assert r.status_code == 200, r.text
shown = client.get("/api/v1/settings", headers=h)
assert "hf_abcdefghijklmnop" not in shown.text and "hf_abcdefghijklmnop" not in str(db.rows("SELECT * FROM app_setting"))
assert shown.json()["components"]["values"]["hf_token"] == {"secret": True, "set": True}
cfg = app.state.settings.current()
assert components.hf_token(cfg) == "hf_abcdefghijklmnop"

sent = []
monkeypatch.setattr(local_llm.urllib.request, "urlopen", lambda req, timeout: sent.append(req))
for url in ("https://huggingface.co/owner/repo/resolve/main/m.gguf", "https://github.com/ggml-org/llama.cpp/releases"):
local_llm._open(url, token=components.hf_token(cfg))
assert [q.get_header("Authorization") for q in sent] == ["Bearer hf_abcdefghijklmnop", None]

monkeypatch.setenv("HF_TOKEN", "hf_fromtheenvironment")
assert components.hf_token(app.state.settings.current()) == "hf_fromtheenvironment"
assert "hf_token" in client.get("/api/v1/settings", headers=h).json()["components"]["locked"]
assert client.put("/api/v1/settings/components", json={"hf_token": None}, headers=h).status_code == 200
1 change: 1 addition & 0 deletions nextjs-frontend/components/settings/bodies.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,7 @@ export function SectionBody({ ctx }: { ctx: BodyCtx }) {
return (
<>
<F ctx={ctx} id="components.auto" />
<F ctx={ctx} id="components.hf_token" />
<ComponentsStatus ctx={ctx} />
</>
);
Expand Down
7 changes: 7 additions & 0 deletions nextjs-frontend/components/settings/model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1594,6 +1594,13 @@ export const FIELDS: FieldSpec[] = [
hint: "Off: engines and models are fetched the first time a recording needs them",
},
{ section: "components", key: "also", label: "Also fetch", kind: "checks" },
{
section: "components",
key: "hf_token",
label: "Hugging Face token",
kind: "secret",
hint: "For gated models, such as pyannote’s speaker diarization; a read-only token is enough",
},
// Voice
{
section: "voice",
Expand Down
2 changes: 1 addition & 1 deletion threat-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -498,7 +498,7 @@ Paths in the Voided by column are relative to `fastapi_backend/app/`. Searches c
| P-HOST | The `Host` header must be a served host. This stops DNS rebinding. | bad-data-accepted | correctness-only | *(documented, docs/configuration.md "stops DNS rebinding")* | Voided by: `*` in `server.allowed_hosts` — `core/middleware.py:70`; `ARCHIVE_ALLOWED_HOSTS` — `domain/settings.py:129`. |
| P-FRAME | The API sends a strict CSP. Only the player (`/embed/<id>`, `/s/<code>`) can be framed, and only by `server.embed_frame_ancestors`. | integrity-bypass | security-critical | *(documented, docs/configuration.md "can be framed")* | Voided by: `server.embed_frame_ancestors` — `core/middleware.py:72` (default `'self'`, `domain/store.py:185`). |
| P-HARDENED | In the compose files the database, API and Fedora ports bind to 127.0.0.1, containers run non-root with `no-new-privileges`, and the web app sends `nosniff`, a referrer policy, a permissions policy and a frame policy (HSTS through Cloudflare). In `docker-compose.prod.yml` the DB password is random and the web app listens on this machine unless `LENS_BIND` is set. | x-missing-hardening | correctness-only | *(documented, docs/deployment.md "What's hardened already")* | Voided by: `LENS_BIND=0.0.0.0` — `docker-compose.prod.yml:111`; dev web port — `docker-compose.yml:95` (§1.3). |
| P-SECRETS | Stored secrets (source credentials, notification secrets, model API keys, iCal addresses) are encrypted and write-only. The API reports whether one is set, never its value. A key from the environment is never shown. They do not leak through the API, logs, exports, or a redirect to another server. | info-leak | security-critical | *(documented, SECURITY.md "stored credentials for sources and model servers leaking")* | Voided by: `secret.key` or `ARCHIVE_SECRET_KEY` holders — `domain/settings.py:196-199` (Q14). An admin moving a model URL sends the key there — `domain/llm.py:30-32` (trusted). LLM errors name the URL as typed — `domain/llm.py:41` (Q18). |
| P-SECRETS | Stored secrets (source credentials, notification secrets, model API keys, the Hugging Face token, iCal addresses) are encrypted and write-only. The API reports whether one is set, never its value. A key from the environment is never shown. They do not leak through the API, logs, exports, or a redirect to another server. | info-leak | security-critical | *(documented, SECURITY.md "stored credentials for sources and model servers leaking")* | Voided by: `secret.key` or `ARCHIVE_SECRET_KEY` holders — `domain/settings.py:196-199` (Q14). An admin moving a model URL sends the key there — `domain/llm.py:30-32` (trusted). LLM errors name the URL as typed — `domain/llm.py:41` (Q18). |
| P-SSRF-GUARD | URLs a non-admin sets (web import and extension `http` tools by editors; notification targets by owners) reach public addresses only, or networks the operator or an admin listed (an admin, in Settings, never loopback or link-local ones). Web paths use ports 80 and 443, follow redirects only to http and https, re-check every redirect and sub-request, and connect to the checked address. | info-leak | security-critical | *(maintainer, 2026-10)* | Voided by: Operator networks — `domain/store.py:475-476`, `domain/netguard.py:40-41`; admin web networks — `domain/settings.py` `_web_networks`; admin notification networks — `domain/notify.py:286-288`; Chromium without sandbox — `domain/convert.py:273`. Search: `grep -rlE 'public_ip\|netguard\.(resolve\|Guard)\|web_networks' fastapi_backend/app` — `netguard.py`, `webcapture.py`, `feeds.py`, `extensions.py`, `notify.py`, `convert.py`, `store.py`, `routes/imports.py`; none disables the check. |
| P-ADMIN-GUARD | Guards Lens documents on admin-set operands hold. iCal feeds are fetched like web captures: public addresses only (and `documents.web_networks`), ports 80 and 443, and the password never goes on to a server the calendar redirects to *(documented, docs/processing.md "fetched the way web pages are captured")*. The telemetry endpoint cannot carry a user name or password or be a link-local address *(documented, domain/settings.py validation message "can't be a link-local address")*. | info-leak | security-critical | *(documented, docs/processing.md "fetched the way web pages are captured")* | Voided by: `documents.web_networks` widens the iCal guard — `domain/feeds.py:377`; telemetry checks — `domain/settings.py:751`, `domain/settings.py:757`. |
| P-NO-FETCH | SPARQL refuses SERVICE and FROM and never loads a named graph. RDF import fetches nothing, refuses remote JSON-LD contexts and does not read RDF/XML. | info-leak | security-critical | *(documented, rdf_import.py docstring "Nothing is fetched while reading")* | Voided by: Process-global rdflib flag, set per call — `domain/rdf.py:576`; SERVICE refused — `domain/rdf.py:551`; remote contexts — `domain/rdf_import.py:54-56`. |
Expand Down
2 changes: 1 addition & 1 deletion threat-model.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Authority order: prose > yaml > json. Regenerate whenever the prose changes.
schema: threat-model-sidecar/v2
project: lens
prose_version: "threat-model.md@sha256:5e6538f117e92f63bd1f03fa685af70fbf8539fe0e46e4da9148eca433005cf6"
prose_version: "threat-model.md@sha256:a2a97e50fe7cd260ac694254bdc37899a89fc1b0214f0b1c53d3a940281bd71a"
model_status: under-review
triage_policy: strict
confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8}
Expand Down
Loading