Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Apps and AI assistants in the setup wizard.** A new optional step, Apps and AI, asks whether apps and AI
assistants (Claude, ChatGPT, Cursor and other MCP clients) may sign people in with their Lens account, how long
their tokens last, and shows the MCP server's address to add to an assistant. Skipping it keeps OAuth on, as
before. Admins can also turn it off in Settings → API keys (`tokens.oauth_enabled`): discovery and registration go
away and apps' tokens stop working until it is back on. API: `PUT /api/v1/setup/oauth`.
- **Budgets for routines, pipelines, workflows and namespaces.** An admin caps what a resource may cost, in USD,
tokens or both, per run, day, week or month. Before a routine runs or a job starts, Lens estimates the run from past
ones; one that would go over is held until someone picks run or skip (or is skipped, or the decision model weighs
Expand Down
3 changes: 3 additions & 0 deletions docs/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,9 @@ app ──► POST /api/v1/oauth/token grant_type=refresh_token before th
what they can't see, the app can't. An admin's app has the admin's roles in every namespace but not the
administration (people, settings, the audit log, everyone's keys): that stays with admins signed in or with their
own API key.
* **On unless turned off**: admins switch it in Settings → API keys or the setup wizard's Apps and AI step
(`tokens.oauth_enabled`). Off, `/.well-known/…` answers 404, apps can't register, ask or renew, and their tokens
stop working; the access people gave comes back when it is turned on again.
* **Lifetimes** are the admins' ([Configuration](configuration.md#api-keys)): the access token lasts
`tokens.oauth_access_minutes` (60), and the app stays signed in for `tokens.oauth_refresh_days` (30) after it last
renewed, at most `tokens.max_days`.
Expand Down
8 changes: 7 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,15 @@ A fresh install (no accounts when the API first starts) walks its first admin th
with a test.
4. **Storage**: where data lives (shown; set in archive.yaml and `SURREAL_URL`), the largest upload, and optionally a
folder inside `sources.local_roots` to watch.
5. **Apps and AI**: whether apps and AI assistants (Claude, ChatGPT, Cursor and other MCP clients) may sign people in
with their Lens account ([OAuth](authentication.md#oauth)), on unless turned off, how long their tokens last, and
the MCP server's address to add to an assistant.
6. **Telemetry**: off unless chosen ([Telemetry](telemetry.md)).

Every step can be skipped, and the whole wizard too; all of it stays in Settings. Values from the environment win and
show locked. Installs that already had accounts never see the wizard. The API side is `GET /api/v1/setup`,
`POST /api/v1/setup/namespace`, `PUT /api/v1/setup/llm`, `PUT /api/v1/setup/storage` and `POST /api/v1/setup/finish`
`POST /api/v1/setup/namespace`, `PUT /api/v1/setup/llm`, `PUT /api/v1/setup/storage`, `PUT /api/v1/setup/oauth`,
`PUT /api/v1/setup/telemetry` and `POST /api/v1/setup/finish`
(admins); `GET /api/v1/auth/status` says whether it is still pending (`wizard_pending`).

## Frontend environment
Expand Down Expand Up @@ -260,6 +265,7 @@ Settings → API keys:
| `tokens.default_days` | 90 | how long a new key lasts when its maker doesn't say, 1–3650 days (at most `tokens.max_days`) |
| `tokens.max_days` | 365 | the longest a key may last, 1–3650 days |
| `tokens.never_expire` | false | whether keys may be made that never expire |
| `tokens.oauth_enabled` | true | whether apps and AI assistants may sign people in with their Lens account ([OAuth](authentication.md#oauth)); off, discovery and registration are gone and apps' tokens stop working until it is back on, while people can still see and revoke the apps they allowed |
| `tokens.oauth_access_minutes` | 60 | how long the access token of an app someone signed in to lasts ([OAuth](authentication.md#oauth)), 5–1440 minutes; the app renews it by itself |
| `tokens.oauth_refresh_days` | 30 | how long such an app stays signed in after it last renewed its access, 1–3650 days (at most `tokens.max_days`) |

Expand Down
3 changes: 2 additions & 1 deletion fastapi_backend/app/api/deps.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ def _principal(request: Request, db: DB) -> Principal | None:
u["id"], u["email"], u.get("name"), bool(u.get("admin")), "token", "write" if u.get("scope") == "write" else "read"
)
elif raw.startswith("lo_"):
u = oauth.token_account(db, raw)
# with OAuth turned off, apps' tokens stop working at once (their grants stay, so turning it back on restores them)
u = oauth.token_account(db, raw) if oauth.enabled(request.app.state.settings.current()) else None
if u:
p = Principal(u["id"], u["email"], u.get("name"), bool(u.get("admin")), "oauth", u["scope"], resource=u.get("resource"))
elif raw:
Expand Down
33 changes: 26 additions & 7 deletions fastapi_backend/app/api/v1/routes/oauth.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@
MCP, MCP_SCOPE = "/mcp", "read" # the MCP server (app/api/mcp.py), and the scope its tools need


OFF = "apps can't sign in to this server; an admin turns it on in Settings → API keys"


def _on(request: Request) -> bool:
"""Whether apps and MCP clients may sign people in (tokens.oauth_enabled). Off, discovery is gone, apps can't
register, ask or renew, and their tokens don't work; people still see and revoke the apps they allowed."""
return oauth.enabled(request.app.state.settings.current())


def _first(value: str | None) -> str:
return (value or "").split(",")[0].strip()

Expand Down Expand Up @@ -107,7 +116,7 @@ def _client_credentials(request: Request, client_id: str | None, client_secret:

async def bearer_challenge(request: Request, exc: StarletteHTTPException) -> Response:
"""401s that ask for a bearer token say where to find out how to get one (RFC 9728 §5.1), as MCP clients expect."""
if exc.status_code == 401 and (exc.headers or {}).get("WWW-Authenticate") == "Bearer":
if exc.status_code == 401 and (exc.headers or {}).get("WWW-Authenticate") == "Bearer" and _on(request):
# the MCP server is a resource of its own (/mcp), so clients that check what the metadata names see their URL;
# its tools only read, so that's all it asks for
mcp = request.url.path == MCP
Expand All @@ -121,6 +130,8 @@ async def bearer_challenge(request: Request, exc: StarletteHTTPException) -> Res
@well_known.get("/.well-known/oauth-authorization-server/{rest:path}")
def authorization_server(request: Request) -> JSONResponse:
"""Authorization server metadata (RFC 8414)."""
if not _on(request):
raise HTTPException(404, "not found")
base = public_base(request)
doc = {
"issuer": base,
Expand All @@ -143,7 +154,7 @@ def authorization_server(request: Request) -> JSONResponse:
def protected_resource(request: Request, rest: str = "") -> JSONResponse:
"""Protected resource metadata (RFC 9728): who issues the tokens this server takes."""
base = public_base(request)
if rest and not RESOURCE.fullmatch(rest):
if not _on(request) or (rest and not RESOURCE.fullmatch(rest)):
raise HTTPException(404, "not found")
doc = {
"resource": f"{base}/{rest}" if rest else base,
Expand All @@ -159,6 +170,8 @@ def protected_resource(request: Request, rest: str = "") -> JSONResponse:
def register(body: ClientRegistration, request: Request, db: Db) -> Any:
"""An app registers itself (RFC 7591), without signing in: it gets a `client_id`, and a `client_secret` if it
asked for one. It can do nothing until someone gives it access. Audited as `oauth.client.register`."""
if not _on(request):
return _oauth_error(oauth.OAuthError("access_denied", OFF, 403))
key = f"oauth-register|{visitor_address(request) or client_ip(request)}"
if auth.throttled(key):
raise HTTPException(429, "too many apps registered from this address; try again in a few minutes")
Expand All @@ -185,13 +198,16 @@ def register(body: ClientRegistration, request: Request, db: Db) -> Any:
)


def _person(user: CurrentUser) -> None:
def _person(user: CurrentUser, request: Request | None = None) -> None:
if request is not None and not _on(request):
raise HTTPException(403, OFF)
if user.via != "access":
raise HTTPException(403, "sign in to give an app access; API tokens and apps can't")


@router.get("/authorize")
def consent(
request: Request,
user: CurrentUser,
db: Db,
client_id: str = Query(),
Expand All @@ -202,20 +218,21 @@ def consent(
scope: str | None = Query(None),
) -> Consent:
"""What the consent page shows for an app's request: the app, where it takes you back to and what it asks for.
400 when the request can't be answered (an unknown app, an address it didn't register, no PKCE challenge)."""
_person(user)
400 when the request can't be answered (an unknown app, an address it didn't register, no PKCE challenge); 403
when apps can't sign in to this server."""
_person(user, request)
with domain_errors():
return Consent(
**oauth.authorization(db, user.id, client_id, redirect_uri, response_type, code_challenge, code_challenge_method, scope)
)


@router.post("/authorize")
def answer(body: ConsentAnswer, user: CurrentUser, db: Db) -> ConsentResult:
def answer(body: ConsentAnswer, request: Request, user: CurrentUser, db: Db) -> ConsentResult:
"""Your answer to an app's request. Yes gives it a one-time code (five minutes) at its redirect address, which
it swaps for tokens that act as you, with your roles; no sends it `error=access_denied`. Audited as
`oauth.grant`."""
_person(user)
_person(user, request)
with domain_errors():
info = oauth.authorization(
db, user.id, body.client_id, body.redirect_uri, body.response_type, body.code_challenge, body.code_challenge_method, body.scope
Expand Down Expand Up @@ -243,6 +260,8 @@ def token(
"""The app swaps its code (`grant_type=authorization_code`, with `code_verifier` and the same `redirect_uri`)
or its refresh token (`grant_type=refresh_token`) for an access token and a new refresh token. Form-encoded, as
OAuth has it; errors are `{error, error_description}`."""
if not _on(request):
return _oauth_error(oauth.OAuthError("access_denied", OFF, 403))
try:
cid, secret = _client_credentials(request, client_id, client_secret)
if grant_type == "authorization_code":
Expand Down
27 changes: 25 additions & 2 deletions fastapi_backend/app/api/v1/routes/setup.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""The first-run setup wizard (admins): the first namespace, the model provider, storage and telemetry, then finishing it.
"""The first-run setup wizard (admins): the first namespace, the model provider, storage, apps signing in (OAuth) and
telemetry, then finishing it.

The first admin is created before this, with the setup code (POST /auth/setup) or from the environment. Fields the
environment sets are left out of every save and reported as locked (domain/setup.py).
Expand All @@ -10,7 +11,17 @@

from app.api.deps import AdminReader, AdminWriter, Db
from app.domain import auth, setup
from app.schemas.setup import LocalModelServer, SetupFinish, SetupLlm, SetupNamespace, SetupSaved, SetupStorage, SetupTelemetry, SetupView
from app.schemas.setup import (
LocalModelServer,
SetupFinish,
SetupLlm,
SetupNamespace,
SetupOAuth,
SetupSaved,
SetupStorage,
SetupTelemetry,
SetupView,
)

router = APIRouter(prefix="/setup", tags=["setup"])

Expand Down Expand Up @@ -85,6 +96,18 @@ def save_telemetry(body: SetupTelemetry, user: AdminWriter, request: Request, db
return SetupSaved(saved=saved)


@router.put("/oauth")
def save_oauth(body: SetupOAuth, user: AdminWriter, request: Request, db: Db) -> SetupSaved:
"""Let apps and AI assistants (MCP clients such as Claude or Cursor) sign people in with their Lens account, or
not, and how long their tokens last (docs/authentication.md#oauth). On unless turned off."""
try:
saved = setup.save_oauth(db, request.app.state.archive.base, body.enabled, body.access_minutes, body.refresh_days, user.email)
except ValueError as e:
raise HTTPException(400, str(e)) from None
auth.audit(db, user.as_audit(), "settings.save", "tokens", saved)
return SetupSaved(saved=saved)


@router.post("/finish")
def finish(body: SetupFinish, user: AdminWriter, db: Db) -> SetupSaved:
"""Finish (or skip) the wizard; the web app stops showing it. Everything stays changeable in Settings."""
Expand Down
5 changes: 5 additions & 0 deletions fastapi_backend/app/domain/oauth.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ def _later(seconds):
return (dt.datetime.now(dt.timezone.utc) + dt.timedelta(seconds=seconds)).isoformat(timespec="seconds")


def enabled(cfg):
"""Whether apps and MCP clients may sign people in here (tokens.oauth_enabled; Settings → API keys, or setup)."""
return bool({**store.DEFAULTS["tokens"], **(cfg.get("tokens") or {})}["oauth_enabled"])


def lifetimes(cfg):
"""How long an app's tokens last (tokens settings): {access_minutes, refresh_days}."""
t = {**store.DEFAULTS["tokens"], **(cfg.get("tokens") or {})}
Expand Down
4 changes: 4 additions & 0 deletions fastapi_backend/app/domain/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,10 @@ def _check(section, key, value, default):
if not (isinstance(value, int) and not isinstance(value, bool) and lo <= value <= hi):
raise ValueError(f"tokens.oauth_access_minutes is a whole number of minutes from {lo} to {hi}")
return value
if (section, key) == ("tokens", "oauth_enabled"):
if not isinstance(value, bool):
raise ValueError("tokens.oauth_enabled is true or false")
return value
if section == "tokens" and key != "never_expire":
lo, hi = TOKEN_DAYS
if not (isinstance(value, int) and not isinstance(value, bool) and lo <= value <= hi):
Expand Down
16 changes: 16 additions & 0 deletions fastapi_backend/app/domain/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,11 @@ def view(db, cfg):
"endpoint": shown["telemetry"].get("endpoint"),
"locked": [k for k in settings.locked("telemetry") if k in ("enabled", "endpoint")],
},
"oauth": {
"enabled": bool(shown["tokens"].get("oauth_enabled", True)),
"access_minutes": int(shown["tokens"]["oauth_access_minutes"]),
"refresh_days": int(shown["tokens"]["oauth_refresh_days"]),
},
}


Expand Down Expand Up @@ -158,6 +163,17 @@ def save_telemetry(db, cfg, enabled, endpoint, user=None):
return sorted(keep)


def save_oauth(db, cfg, enabled, access_minutes=None, refresh_days=None, user=None):
"""Whether apps and MCP clients may sign people in through OAuth, and how long their tokens last."""
values = {"oauth_enabled": bool(enabled)}
if access_minutes is not None:
values["oauth_access_minutes"] = access_minutes
if refresh_days is not None:
values["oauth_refresh_days"] = refresh_days
settings.save(db, cfg, "tokens", values, user)
return sorted(values)


def save_storage(db, cfg, max_upload_mb=None, folder=None, namespace=None, user=None):
"""The upload limit, and optionally a folder on this machine (inside sources.local_roots) watched into a
namespace. Returns the watch's id, if one was made."""
Expand Down
13 changes: 10 additions & 3 deletions fastapi_backend/app/domain/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,16 @@ def labels():
# without passwords; an install that already had them keeps them until an admin turns them off.
"auth": {"passwords": False},
# how long API keys last (docs/configuration.md): what a new key gets, the most it may get, and whether keys may
# never expire; and how long the tokens of apps given access through OAuth last (domain/oauth.py): the access token,
# and the grant after the app last renewed it
"tokens": {"default_days": 90, "max_days": 365, "never_expire": False, "oauth_access_minutes": 60, "oauth_refresh_days": 30},
# never expire; and whether apps and MCP clients may sign people in through OAuth (domain/oauth.py), and how long
# their tokens last: the access token, and the grant after the app last renewed it
"tokens": {
"default_days": 90,
"max_days": 365,
"never_expire": False,
"oauth_enabled": True,
"oauth_access_minutes": 60,
"oauth_refresh_days": 30,
},
# audio, video, documents and images uploaded in the web app, in pieces (docs/configuration.md); transcript files use
# server.max_upload_mb
"uploads": {"max_mb": 4096, "extensions": list(MEDIA_EXT + DOCUMENT_EXT + IMAGE_EXT), "chunk_mb": 8, "expire_hours": 24},
Expand Down
15 changes: 15 additions & 0 deletions fastapi_backend/app/schemas/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,21 @@ class SetupTelemetryView(ResponseModel):
locked: list[str]


class SetupOAuthView(ResponseModel):
# whether apps and MCP clients may sign people in with their Lens account (tokens.oauth_enabled)
enabled: bool
access_minutes: int
refresh_days: int


class SetupView(ResponseModel):
pending: bool
admin: SetupAdmin
namespace: SetupNamespaceView
llm: SetupLlmView
storage: SetupStorageView
telemetry: SetupTelemetryView
oauth: SetupOAuthView


class SetupNamespace(RequestModel):
Expand Down Expand Up @@ -79,6 +87,13 @@ class SetupTelemetry(RequestModel):
endpoint: str | None = Field(default=None, max_length=500)


class SetupOAuth(RequestModel):
enabled: bool = True
# left out: unchanged
access_minutes: int | None = None
refresh_days: int | None = None


class SetupFinish(RequestModel):
skipped: bool = False

Expand Down
26 changes: 26 additions & 0 deletions fastapi_backend/tests/api/test_oauth.py
Original file line number Diff line number Diff line change
Expand Up @@ -426,3 +426,29 @@ def test_redirects_keep_the_apps_own_query():
assert oauth.scopes("write") == "read write" and oauth.scopes("email profile") == "read" and oauth.scopes(None) == "read"
assert not oauth.redirect_ok(["https://a.example/cb"], "http://[bad")
assert oauth.token_account(None, "la_not_oauth") is None and oauth.get_client(None, "nope") is None


def test_admins_can_turn_apps_signing_in_off(client, db):
make_user(db, "root@x.io", "root password 1", admin=True)
make_user(db, "vi@x.io", "viewer password 1", roles={"pods": "viewer"})
hr, h = login(client, "root@x.io", "root password 1"), login(client, "vi@x.io", "viewer password 1")
app, t = _grant(client, h)
assert client.put("/api/v1/settings/tokens", headers=hr, json={"oauth_enabled": "no"}).status_code == 400
assert client.put("/api/v1/settings/tokens", headers=hr, json={"oauth_enabled": False}).status_code == 200
# off: no discovery, no new apps, no consent, no renewing, and the tokens apps have stop working
assert client.get("/.well-known/oauth-authorization-server").status_code == 404
assert client.get("/.well-known/oauth-protected-resource").status_code == 404
r = client.post(f"{O}/register", json={"client_name": "New", "redirect_uris": [BACK]})
assert (r.status_code, r.json()["error"]) == (403, "access_denied")
_, challenge = _pkce()
assert client.get(f"{O}/authorize", headers=h, params=_ask(app, challenge)).status_code == 403
assert client.post(f"{O}/authorize", headers=h, json={**_ask(app, challenge), "approve": True}).status_code == 403
r = client.post(f"{O}/token", data={"grant_type": "refresh_token", "client_id": app["client_id"], "refresh_token": t["refresh_token"]})
assert r.status_code == 403
r = client.get("/api/v1/auth/me", headers=_bearer(t))
assert r.status_code == 401 and r.headers["www-authenticate"] == "Bearer" # nowhere to sign in, so it names nothing
# people still see the apps they allowed and can revoke them
assert len(client.get(f"{O}/grants", headers=h).json()) == 1
# back on, the access people gave comes back with it
assert client.put("/api/v1/settings/tokens", headers=hr, json={"oauth_enabled": True}).status_code == 200
assert client.get("/api/v1/auth/me", headers=_bearer(t)).status_code == 200
Loading
Loading