Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
shows only Projects, Areas, Resources or Archive, and each note says which notes it's inside. Jump to (⌘K) finds notes
as well. `GET /api/v1/notes/search`; `find_notes` gives agents the words around each match
([Finding notes](docs/notes.md#finding-notes)).
- **Connect Google Drive by signing in with Google.** Sensors › Add connection › Google Drive › Google account now
has **Sign in with Google**: Google's page opens in a small window, and the connection is made, tested and named
after the account, ready to pick a folder whose files Lens imports. Nothing to run or paste. It uses your own
Google app, the one people can sign in to Lens with; set it up once in the dialog (client ID and secret, with the
redirect URI shown) or in Settings › Sign-in, and signing in to Lens with Google can stay off. Agents get the same
link from `POST /api/v1/sources/signin` to hand to their person. Pasting a token from rclone and service accounts
work as before ([Storage sources](docs/processing.md#storage-sources)).

## 0.10.0 <small>October 15, 2026</small> {id="0.10.0"}

Expand Down
4 changes: 4 additions & 0 deletions docs/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ has a "Continue with ..." button for each provider that's turned on.
Removing a provider disconnects the accounts connected through it. Audited as `login_provider.add`, `.change`,
`.remove`, `external.connect` and `external.disconnect`; sign-ins as `login` with `external:<name>`.

The Google app also connects Google Drive (Sensors › Add connection › Google Drive › Sign in with Google): the same
client id, secret and redirect URI, with Drive's scope asked for, so it works with signing in to Lens turned off
([Storage sources](processing.md#storage-sources)).

## The web app: NextAuth with API tokens

```
Expand Down
15 changes: 15 additions & 0 deletions docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,21 @@ files, then Import (`POST /api/v1/import/source`). The listing marks files that
- Credentials are encrypted in the database (AES-GCM, key from `ARCHIVE_SECRET_KEY` or `data_dir/secret.key`) and given
to rclone in a private temporary config file per call. For Dropbox, Drive and OneDrive, paste the token from
`rclone authorize dropbox` (or drive, onedrive); tokens rclone refreshes are saved back.
- **Google Drive signs in with Google** in the web app, with nothing to paste: choose Google account, then Sign in
with Google. A small window opens Google's consent page; when Google sends it back, the connection is made, tested
and named after the account (Google Drive · you@gmail.com), and the dialog goes on to pick a folder to watch. It
uses your own Google app, the one people can sign in to Lens with (Settings › Sign-in): set it up once, in the
dialog or there, with signing in to Lens left off if you like. In Google Cloud console › APIs & Services, enable
the Google Drive API; create an OAuth client ID (Web application) whose redirect URI is the address the dialog
shows (`<your Lens address>/api/v1/auth/external/google/callback`, the same as sign-in's); and publish the app on
the OAuth consent screen, or Google ends its sign-ins after 7 days. Google accepts `https://` redirect addresses
and `http://localhost`. The token is kept encrypted, and rclone renews it with that app's client id and secret.
A connection whose sign-in ran out shows Sign in again. Without a Google app, paste a token from rclone as before.
- Agents connect Drive the same way: `POST /api/v1/sources/signin` with `{"type": "drive", "params": {...}}` (or
`{"source": <id>}` to sign one in again) answers `{url}`. From the web app the link is tied to that browser; from
an API token or app it isn't, so an agent can hand it to its person, who opens it once within ten minutes and ends
on a page saying it's connected (`/connected`). Then `POST /api/v1/watches` imports its files. Code that calls
Google itself gets a fresh access token from `source_oauth.fresh_token`. Audited as `source.signin`.
- Google Drive can also sign in as a service account: choose Service account and upload its JSON key file
(`service_account_credentials`). Share the folders Lens should read with the account's address, or give a shared
drive ID (`team_drive`); in Google Workspace, `impersonate` reads a user's own Drive through domain-wide delegation.
Expand Down
34 changes: 31 additions & 3 deletions fastapi_backend/app/api/v1/routes/external.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
cookie tying the sign-in to this browser. The provider sends it back to the callback, which ends on the web app's
/external-signin page with a one-time ticket in the URL fragment; the web app's server swaps it for a session at
POST /auth/ticket, as for passkeys.

Connecting a storage source by signing in (source_oauth.py, POST /sources/signin) comes back through the same callback,
so the app registered with the provider needs no second address; it ends on the web app's page it started from, with
?connected=<id> or ?connect_error=<why>.
"""

from __future__ import annotations
Expand All @@ -15,7 +19,7 @@
from fastapi.responses import RedirectResponse

from app.api.deps import AdminWriter, Cfg, CurrentUser, Db, Writer, domain_errors, web_origin
from app.domain import auth, external_login, passkeys
from app.domain import auth, external_login, passkeys, source_oauth
from app.schemas.auth import (
ExternalIdentity,
ExternalProvider,
Expand All @@ -30,7 +34,8 @@
COOKIE = "lens_external"


def _cookie(response: Response, origin: str, value: str) -> None:
def flow_cookie(response: Response, origin: str, value: str) -> None:
"""The cookie that ties a round trip through a provider to the browser that started it."""
response.set_cookie(
COOKIE,
value,
Expand All @@ -50,7 +55,7 @@ def _redirect(response: Response, request: Request, db: Db, cfg: Cfg, key: str,
raise HTTPException(404, "no such sign-in provider") from None
except ValueError as e:
raise HTTPException(400, str(e)) from None
_cookie(response, origin, browser)
flow_cookie(response, origin, browser)
return ExternalRedirect(url=url)


Expand Down Expand Up @@ -96,6 +101,8 @@ def external_callback(
flow = external_login.claim(db, state, request.cookies.get(COOKIE))
except ValueError as e:
return _back(web_origin(request), error=str(e), next="/login")
if flow.get("mode") == "source":
return _source_back(db, cfg, flow, code, error, error_description)
origin, connect = flow["origin"], flow.get("mode") == "connect"
away = "/account" if connect else "/login"
if error or not code:
Expand All @@ -114,6 +121,27 @@ def external_callback(
return _back(origin, ticket=passkeys.issue_ticket(db, u["id"], f"external:{key}"), next=flow.get("next") or "/")


def _source_back(db: Db, cfg: Cfg, flow: dict, code: str, error: str, error_description: str) -> RedirectResponse:
"""A source signed in (or not): back to the page it started from, with ?connected=<id> or ?connect_error=<why>."""
if error or not code:
why = "you cancelled" if error == "access_denied" else (error_description or error or "no code came back")
q = {"connect_error": f"signing in didn't finish: {why}"}
else:
try:
sid = source_oauth.finish(db, cfg, flow, code)
except ValueError as e:
q = {"connect_error": str(e)}
else:
q = {"connected": str(sid)}
auth.audit(
db, {"id": flow.get("account"), "email": (flow.get("source") or {}).get("by")}, "source.signin", f"storage_source:{sid}"
)
page = flow.get("next") or "/sources"
r = RedirectResponse(f"{flow['origin']}{page}{'&' if '?' in page else '?'}{urllib.parse.urlencode(q)}", status_code=303)
r.delete_cookie(COOKIE, path="/api/v1/auth/external")
return r


# ---------- your outside accounts ----------
def _signed_in(user: CurrentUser) -> None:
if user.via != "access":
Expand Down
51 changes: 45 additions & 6 deletions fastapi_backend/app/api/v1/routes/sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@
import logging
from collections import Counter

from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response

from app.api.deps import Acl, AdminReader, AdminWriter, Cfg, CurrentUser, Db
from app.domain import auth, sources, store
from app.api.deps import Acl, AdminReader, AdminWriter, Cfg, CurrentUser, Db, web_origin
from app.api.v1.routes.external import flow_cookie
from app.domain import auth, source_oauth, sources, store
from app.domain.store import DB
from app.schemas.admin import Started
from app.schemas.common import Created, Ok
Expand All @@ -21,6 +22,8 @@
SourceCreate,
SourceCreated,
SourceHealth,
SourceSignIn,
SourceSignInLink,
SourceUpdate,
SourceUpdated,
Watch,
Expand All @@ -47,10 +50,13 @@ def _watch(db: DB, wid: int) -> None:


@router.get("/sources/backends")
def list_backends(user: CurrentUser) -> dict[str, Backend]:
"""The kinds of storage a source can be, with their settings and secrets."""
def list_backends(user: CurrentUser, db: Db) -> dict[str, Backend]:
"""The kinds of storage a source can be, with their settings and secrets, and how each signs in in the browser."""
keys = {"label", "fields", "secrets", "oauth"}
return {k: Backend.model_validate({f: x for f, x in v.items() if f in keys}) for k, v in sources.BACKENDS.items()}
return {
k: Backend.model_validate({**{f: x for f, x in v.items() if f in keys}, "signin": source_oauth.signin(db, k)})
for k, v in sources.BACKENDS.items()
}


@router.get("/sources")
Expand Down Expand Up @@ -80,6 +86,39 @@ def update_source(sid: int, body: SourceUpdate, user: AdminWriter, db: Db, cfg:
return SourceUpdated(health=sources.test(db, cfg, sid))


@router.post("/sources/signin")
def start_signin(body: SourceSignIn, user: AdminWriter, request: Request, response: Response, db: Db, cfg: Cfg) -> SourceSignInLink:
"""Connect Google Drive by signing in with Google (through the Google app in Settings › Sign-in), or sign a
connection in again (`source`): open the returned address in a browser. The provider sends it back to the web app
page `next`, with ?connected=<id> (tested; next, watch one of its folders) or ?connect_error=<why>. Started from
the web app it's tied to that browser; from an API token or app it isn't, so an agent can hand the link to its
person. Either way it works once, within ten minutes. Audited as `source.signin` when it finishes."""
if body.source is None and body.type is None:
raise HTTPException(400, "say which type of connection to make, or which connection to sign in again")
origin, bound = web_origin(request), user.via == "access"
try:
url, browser = source_oauth.start(
db,
cfg,
origin,
body.type,
body.params,
body.name,
body.source,
account=user.id,
by=user.email,
bound=bound,
next_path=body.next,
)
except KeyError:
raise HTTPException(404, "not found") from None
except ValueError as e:
raise HTTPException(400, str(e)) from None
if browser:
flow_cookie(response, origin, browser)
return SourceSignInLink(url=url, bound=bool(browser))


@router.delete("/sources/{sid}")
def delete_source(sid: int, user: AdminWriter, db: Db) -> Ok:
"""Remove a source and its watched folders."""
Expand Down
17 changes: 13 additions & 4 deletions fastapi_backend/app/domain/external_login.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,11 @@ def callback_path(key):
return f"/api/v1/auth/external/{key}/callback"


def client_secret(cfg, row):
"""A provider's client secret, unsealed, for its token endpoint."""
return settings.unseal(cfg, row["client_secret"], _context(row["key"]))


# ---------- signing in ----------
def _endpoints(row):
kind = row["kind"]
Expand Down Expand Up @@ -246,7 +251,7 @@ def start(db, cfg, key, origin, mode="login", account=None, next_path="/"):
"origin": origin.rstrip("/"),
"mode": mode,
"account": account,
"next": _safe_next(next_path),
"next": safe_next(next_path),
"expires_at": _later(FLOW_MINUTES * 60),
}
),
Expand All @@ -268,19 +273,23 @@ def start(db, cfg, key, origin, mode="login", account=None, next_path="/"):
return ep["authorize"] + sep + urllib.parse.urlencode(q), browser


def _safe_next(path):
def safe_next(path):
"""A path on this site. Browsers drop tabs and line breaks from a URL ("/\\t/evil.example" is "//evil.example")."""
path = path or "/"
ok = path.startswith("/") and not path.startswith("//") and "\\" not in path and not re.search(r"[\x00-\x1f\x7f]", path)
return path if ok else "/"


def claim(db, state, browser):
"""The flow a callback answers, used up. ExternalLoginError when it's gone, expired, or started in another browser."""
"""The flow a callback answers, used up. ExternalLoginError when it's gone, expired, or started in another browser.
A flow an agent started through the API for its person to open (connecting a source, source_oauth.py) isn't tied
to a browser: its link alone finishes it, once."""
rows = db.rows("DELETE $r RETURN BEFORE", r=R("external_flow", auth.sha(state))) if state else []
row = rows[0] if rows else None
if not row or (row.get("expires_at") or "") < store.now():
raise ExternalLoginError("that sign-in took too long or was already used; try again")
if row.get("unbound") and row.get("mode") == "source":
return row
if not browser or auth.sha(browser) != row.get("browser"):
raise ExternalLoginError("that sign-in was started in another browser; start it again here")
return row
Expand All @@ -289,7 +298,7 @@ def claim(db, state, browser):
def _profile(row, cfg, flow, code):
"""{sub, email, trusted, name} from the provider, for the code it sent back."""
ep = _endpoints(row)
secret = settings.unseal(cfg, row["client_secret"], _context(row["key"]))
secret = client_secret(cfg, row)
try:
r = httpx.post(
ep["token"],
Expand Down
Loading
Loading