From ce40b69805feb28340664b54e6d2c7b9741e4bff Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 11:30:04 +0000 Subject: [PATCH 1/3] fix(clipper): sign in, or use your API token, not only an import webhook's The web clipper only took import webhook tokens (lih_...): pasting an API token (la_...) failed with "Lens didn't recognise this token: make a new import webhook and paste its token". - Lens: POST /api/v1/hooks/import also takes the person's own token (an API token with write access, an app's, or the web app's), with ?namespace= naming one they can import into (default: their only one). Audited as import.push, by the person. GET /api/v1/hooks/import says where a token's pushes go. 401s say which kind of token Lens didn't know; the address form still takes hook tokens only. - Clipper: Sign in in the options (OAuth with PKCE, the side panel's own sign-in, shared with it), or paste an API token or a hook token, told apart by their prefix. The popup offers each namespace the person can add to. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01NZxdXgdzoJfnSLju6B6Ygz --- docs/api.md | 21 ++- docs/web-clipper.md | 28 +++- extensions/browser/README.md | 3 +- extensions/browser/src/background.js | 68 +++++--- extensions/browser/src/core/lens.js | 85 ++++++++-- extensions/browser/src/core/settings.js | 22 ++- extensions/browser/src/ui/options.html | 27 ++-- extensions/browser/src/ui/options.js | 151 +++++++++++++++--- extensions/browser/src/ui/popup.js | 11 +- extensions/browser/test/lens.test.js | 98 ++++++++++-- .../app/api/v1/routes/import_hooks.py | 134 ++++++++++++++-- fastapi_backend/app/domain/import_hooks.py | 18 ++- fastapi_backend/app/schemas/import_hooks.py | 11 ++ .../tests/api/test_import_hooks.py | 89 ++++++++++- nextjs-frontend/app/openapi-client/index.ts | 4 + nextjs-frontend/app/openapi-client/sdk.gen.ts | 31 +++- .../app/openapi-client/types.gen.ts | 67 ++++++++ nextjs-frontend/openapi.json | 113 ++++++++++++- threat-model.md | 1 + 19 files changed, 857 insertions(+), 125 deletions(-) diff --git a/docs/api.md b/docs/api.md index eab222305..04ead274d 100644 --- a/docs/api.md +++ b/docs/api.md @@ -829,6 +829,7 @@ POST /api/v1/namespaces/{name}/import-hooks PATCH /api/v1/namespaces/{name}/import-hooks/{hid} POST /api/v1/namespaces/{name}/import-hooks/{hid}/token DELETE /api/v1/namespaces/{name}/import-hooks/{hid} +GET /api/v1/hooks/import POST /api/v1/hooks/import POST /api/v1/hooks/import/{token} ``` @@ -849,14 +850,28 @@ a URL; the address form ends up in proxy logs, so prefer the header). Three ways - **JSON** `{url?, text?, title?}`: a web page or PDF kept as a document, as [web pages](#web-pages) are (a page needs Chromium on the server; a link to a PDF doesn't), and text imported as a transcript, as pasted text is. -The answer (202) is `{namespace, items: [{kind, name, recording, job, duplicate}]}`. 401 for a token that isn't a hook's, -403 while the hook is paused, 413 over `uploads.max_mb`, 507 when the disk can't hold the file. Each item is audited as -`import.hook`, by `import_hook:` rather than a person. +The answer (202) is `{namespace, items: [{kind, name, recording, job, duplicate}]}`. 401 for a token Lens doesn't know, +saying which kind it took it for (a hook's, an API token, a sign-in), 403 while the hook is paused, 413 over +`uploads.max_mb`, 507 when the disk can't hold the file. Each item is audited as `import.hook`, by `import_hook:` +rather than a person. + +**Your own token works there too**, for the [web clipper](web-clipper.md), an agent or a script acting as you: an API +token with write access (`la_...`), an app's token (`lo_...`, with the write scope) or the web app's own, with +`?namespace=` naming one you can import into (the import permission; editors have it). Without `?namespace=`, it's +the only namespace you can import into, else 400 listing them. A read-only token gets 403. What you push this way is +yours, audited as `import.push`, and isn't counted on any hook. Your own token only goes in the `Authorization` +header: the address form takes hook tokens only. + +`GET /api/v1/hooks/import` says where a token's pushes go without importing anything: +`{kind: "hook" | "person", hook, email, namespace, namespaces, write}`. For a hook's token, `namespace` is the hook's; +for yours, `namespaces` lists every namespace you can import into and `namespace` is the one used without +`?namespace=` (null when there's a choice), and `write` is false for a read-only key. ``` curl -H "Authorization: Bearer $TOKEN" --data-binary @minutes.pdf "https://lens.example/api/v1/hooks/import?filename=minutes.pdf" curl -H "Authorization: Bearer $TOKEN" -F file=@call.m4a -F title="Call with Sam" https://lens.example/api/v1/hooks/import curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"url": "https://example.org/report.pdf"}' https://lens.example/api/v1/hooks/import +curl -H "Authorization: Bearer $LENS_API_TOKEN" -H "Content-Type: application/json" -d '{"text": "Notes", "title": "Notes"}' "https://lens.example/api/v1/hooks/import?namespace=pods" ``` ### Uploads diff --git a/docs/web-clipper.md b/docs/web-clipper.md index ac64952f0..84d05bf17 100644 --- a/docs/web-clipper.md +++ b/docs/web-clipper.md @@ -23,14 +23,26 @@ tables, strikethrough and task lists become GitHub-style Markdown. ## Setting it up -1. In Lens, open a namespace's admin page and make an **import webhook** (see [the API](api.md), Import webhooks). - Copy its token: it's shown once. -2. In the clipper's options, enter your Lens address and the token and press **Add and check**. The browser asks to - let the clipper talk to that address; the clipper then checks the token without importing anything. - -Add one per namespace you clip into, and pick between them in the popup. The tokens stay on this device; every other -setting syncs through your browser account. Plain `http://` addresses are only accepted on this computer or the local -network. +In the clipper's options, enter your Lens address and press **Sign in** (or Enter). The browser asks to let +the clipper talk to that address, then Lens shows its own page asking you (signed in with your passkey there) to allow +the clipper. That's all: nothing to copy or paste. Clips go into the namespace you pick in the options, and the popup +lets you pick another one you can add to. Signing in here signs the assistant's side panel in too, and the other way +round. The sign-in renews itself; removing the clipper's access in Lens, or signing out in the side panel, ends it. + +Instead of signing in, **Use a token instead** takes either of these, and the clipper tells them apart: + +- **Your API token** (`la_...`), made in Lens under Settings, API keys, with write access. Clips go into the namespace + you pick, as if you'd added them yourself. A read-only key can't send clips, and the clipper says so. +- **An import webhook's token** (`lih_...`), from a namespace's admin page (see [the API](api.md), Import webhooks). + Clips go into that hook's namespace and are counted and audited as the hook's, not yours. + +Safari can't show Lens's page from an extension, so there you paste your API token. + +The clipper checks the sign-in or token without importing anything and says what's wrong if Lens refuses it: a token +Lens doesn't know (revoked, replaced, run out, or from another Lens), a read-only key, or a Lens too old to take API +tokens (update it, or use an import webhook's token). Add as many Lenses as you like; tokens and the sign-in stay on +this device, and every other setting syncs through your browser account. Plain `http://` addresses are only accepted +on this computer or the local network. ## Ways to clip diff --git a/extensions/browser/README.md b/extensions/browser/README.md index f8ebe2986..cbd1caf53 100644 --- a/extensions/browser/README.md +++ b/extensions/browser/README.md @@ -1,7 +1,8 @@ # Lens web clipper A browser extension that clips pages, selections or any part of a page into Lens as Markdown, with the page's -metadata and images, through a namespace's import webhook. See [docs/web-clipper.md](../../docs/web-clipper.md). +metadata and images, signed in as you (or with your API token, or a namespace's import webhook token). See +[docs/web-clipper.md](../../docs/web-clipper.md). ```sh npm ci && npm test && npm run build # then Load unpacked: dist/chrome diff --git a/extensions/browser/src/background.js b/extensions/browser/src/background.js index b93851b6d..ccca368e5 100644 --- a/extensions/browser/src/background.js +++ b/extensions/browser/src/background.js @@ -1,6 +1,7 @@ // The extension's background: context menus, shortcuts, and every push to Lens (so a send finishes even when the // popup closes). The assistant lives in the side panel (ui/sidepanel.js); this only opens it. -import { fetchImages, push } from "./core/lens.js"; +import * as account from "./core/account.js"; +import { fetchImages, push, pushLink } from "./core/lens.js"; import * as settingsStore from "./core/settings.js"; const ext = globalThis.browser ?? globalThis.chrome; @@ -63,28 +64,55 @@ async function startPicker(tabId, intent) { } // ---------- sending ---------- -async function send(clip, destinationId) { - const settings = await settingsStore.load(); +const signInAgain = (dest) => new Error(`sign in to ${new URL(dest.server).host} again in the clipper's options`); + +// The person's sign-in for a signed-in destination (renewed first when it's about to run out), else the token kept on +// this device for it. +async function credentials(dest, namespace) { + const where = { server: dest.server, namespace: settingsStore.asPerson(dest) ? (namespace || dest.namespace) || undefined : undefined }; + if (dest.kind !== "signin") return { ...where, token: await settingsStore.token(dest.id) }; + if ((await account.load())?.server !== dest.server) throw signInAgain(dest); + try { + return { ...where, token: await account.accessToken() }; + } catch (e) { + throw e instanceof account.SignedOut ? signInAgain(dest) : e; + } +} + +/** `fn` with a destination's credentials; a signed-in one renews its sign-in once when Lens says it has run out. */ +async function withCredentials(dest, namespace, fn) { + const creds = await credentials(dest, namespace); + try { + return await fn(creds); + } catch (e) { + if (e.status !== 401 || dest.kind !== "signin") throw e; + let token; + try { + token = await account.accessToken({ force: true }); + } catch (again) { + throw again instanceof account.SignedOut ? signInAgain(dest) : again; + } + return fn({ ...creds, token }); + } +} + +function chosen(settings, destinationId) { const dest = settingsStore.destination(settings, destinationId); if (!dest) throw new Error("Add your Lens in the clipper's options first"); - const token = await settingsStore.token(dest.id); + return dest; +} + +async function send(clip, destinationId, namespace) { + const settings = await settingsStore.load(); + const dest = chosen(settings, destinationId); const images = settings.sendImages && clip.images?.length ? await fetchImages(clip.images, { max: settings.maxImages }) : []; - const answer = await push({ server: dest.server, token }, clip, { images, captureUrl: settings.captureOnServer }); + const answer = await withCredentials(dest, namespace, (creds) => push(creds, clip, { images, captureUrl: settings.captureOnServer })); return { ...answer, server: dest.server, destination: dest.name }; } async function sendLink(url, title) { - const settings = await settingsStore.load(); - const dest = settingsStore.destination(settings); - if (!dest) throw new Error("Add your Lens in the clipper's options first"); - const token = await settingsStore.token(dest.id); - const res = await fetch(`${dest.server}/api/v1/hooks/import`, { - method: "POST", - headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json" }, - body: JSON.stringify(title ? { url, title: title.slice(0, 200) } : { url }), - credentials: "omit", - }); - if (!res.ok) throw new Error((await res.json().catch(() => ({}))).detail ?? `Lens answered ${res.status}`); + const dest = chosen(await settingsStore.load()); + await withCredentials(dest, undefined, (creds) => pushLink(creds, url, title)); } // ---------- telling the user ---------- @@ -131,14 +159,12 @@ ext.contextMenus.onClicked.addListener(async (info, tab) => { return flag(tab.id, true, "link sent; Lens captures it"); } if (info.menuItemId === "clip-image") { - const settings = await settingsStore.load(); - const dest = settingsStore.destination(settings); - if (!dest) throw new Error("Add your Lens in the clipper's options first"); + const dest = chosen(await settingsStore.load()); const images = await fetchImages([info.srcUrl], { max: 1 }); if (!images.length) throw new Error("couldn't download that image"); const title = tab.title || info.srcUrl; const markdown = `![](${info.srcUrl})\n\nFrom [${title}](${info.pageUrl})\n`; - await push({ server: dest.server, token: await settingsStore.token(dest.id) }, { title, markdown, url: info.pageUrl }, { images }); + await withCredentials(dest, undefined, (creds) => push(creds, { title, markdown, url: info.pageUrl }, { images })); return flag(tab.id, true, "image sent"); } } catch (e) { @@ -168,7 +194,7 @@ ext.runtime.onMessage.addListener((msg, sender, reply) => { return false; } if (msg?.type === "send") { - send(msg.clip, msg.destination).then( + send(msg.clip, msg.destination, msg.namespace).then( (answer) => reply({ ok: true, answer }), (e) => reply({ ok: false, error: e.message }), ); diff --git a/extensions/browser/src/core/lens.js b/extensions/browser/src/core/lens.js index a0de3928f..8884e4054 100644 --- a/extensions/browser/src/core/lens.js +++ b/extensions/browser/src/core/lens.js @@ -1,23 +1,46 @@ -// Sending clips to a Lens server through its import webhooks (docs/api.md, Import webhooks): a namespace's owners make -// a hook in Lens, and its token is all the clipper needs. +// Sending clips to a Lens server's push address (docs/api.md, Import webhooks), with any of the tokens it takes: the +// person's own sign-in (account.js), an API token they made (la_...), or an import webhook's token (lih_...). With the +// person's own, the clip goes into the namespace they picked; a hook's goes into the hook's namespace. export const PUSH_PATH = "/api/v1/hooks/import"; const MAX_FILES = 20; // the server takes at most 20 files a push +const OLD_HOOKS_ONLY = "that token isn't an import hook's"; // what a Lens from before personal tokens says -async function explain(res) { +/** What kind of token this is, by its prefix: "hook", "key" (an API token) or "" (a sign-in, or not one of Lens's). */ +export const tokenKind = (token) => (/^lih_/.test(token ?? "") ? "hook" : /^la_/.test(token ?? "") ? "key" : ""); + +const UNKNOWN = { + hook: "Lens doesn't know this import webhook token: it was replaced or deleted. Sign in instead, or paste a new one", + key: "Lens doesn't know this API token: it was revoked, ran out, or is from another Lens. Sign in instead, or make a new one", + "": "Lens didn't recognise this sign-in: sign in again from the clipper's options", +}; + +async function explain(res, token) { let detail = ""; try { const body = await res.json(); - detail = typeof body.detail === "string" ? body.detail : JSON.stringify(body.detail ?? body); + detail = body.detail == null ? "" : typeof body.detail === "string" ? body.detail : JSON.stringify(body.detail); } catch { // not JSON } - if (res.status === 401) return "Lens didn't recognise this token: make a new import webhook and paste its token"; + if (res.status === 401) { + if (detail === OLD_HOOKS_ONLY && tokenKind(token) !== "hook") { + return "this Lens takes only import webhook tokens: update Lens to clip with your API token or by signing in"; + } + return detail && detail !== OLD_HOOKS_ONLY ? detail : UNKNOWN[tokenKind(token)]; + } if (res.status === 403) return detail || "this import webhook is paused in Lens"; if (res.status === 413) return detail || "that's more than this Lens takes at once"; return detail || `Lens answered ${res.status} ${res.statusText}`; } +const failed = async (res, token) => Object.assign(new Error(await explain(res, token)), { status: res.status }); + +/** The push address, with the namespace when the token is the person's own. */ +export function pushUrl(server, namespace) { + return `${server}${PUSH_PATH}${namespace ? `?namespace=${encodeURIComponent(namespace)}` : ""}`; +} + const filenameFor = (src, i, type) => { let name = ""; try { @@ -54,8 +77,8 @@ export async function fetchImages(urls, { max = 10, fetchImpl = fetch } = {}) { * Push a clip: its Markdown as text (Lens imports it like pasted text), plus, when asked, its images as files and its * address for Lens to capture itself. Resolves to the server's answer: {namespace, items: [{kind, name, recording}]}. */ -export async function push({ server, token }, { title, markdown, url }, { images = [], captureUrl = false, fetchImpl = fetch } = {}) { - if (!token) throw new Error("add this Lens's import webhook token in the clipper's options"); +export async function push({ server, token, namespace }, { title, markdown, url }, { images = [], captureUrl = false, fetchImpl = fetch } = {}) { + if (!token) throw new Error("sign in to this Lens in the clipper's options"); const headers = { Authorization: `Bearer ${token}`, Accept: "application/json" }; let body; if (!images.length && !captureUrl) { @@ -70,30 +93,62 @@ export async function push({ server, token }, { title, markdown, url }, { images } let res; try { - res = await fetchImpl(`${server}${PUSH_PATH}`, { method: "POST", headers, body, credentials: "omit" }); + res = await fetchImpl(pushUrl(server, namespace), { method: "POST", headers, body, credentials: "omit" }); } catch (e) { throw new Error(`couldn't reach ${server}: ${e.message}`); } - if (!res.ok) throw new Error(await explain(res)); + if (!res.ok) throw await failed(res, token); return res.json(); } -/** Check an address and token without importing anything: an empty push is refused with 400 once the token is known. */ -export async function check({ server, token }, { fetchImpl = fetch } = {}) { +/** Send a web address for Lens to capture itself (a PDF as it is, a page printed by Chromium on the full image). */ +export async function pushLink({ server, token, namespace }, url, title, { fetchImpl = fetch } = {}) { + if (!token) throw new Error("sign in to this Lens in the clipper's options"); let res; try { - res = await fetchImpl(`${server}${PUSH_PATH}`, { + res = await fetchImpl(pushUrl(server, namespace), { method: "POST", headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json", Accept: "application/json" }, - body: "{}", + body: JSON.stringify(title ? { url, title: title.slice(0, 200) } : { url }), credentials: "omit", }); + } catch (e) { + throw new Error(`couldn't reach ${server}: ${e.message}`); + } + if (!res.ok) throw await failed(res, token); + return res.json(); +} + +/** + * Check an address and token without importing anything. `target` is where clips go: {kind: "hook" | "person", + * namespace, namespaces, email, hook, write}. A Lens from before personal tokens has no GET there (405): an empty push + * is refused with 400 once its hook token is known. + */ +export async function check({ server, token }, { fetchImpl = fetch } = {}) { + const headers = { Authorization: `Bearer ${token}`, Accept: "application/json" }; + let res; + try { + res = await fetchImpl(`${server}${PUSH_PATH}`, { headers, credentials: "omit" }); + if (res.status === 405) { + res = await fetchImpl(`${server}${PUSH_PATH}`, { + method: "POST", + headers: { ...headers, "Content-Type": "application/json" }, + body: "{}", + credentials: "omit", + }); + if (res.status === 400) return { ok: true, message: "Connected: clips will go into this hook's namespace", target: { kind: "hook" } }; + } } catch (e) { return { ok: false, message: `couldn't reach ${server}: ${e.message}` }; } - if (res.status === 400) return { ok: true, message: "Connected: clips will go into this hook's namespace" }; if (res.status === 404) return { ok: false, message: "that address answers, but not as a Lens (no import webhooks there)" }; - return { ok: false, message: await explain(res) }; + if (!res.ok) return { ok: false, message: await explain(res, token) }; + const target = await res.json(); + if (target.kind === "hook") return { ok: true, message: `Connected: clips go into ${target.namespace}`, target }; + if (!target.write) return { ok: false, message: "this API token is read-only: make one with write access in Lens, Settings → API keys", target }; + if (!target.namespaces?.length) return { ok: false, message: "you can't add anything to a namespace in this Lens yet: ask an owner for the editor role", target }; + const into = target.namespaces.length > 1 ? `${target.namespace ?? target.namespaces[0]} (or pick another)` : target.namespaces[0]; + return { ok: true, message: `Connected${target.email ? ` as ${target.email}` : ""}: clips go into ${into}`, target }; } /** Where a pushed item can be opened in Lens. */ diff --git a/extensions/browser/src/core/settings.js b/extensions/browser/src/core/settings.js index f07675190..fa28d0ba3 100644 --- a/extensions/browser/src/core/settings.js +++ b/extensions/browser/src/core/settings.js @@ -18,7 +18,11 @@ tags: {tags|list} `; export const DEFAULTS = { - destinations: [], // [{id, name, server}]; each one's token is in local storage under token: + // [{id, name, server, kind, namespace?, namespaces?, email?}], kind "signin" (the person's sign-in, account.js), + // "key" (their API token) or "hook" (an import webhook's token; destinations from before kinds are hooks). A key or + // a hook's token is in local storage under token:. As the person, clips go into `namespace` (one of + // `namespaces`); with a hook's token, into the hook's namespace. + destinations: [], defaultDestination: "", template: DEFAULT_TEMPLATE, titleTemplate: "{title}", @@ -60,6 +64,22 @@ export function destination(settings, id) { return all.find((d) => d.id === (id || settings.defaultDestination)) ?? all[0] ?? null; } +/** Whether clips to a destination go as the person, into a namespace they pick (not a hook's own). */ +export const asPerson = (d) => d?.kind === "signin" || d?.kind === "key"; + +/** Every place a clip can go, for the popup: one per namespace of each destination, the default one first-chosen. */ +export function choices(settings) { + const fallback = destination(settings)?.id; + return (settings.destinations ?? []).flatMap((d) => { + const names = d.namespaces?.length ? d.namespaces : [d.namespace ?? ""]; + return names.map((ns) => ({ + value: JSON.stringify([d.id, ns]), + label: ns ? `${d.name || d.server} · ${ns}` : d.name || d.server, + selected: d.id === fallback && (ns === (d.namespace ?? "") || names.length === 1), + })); + }); +} + // plain http is fine on this computer or the local network, nowhere else export const isLocal = (host) => host === "localhost" || diff --git a/extensions/browser/src/ui/options.html b/extensions/browser/src/ui/options.html index c3ad81eaa..d807651e0 100644 --- a/extensions/browser/src/ui/options.html +++ b/extensions/browser/src/ui/options.html @@ -10,6 +10,8 @@ #selectionTemplate { height: 120px; } .dest { border: 1px solid var(--line); border-radius: 8px; padding: 10px 12px; margin: 8px 0; } .dest .row { flex-wrap: wrap; } + .dest select { width: auto; } + [hidden] { display: none !important; } .grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(200px, 1fr)); gap: 0 16px; } code { font-size: 12px; } dl { display: grid; grid-template-columns: max-content 1fr; gap: 2px 12px; font-size: 12px; color: var(--muted); } @@ -22,19 +24,24 @@

Lens clipper

Your Lens

- In Lens, open a namespace's admin page and make an import webhook; paste its address and token here. Clips go into that - namespace. Add one per namespace you clip into. + Enter your Lens address and sign in: Lens asks you to allow the clipper, and clips go into the namespace you pick. Signing in here + signs the assistant's side panel in too.

-
-
-
-
+
+ + + - - -

-
+

+ + + +

+

Template

Variables in braces, with filters after a bar: {title|kebab}, dates with a format: {date:YYYY-MM-DD}.

diff --git a/extensions/browser/src/ui/options.js b/extensions/browser/src/ui/options.js index 03c9ec106..28e1dc91d 100644 --- a/extensions/browser/src/ui/options.js +++ b/extensions/browser/src/ui/options.js @@ -1,5 +1,6 @@ // The options page: where clips go, the template, Markdown style and clipping choices, and what the assistant may do // in this browser. Changes save as they're made. +import * as account from "../core/account.js"; import * as browserCtl from "../core/browser.js"; import * as settingsStore from "../core/settings.js"; import { check } from "../core/lens.js"; @@ -25,6 +26,32 @@ async function patch(p) { const originPattern = (server) => `${new URL(server).origin}/*`; // ---------- destinations ---------- +// Each is one Lens, reached as the person (signed in, or with their API token: clips go into the namespace picked here) +// or with an import webhook's token (clips go into the hook's namespace). +const KINDS = { signin: "Signed in", key: "API token", hook: "Import webhook" }; + +/** The token a destination sends: the person's sign-in for a signed-in one, else the one kept for it on this device. */ +async function tokenFor(d) { + if (d.kind !== "signin") return settingsStore.token(d.id); + if ((await account.load())?.server !== d.server) return ""; + try { + return await account.accessToken(); + } catch { + return ""; + } +} + +async function saveDestination(d) { + await patch({ destinations: settings.destinations.map((x) => (x.id === d.id ? d : x)) }); +} + +/** What a check found, kept on the destination: where its clips can go. */ +const placed = (d, target) => { + if (!target || target.kind === "hook" || !target.namespaces?.length) return { ...d, namespace: target?.namespace ?? d.namespace }; + const namespace = target.namespaces.includes(d.namespace) ? d.namespace : (target.namespace ?? target.namespaces[0]); + return { ...d, namespace, namespaces: target.namespaces }; +}; + function renderDestinations() { const box = $("destinations"); box.replaceChildren( @@ -40,23 +67,52 @@ function renderDestinations() { strong.textContent = d.name || "Lens"; const where = document.createElement("div"); where.className = "hint"; - where.textContent = d.server; + where.textContent = [d.server, KINDS[d.kind ?? "hook"], d.email].filter(Boolean).join(" · "); name.append(strong, isDefault ? " (default)" : "", where); const state = document.createElement("span"); state.className = "status"; const button = (label, fn) => { const b = document.createElement("button"); + b.type = "button"; b.textContent = label; b.addEventListener("click", fn); return b; }; + const into = []; + if ((d.namespaces?.length ?? 0) > 1) { + const pick = document.createElement("select"); + pick.setAttribute("aria-label", `Namespace clips go into on ${d.name}`); + pick.title = "The namespace clips go into (the popup lets you pick another)"; + for (const ns of d.namespaces) pick.append(new Option(ns, ns, false, ns === d.namespace)); + pick.addEventListener("change", () => saveDestination({ ...d, namespace: pick.value }).then(renderDestinations)); + into.push(pick); + } else if (d.namespace) { + const label = document.createElement("span"); + label.className = "hint"; + label.textContent = `into ${d.namespace}`; + into.push(label); + } row.append( name, + ...into, button("Check", async () => { + state.className = "status"; state.textContent = "Checking…"; - const r = await check({ server: d.server, token: await settingsStore.token(d.id) }); + const token = await tokenFor(d); + if (!token) { + state.className = "status bad"; + state.textContent = "Signed out: sign in again below"; + $("newServer").value = d.server; + $("add").focus(); + return; + } + const r = await check({ server: d.server, token }); state.className = `status ${r.ok ? "ok" : "bad"}`; state.textContent = r.message; + if (r.ok && d.kind !== "hook") { + await saveDestination(placed(d, r.target)); + renderDestinations(); + } }), ...(isDefault ? [] : [button("Make default", () => patch({ defaultDestination: d.id }).then(renderDestinations))]), button("Remove", async () => { @@ -72,42 +128,80 @@ function renderDestinations() { ); } -$("add").addEventListener("click", () => { - const out = $("addStatus"); +function addStatus(text, kind = "") { + $("addStatus").className = `status grow ${kind}`; + $("addStatus").textContent = text; +} + +function showTokenRow(show) { + $("tokenRow").hidden = !show; + $("useToken").hidden = show || !account.canOAuth(); + $("add").textContent = !account.canOAuth() || $("newToken").value.trim() ? "Add and check" : "Sign in"; +} + +$("useToken").addEventListener("click", () => { + showTokenRow(true); + $("newToken").focus(); +}); +$("newToken").addEventListener("input", () => showTokenRow(true)); + +/** Add a Lens, or replace the one there was for the same address and kind (signing in again, a new token). */ +async function addDestination(d, token) { + const before = settings.destinations.find((x) => x.server === d.server && (x.kind ?? "hook") === d.kind && (d.kind !== "hook" || x.namespace === d.namespace)); + const id = before?.id ?? crypto.randomUUID(); + if (token) await settingsStore.setToken(id, token); + const dest = { ...before, ...d, id }; + const destinations = before ? settings.destinations.map((x) => (x.id === id ? dest : x)) : [...settings.destinations, dest]; + await patch({ destinations, defaultDestination: settings.defaultDestination || id }); + renderDestinations(); +} + +$("addForm").addEventListener("submit", (e) => { + e.preventDefault(); let server; try { server = settingsStore.normalizeServer($("newServer").value); - } catch (e) { - out.className = "status bad"; - out.textContent = e.message; + } catch (err) { + addStatus(err.message, "bad"); return; } const token = $("newToken").value.trim(); - if (!token) { - out.className = "status bad"; - out.textContent = "paste the import webhook's token"; + if (!token && !account.canOAuth()) { + addStatus("paste an API token with write access: in Lens, Settings → API keys", "bad"); return; } // asked for straight away, while the click still counts as the user's: browsers only ask from a click const asked = ext.permissions.request({ origins: [originPattern(server)] }); (async () => { - if (!(await asked)) { - out.className = "status bad"; - out.textContent = "the clipper needs your permission to talk to that address"; - return; + if (!(await asked)) return addStatus("the clipper needs your permission to talk to that address", "bad"); + $("add").disabled = true; + try { + const name = new URL(server).host; + if (token) { + addStatus("Checking…"); + const r = await check({ server, token }); + addStatus(r.message, r.ok ? "ok" : "bad"); + if (!r.ok) return; + const kind = r.target?.kind === "person" ? "key" : "hook"; + await addDestination(placed({ name, server, kind, email: r.target?.email ?? undefined }, r.target), token); + } else { + const me = await account.load(); + if (me?.server !== server || me.apiToken) { + addStatus("Signing in: Lens asks you to allow the clipper…"); + await account.signIn(server); + } + const r = await check({ server, token: await account.accessToken() }); + addStatus(r.message, r.ok ? "ok" : "bad"); + if (!r.ok) return; + await addDestination(placed({ name, server, kind: "signin", email: r.target?.email ?? undefined }, r.target)); + } + $("newServer").value = $("newToken").value = ""; + showTokenRow(!account.canOAuth()); + } catch (err) { + addStatus(`Couldn't sign in: ${err.message}`, "bad"); + } finally { + $("add").disabled = false; } - out.className = "status"; - out.textContent = "Checking…"; - const r = await check({ server, token }); - out.className = `status ${r.ok ? "ok" : "bad"}`; - out.textContent = r.message; - if (!r.ok) return; - const id = crypto.randomUUID(); - await settingsStore.setToken(id, token); - const destinations = [...settings.destinations, { id, name: $("newName").value.trim() || new URL(server).host, server }]; - await patch({ destinations, defaultDestination: settings.defaultDestination || id }); - $("newName").value = $("newServer").value = $("newToken").value = ""; - renderDestinations(); })(); }); @@ -205,6 +299,11 @@ $("agentEnabled").addEventListener("change", async () => { settings = await settingsStore.load(); fill(); renderDestinations(); + showTokenRow(!account.canOAuth()); + // the first thing left to do has focus: the address, or with a sign-in already here (the side panel's), Enter adds it + const me = await account.load(); + if (me && !me.apiToken && !settings.destinations.some((d) => d.kind === "signin" && d.server === me.server)) $("newServer").value = me.server; + if (!settings.destinations.length) ($("newServer").value ? $("add") : $("newServer")).focus(); $("agentEnabled").checked = await browserCtl.enabled(); renderSites(); })(); diff --git a/extensions/browser/src/ui/popup.js b/extensions/browser/src/ui/popup.js index 1570b4988..83f973168 100644 --- a/extensions/browser/src/ui/popup.js +++ b/extensions/browser/src/ui/popup.js @@ -46,11 +46,11 @@ async function destinations() { const s = await settingsStore.load(); const sel = $("destination"); sel.replaceChildren( - ...s.destinations.map((d) => { + ...settingsStore.choices(s).map((c) => { const o = document.createElement("option"); - o.value = d.id; - o.textContent = d.name || d.server; - o.selected = d.id === s.defaultDestination; + o.value = c.value; + o.textContent = c.label; + o.selected = c.selected; return o; }), ); @@ -67,7 +67,8 @@ $("send").addEventListener("click", async () => { if (!clip) return; $("send").disabled = true; status("Sending…"); - const answer = await ext.runtime.sendMessage({ type: "send", clip: current(), destination: $("destination").value }); + const [destination, into] = JSON.parse($("destination").value); + const answer = await ext.runtime.sendMessage({ type: "send", clip: current(), destination, namespace: into || undefined }); $("send").disabled = false; if (!answer?.ok) return status(answer?.error ?? "Couldn't send it", "bad"); const { server, namespace, items } = answer.answer; diff --git a/extensions/browser/test/lens.test.js b/extensions/browser/test/lens.test.js index bc7c769c0..e254eaca2 100644 --- a/extensions/browser/test/lens.test.js +++ b/extensions/browser/test/lens.test.js @@ -1,7 +1,7 @@ import assert from "node:assert/strict"; import { test } from "node:test"; -import { check, fetchImages, push } from "../src/core/lens.js"; -import { normalizeServer } from "../src/core/settings.js"; +import { check, fetchImages, push, pushLink, tokenKind } from "../src/core/lens.js"; +import { asPerson, choices, normalizeServer } from "../src/core/settings.js"; const reply = (status, body = {}, type = "application/json") => new Response(JSON.stringify(body), { status, headers: { "content-type": type } }); @@ -32,16 +32,73 @@ test("images and server capture go as multipart", async () => { assert.equal(body.getAll("file").length, 1); }); -test("errors say what to do", async () => { - const fetchImpl = async () => reply(401, { detail: "unknown" }); - await assert.rejects(push({ server: "https://l", token: "t" }, { title: "T", markdown: "m" }, { fetchImpl }), /didn't recognise this token/); - await assert.rejects(push({ server: "https://l", token: "" }, { title: "T", markdown: "m" }, { fetchImpl }), /token/); +test("with the person's own token, the clip names its namespace", async () => { + let seen; + const fetchImpl = async (url, init) => { + seen = { url, init }; + return reply(202, { namespace: "home life", items: [] }); + }; + await push({ server: "https://lens.test", token: "la_key", namespace: "home life" }, { title: "T", markdown: "m" }, { fetchImpl }); + assert.equal(seen.url, "https://lens.test/api/v1/hooks/import?namespace=home%20life"); + assert.equal(seen.init.headers.Authorization, "Bearer la_key"); + await pushLink({ server: "https://lens.test", token: "la_key", namespace: "pods" }, "https://x.test/a.pdf", "A", { fetchImpl }); + assert.equal(seen.url, "https://lens.test/api/v1/hooks/import?namespace=pods"); + assert.deepEqual(JSON.parse(seen.init.body), { url: "https://x.test/a.pdf", title: "A" }); +}); + +test("errors say what to do, by the kind of token", async () => { + const says = async (token, status, detail) => { + const fetchImpl = async () => reply(status, detail === undefined ? {} : { detail }); + const err = await push({ server: "https://l", token }, { title: "T", markdown: "m" }, { fetchImpl }).catch((e) => e); + return err; + }; + // Lens says why, and that's shown + assert.match((await says("la_x", 401, "Lens doesn't know this API token: it was revoked")).message, /revoked/); + // without a reason, the clipper says it for the kind of token + assert.match((await says("la_x", 401)).message, /doesn't know this API token.*Sign in instead/); + assert.match((await says("lih_x", 401)).message, /import webhook token.*Sign in instead/); + assert.match((await says("access", 401)).message, /sign in again/); + assert.equal((await says("access", 401)).status, 401); + // a Lens from before personal tokens refuses an API token: update it + assert.match((await says("la_x", 401, "that token isn't an import hook's")).message, /update Lens/); + assert.match((await says("lih_x", 401, "that token isn't an import hook's")).message, /replaced or deleted/); + assert.match((await says("la_x", 403, "this API token is read-only")).message, /read-only/); + await assert.rejects(push({ server: "https://l", token: "" }, { title: "T", markdown: "m" }, { fetchImpl: async () => reply(202) }), /sign in/); + assert.equal(tokenKind("lih_a"), "hook"); + assert.equal(tokenKind("la_a"), "key"); + assert.equal(tokenKind("eyJ"), ""); +}); + +test("check asks Lens where the token's clips go", async () => { + const at = (status, body) => ({ fetchImpl: async (_url, init) => (init?.method === "POST" ? reply(500) : reply(status, body)) }); + const person = { kind: "person", email: "ed@x.io", namespace: null, namespaces: ["calls", "pods"], write: true }; + let r = await check({ server: "https://l", token: "la_x" }, at(200, person)); + assert.equal(r.ok, true); + assert.match(r.message, /as ed@x\.io: clips go into calls \(or pick another\)/); + assert.deepEqual(r.target.namespaces, ["calls", "pods"]); + r = await check({ server: "https://l", token: "la_x" }, at(200, { ...person, namespace: "pods", namespaces: ["pods"] })); + assert.match(r.message, /clips go into pods$/); + r = await check({ server: "https://l", token: "la_x" }, at(200, { ...person, write: false })); + assert.equal(r.ok, false); + assert.match(r.message, /read-only/); + r = await check({ server: "https://l", token: "la_x" }, at(200, { ...person, namespaces: [] })); + assert.equal(r.ok, false); + assert.match(r.message, /editor role/); + r = await check({ server: "https://l", token: "lih_x" }, at(200, { kind: "hook", hook: "Clipper", namespace: "pods", namespaces: ["pods"], write: true })); + assert.equal(r.ok, true); + assert.match(r.message, /into pods/); + r = await check({ server: "https://l", token: "la_x" }, at(401, { detail: "Lens doesn't know this API token" })); + assert.equal(r.ok, false); + assert.match(r.message, /doesn't know this API token/); + assert.match((await check({ server: "https://l", token: "t" }, at(404))).message, /not as a Lens/); }); -test("check reads 400 as a known token", async () => { - assert.equal((await check({ server: "https://l", token: "t" }, { fetchImpl: async () => reply(400, { detail: "send a url" }) })).ok, true); - assert.equal((await check({ server: "https://l", token: "t" }, { fetchImpl: async () => reply(401) })).ok, false); - assert.match((await check({ server: "https://l", token: "t" }, { fetchImpl: async () => reply(404) })).message, /not as a Lens/); +test("check on a Lens from before personal tokens: an empty push, refused once the hook token is known", async () => { + const old = (push) => ({ fetchImpl: async (_url, init) => (init?.method === "POST" ? push : reply(405, { detail: "Method Not Allowed" })) }); + assert.equal((await check({ server: "https://l", token: "lih_x" }, old(reply(400, { detail: "send a url" })))).ok, true); + const r = await check({ server: "https://l", token: "la_x" }, old(reply(401, { detail: "that token isn't an import hook's" }))); + assert.equal(r.ok, false); + assert.match(r.message, /update Lens/); }); test("images: only real images, named sensibly, failures skipped", async () => { @@ -61,3 +118,24 @@ test("server addresses", () => { assert.throws(() => normalizeServer("http://lens.example.com"), /https/); assert.throws(() => normalizeServer("http://localhost.evil.com"), /https/); }); + +test("the popup offers each namespace a signed-in Lens can take, the picked one chosen", () => { + const settings = { + defaultDestination: "a", + destinations: [ + { id: "a", name: "lens.test", server: "https://lens.test", kind: "signin", namespace: "pods", namespaces: ["calls", "pods"] }, + { id: "b", name: "Scanner", server: "https://lens.test", kind: "hook", namespace: "inbox" }, + { id: "c", name: "Old one", server: "https://old.test" }, // from before kinds: a hook + ], + }; + assert.deepEqual( + choices(settings).map((c) => [JSON.parse(c.value), c.label, c.selected]), + [ + [["a", "calls"], "lens.test · calls", false], + [["a", "pods"], "lens.test · pods", true], + [["b", "inbox"], "Scanner · inbox", false], + [["c", ""], "Old one", false], + ], + ); + assert.deepEqual(settings.destinations.map(asPerson), [true, false, false]); +}); diff --git a/fastapi_backend/app/api/v1/routes/import_hooks.py b/fastapi_backend/app/api/v1/routes/import_hooks.py index fd4c71bae..ce0f2ac67 100644 --- a/fastapi_backend/app/api/v1/routes/import_hooks.py +++ b/fastapi_backend/app/api/v1/routes/import_hooks.py @@ -1,5 +1,6 @@ """Import webhooks (docs/api.md, Import webhooks): a namespace's owners make and revoke them; other services push -files, web addresses or text to them with the hook's token, which is all they need.""" +files, web addresses or text to them with the hook's token, which is all they need. A person (or their agent, or the +web clipper signed in as them) pushes to the same address with their own token, naming the namespace.""" from __future__ import annotations @@ -18,8 +19,8 @@ from starlette.datastructures import UploadFile from starlette.requests import ClientDisconnect -from app.api.deps import Acl, Cfg, CurrentUser, Db, Writer, domain_errors -from app.domain import auth, import_hooks, uploads +from app.api.deps import Access, Acl, Cfg, CurrentUser, Db, Writer, domain_errors +from app.domain import auth, import_hooks, store, uploads from app.schemas.common import Ok from app.schemas.import_hooks import ( HookItem, @@ -30,6 +31,7 @@ ImportHookCreated, ImportHookToken, ImportHookUpdate, + ImportTarget, ) router = APIRouter(tags=["import hooks"]) @@ -144,15 +146,71 @@ def _bearer(request: Request) -> str | None: return h[7:].strip() if h.lower().startswith("bearer ") else None +UNKNOWN = { + import_hooks.TOKEN_PREFIX: "Lens doesn't know this import webhook token: it was replaced or deleted. Make a new one, or " + "use your own API token instead", + "la_": "Lens doesn't know this API token: it was revoked, ran out, or is from another Lens", + "lo_": "this sign-in has ended: sign in again", +} +PERSONAL = ("la_", "lo_") # a person's own tokens (and the web app's session) go in the header, never the address + + +def _unknown(raw: str | None) -> HTTPException: + """401 saying what was wrong with the token, by its kind.""" + if not raw: + why = "send an import webhook token (lih_...) or your own API token (la_...) as Authorization: Bearer" + else: + why = next((m for p, m in UNKNOWN.items() if raw.startswith(p)), "") + why = why or "Lens doesn't know this token: sign in again, or send an API token (la_...) or an import webhook token" + return HTTPException(401, why, headers={"WWW-Authenticate": "Bearer"}) + + def _hook(db: Any, token: str | None) -> dict[str, Any]: try: return import_hooks.by_token(db, token) except KeyError: - raise HTTPException(401, "that token isn't an import hook's") from None + raise _unknown(token) from None except import_hooks.Paused as e: raise HTTPException(403, str(e)) from None +def _importable(db: Any, acl: Access) -> list[str]: + """The namespaces this person can import into, by name.""" + names = store.space_names(db) + return sorted(names[s] for s in acl.spaces() if s in names and acl.can(s, "import")) + + +def _person(db: Any, acl: Access, raw: str | None, namespace: str | None) -> dict[str, Any]: + """Where a person's own push goes: the namespace they named, or the only one they can import into.""" + user = acl.user + if not user: + raise _unknown(raw) + if not user.can_write: + raise HTTPException(403, "this API token is read-only: make one with write access in Settings, API keys") + if namespace: + sid = acl.namespace(namespace, "import") + else: + able = _importable(db, acl) + if not able: + raise HTTPException(403, "you can't import into any namespace: ask one of its owners for the editor role") + if len(able) > 1: + raise HTTPException(400, f"name the namespace with ?namespace=: you can import into {', '.join(able)}") + sid = store.ns_id(db, able[0], create=False) + return import_hooks.for_person(db, sid, user.as_audit()) + + +def _target(db: Any, acl: Access, raw: str | None, in_path: bool, namespace: str | None) -> dict[str, Any]: + """What a push goes to: the import webhook its token belongs to, or the person whose token it is.""" + if in_path and raw and raw.startswith(PERSONAL): + raise HTTPException(401, "only an import webhook's token goes in the address: send your own token as Authorization: Bearer") + if in_path or (raw or "").startswith(import_hooks.TOKEN_PREFIX): + hook = _hook(db, raw) + if namespace and namespace != hook["namespace"]: + raise HTTPException(400, f"this import webhook imports into {hook['namespace']}: leave out ?namespace=") + return hook + return _person(db, acl, raw, namespace) + + def _disposition_name(value: str | None) -> str | None: if not value: return None @@ -191,8 +249,17 @@ async def _upload_chunks(up: UploadFile) -> Any: yield piece -async def _push(request: Request, db: Any, cfg: Any, token: str | None, filename: str | None, title: str | None) -> HookPushed: - hook = await run_in_threadpool(_hook, db, token or _bearer(request)) +async def _push( + request: Request, + db: Any, + cfg: Any, + acl: Access, + token: str | None, + filename: str | None, + title: str | None, + namespace: str | None, +) -> HookPushed: + hook = await run_in_threadpool(_target, db, acl, token or _bearer(request), token is not None, namespace) ctype = (request.headers.get("content-type") or "").split(";")[0].strip().lower() items: list[dict[str, Any]] = [] try: @@ -247,22 +314,57 @@ async def _push(request: Request, db: Any, cfg: Any, token: str | None, filename raise HTTPException(400, "the upload broke off; send it again") from None finally: for it in items: - detail = {"hook": hook["name"], "namespace": hook["namespace"], "kind": it["kind"], "name": it["name"]} - await run_in_threadpool(auth.audit, db, import_hooks.actor(hook), "import.hook", f"recording:{it['recording']}", detail) + detail = {"namespace": hook["namespace"], "kind": it["kind"], "name": it["name"]} + action = "import.push" if hook.get("person") else "import.hook" + if not hook.get("person"): + detail = {"hook": hook["name"], **detail} + await run_in_threadpool(auth.audit, db, import_hooks.actor(hook), action, f"recording:{it['recording']}", detail) if items: request.app.state.graph_cache.clear() return HookPushed(namespace=hook["namespace"], items=[HookItem.model_validate(it) for it in items]) +NAMESPACE = Query( + None, + max_length=40, + description="with your own token: the namespace it goes into (default: the only one you can import into)", +) + + +@router.get("/hooks/import") +def import_target(request: Request, acl: Acl, db: Db) -> ImportTarget: + """Where pushes with this bearer token go, without importing anything: an import webhook's token goes into its + namespace; a person's own token (an API token, an app's, or the web app's) into any namespace they can import into, + named with `?namespace=`. 401 for a token Lens doesn't know, saying which kind it is; 403 while a hook is paused.""" + raw = _bearer(request) + if (raw or "").startswith(import_hooks.TOKEN_PREFIX): + hook = _hook(db, raw) + return ImportTarget(kind="hook", hook=hook["name"], namespace=hook["namespace"], namespaces=[hook["namespace"]], write=True) + if not acl.user: + raise _unknown(raw) + able = _importable(db, acl) + return ImportTarget( + kind="person", + email=acl.user.email, + namespace=able[0] if len(able) == 1 else None, + namespaces=able, + write=acl.user.can_write, + ) + + @router.post("/hooks/import", status_code=202, openapi_extra=FILE_BODY) async def push_to_import_hook( request: Request, db: Db, cfg: Cfg, + acl: Acl, filename: str | None = Query(None, max_length=1000, description="the file's name, for a raw body"), title: str | None = Query(None, max_length=200, description="its title (default: the file's name, or the page's)"), + namespace: str | None = NAMESPACE, ) -> HookPushed: - """Push something into an import webhook's namespace, with its token as a bearer token. Three ways: + """Push something into an import webhook's namespace, with its token as a bearer token, or into one of your own + namespaces with your own token (an API token with write access, an app's, or the web app's) and `?namespace=`. + Three ways: - a file as the raw body, named by `?filename=`, Content-Disposition or X-Filename: audio, video, documents and images become recordings and resources as uploads do; .srt, .vtt, .json, .jsonl and .ics are imported as @@ -271,9 +373,10 @@ async def push_to_import_hook( - JSON `{url}` (a page or PDF kept as a document) and/or `{text}` (imported as a transcript), with an optional `title`. - Each runs the hook's pipeline, or the namespace's. 401 for an unknown token, 403 while the hook is paused, 413 over - uploads.max_mb. Audited as `import.hook`.""" - return await _push(request, db, cfg, None, filename, title) + Each runs the hook's pipeline, or the namespace's. 401 for an unknown token, saying which kind it is; 403 while the + hook is paused, for a read-only token, or without the import permission; 413 over uploads.max_mb. Audited as + `import.hook` (by the hook) or `import.push` (by the person).""" + return await _push(request, db, cfg, acl, None, filename, title, namespace) @router.post("/hooks/import/{token}", status_code=202, openapi_extra=FILE_BODY) @@ -282,9 +385,10 @@ async def push_to_import_hook_token( request: Request, db: Db, cfg: Cfg, + acl: Acl, filename: str | None = Query(None, max_length=1000, description="the file's name, for a raw body"), title: str | None = Query(None, max_length=200), ) -> HookPushed: - """The same, with the token in the address, for services that can only be given a URL. Prefer the bearer token: - addresses end up in logs.""" - return await _push(request, db, cfg, token, filename, title) + """The same, with an import webhook's token in the address, for services that can only be given a URL. Prefer the + bearer token: addresses end up in logs. A person's own token is never taken here.""" + return await _push(request, db, cfg, acl, token, filename, title, None) diff --git a/fastapi_backend/app/domain/import_hooks.py b/fastapi_backend/app/domain/import_hooks.py index 4fa1220d7..69d758db0 100644 --- a/fastapi_backend/app/domain/import_hooks.py +++ b/fastapi_backend/app/domain/import_hooks.py @@ -12,6 +12,9 @@ - text: imported as a transcript, as pasted text is. Everything a hook brings in is audited under the hook's own name (`import.hook`), never a person's. + +A person can push the same way with their own token (an API token with write access, an app's, or the web app's +session), naming the namespace: for_person() makes the target, and what it brings in is theirs (`import.push`). """ from __future__ import annotations @@ -147,12 +150,22 @@ def by_token(db, token): return hook +def for_person(db, sid, who): + """Where a person's own push goes: namespace `sid`, its default collection and its pipeline, by `who` + ({id, email}), the way a hook's target is shaped.""" + return {"space": sid, "namespace": store.space_names(db).get(sid), "collection": None, "pipeline": None, "person": who} + + def actor(hook): - """Who a hook's imports are by, in jobs, uploads and the audit log.""" + """Who a hook's imports are by, in jobs, uploads and the audit log (for a person's push, the person).""" + if hook.get("person"): + return hook["person"] return {"id": f"import_hook:{hook['id']}", "email": f"import hook: {hook['name']}"} def _used(db, hook, n): + if hook.get("person"): + return # a person's push isn't a hook's: nothing to count db.q("UPDATE $r SET last_used_at = $t, used += $n", r=R("import_hook", hook["id"]), t=store.now(), n=n) @@ -206,7 +219,8 @@ def accept_file(db, cfg, hook, path, filename, title=None): ) except SystemExit as e: raise ValueError(str(e)) from None - db.q("UPDATE $r SET path = $p", r=R("recording", rid), p=f"hook:{hook['id']}:{name}") + origin = f"upload:{name}" if hook.get("person") else f"hook:{hook['id']}:{name}" + db.q("UPDATE $r SET path = $p", r=R("recording", rid), p=origin) job, dup = jobs.enqueue(db, rid, None, by=by["email"], pipeline=hook.get("pipeline")), False finally: path.unlink(missing_ok=True) diff --git a/fastapi_backend/app/schemas/import_hooks.py b/fastapi_backend/app/schemas/import_hooks.py index f91c74b63..7161804a8 100644 --- a/fastapi_backend/app/schemas/import_hooks.py +++ b/fastapi_backend/app/schemas/import_hooks.py @@ -63,3 +63,14 @@ class HookItem(ResponseModel): class HookPushed(ResponseModel): namespace: str items: list[HookItem] + + +class ImportTarget(ResponseModel): + """Where pushes with a token go (GET /hooks/import).""" + + kind: Literal["hook", "person"] = Field(description="an import webhook's token, or a person's own") + hook: str | None = Field(None, description="the hook's name") + email: str | None = Field(None, description="whom a person's own token acts as") + namespace: str | None = Field(None, description="where a push goes without ?namespace=; null when there's a choice") + namespaces: list[str] = Field(default_factory=list, description="every namespace this token can push into") + write: bool = Field(description="false for a read-only API token, which can't push") diff --git a/fastapi_backend/tests/api/test_import_hooks.py b/fastapi_backend/tests/api/test_import_hooks.py index a57e65a1b..c93a8fa50 100644 --- a/fastapi_backend/tests/api/test_import_hooks.py +++ b/fastapi_backend/tests/api/test_import_hooks.py @@ -5,7 +5,7 @@ import pytest -from app.domain import convert, import_hooks, store, uploads +from app.domain import auth, convert, import_hooks, store, uploads from tests.helpers import login, make_user, text_pdf, write_wav R = store.R @@ -150,3 +150,90 @@ def test_size_limit_and_collection(client, env, db, cfg, monkeypatch): monkeypatch.setattr(uploads, "MB", 1) # files up to 4096 bytes r = client.post(f"{PUSH}?filename=other.wav", headers=_bearer(token), content=b"RIFF" + b"\0" * 5000) assert r.status_code == 413 + + +def _api_token(client, h, scope="write"): + r = client.post("/api/v1/tokens", headers=h, json={"name": "Web clipper", "scope": scope}) + assert r.status_code == 200, r.text + return r.json()["token"] + + +def test_people_push_with_their_own_token(client, env, db): + """The web clipper (or an agent) with a person's API token, an app's token or the web app's session: the namespace + is named, or the only one they can import into; it's audited as theirs.""" + he, ho = env["he"], env["ho"] + key = _api_token(client, he) + h = _bearer(key) + + # where it goes, without importing anything + got = client.get(PUSH, headers=h).json() + assert (got["kind"], got["email"], got["namespace"], got["namespaces"], got["write"]) == ("person", "ed@x.io", "pods", ["pods"], True) + hook = _hook(client, ho)["token"] + got = client.get(PUSH, headers=_bearer(hook)).json() + assert (got["kind"], got["hook"], got["namespace"]) == ("hook", "Scanner", "pods") + + # their only namespace without ?namespace=, or the one named + r = client.post(PUSH, headers=h, json={"text": "Clipped from the harbour blog.", "title": "Harbour blog"}) + assert r.status_code == 202, r.text + assert r.json()["namespace"] == "pods" and r.json()["items"][0]["kind"] == "text" + r = client.post(f"{PUSH}?namespace=pods", headers=h, files=[("file", ("tide.srt", SRT.encode(), "application/x-subrip"))]) + assert r.status_code == 202, r.text + rid = r.json()["items"][0]["recording"] + assert db.one("SELECT path FROM $r", r=R("recording", rid))["path"] == "upload:tide.srt" + assert client.post(PUSH, headers=env["he"], json={"text": "From the web app's own session."}).status_code == 202 + + # audited as the person, not as a hook + by = db.rows("SELECT account, email, detail FROM audit_log WHERE action = 'import.push'") + assert len(by) == 3 and {b["email"] for b in by} == {"ed@x.io"} and by[0]["detail"]["namespace"] == "pods" + + # with a choice of namespaces, it has to be named; one they can't import into is refused + store.ns_id(db, "calls") + store.ns_id(db, "quiet") + uid = db.values("SELECT VALUE record::id(id) FROM account WHERE email = 'ed@x.io'")[0] + auth.set_role(db, uid, store.ns_id(db, "calls"), "editor") + auth.set_role(db, uid, store.ns_id(db, "quiet"), "viewer") + assert client.get(PUSH, headers=h).json()["namespaces"] == ["calls", "pods"] + r = client.post(PUSH, headers=h, json={"text": "Which one?"}) + assert r.status_code == 400 and "calls, pods" in r.json()["detail"] + assert client.post(f"{PUSH}?namespace=calls", headers=h, json={"text": "Into calls."}).json()["namespace"] == "calls" + assert client.post(f"{PUSH}?namespace=quiet", headers=h, json={"text": "x"}).status_code == 403 + assert client.post(f"{PUSH}?namespace=nowhere", headers=h, json={"text": "x"}).status_code == 404 + + # a hook's token goes only into its own namespace + assert client.post(f"{PUSH}?namespace=calls", headers=_bearer(hook), json={"text": "x"}).status_code == 400 + + # read-only keys can't push; a person's token never goes in the address + ro = _bearer(_api_token(client, he, "read")) + assert client.get(PUSH, headers=ro).json()["write"] is False + r = client.post(PUSH, headers=ro, json={"text": "x"}) + assert r.status_code == 403 and "read-only" in r.json()["detail"] + r = client.post(f"{PUSH}/{key}", json={"text": "x"}) + assert r.status_code == 401 and "Authorization" in r.json()["detail"] + + +def test_unknown_tokens_say_which_kind(client, env): + for token, says in [ + (None, "import webhook token (lih_...) or your own API token"), + ("lih_" + "a" * 32, "import webhook token"), + ("la_" + "a" * 32, "API token"), + ("lo_" + "a" * 32, "sign in again"), + ("t", "sign in again"), + ]: + h = _bearer(token) if token else {} + for r in (client.get(PUSH, headers=h), client.post(PUSH, headers=h, json={"text": "x"})): + assert r.status_code == 401 and says in r.json()["detail"], (token, r.text) + + +def test_the_clipper_signed_in_as_the_person(client, env): + """The web clipper's sign-in is an OAuth app's token (docs/web-clipper.md): with the write scope it pushes into the + person's namespace; a read-only app's can't.""" + from tests.api.test_oauth import _bearer as app_bearer + from tests.api.test_oauth import _grant + + _, tokens = _grant(client, env["he"], scope="read write") + h = app_bearer(tokens) + assert client.get(PUSH, headers=h).json()["namespaces"] == ["pods"] + r = client.post(f"{PUSH}?namespace=pods", headers=h, json={"text": "Clipped while signed in.", "title": "Signed in"}) + assert r.status_code == 202, r.text + _, read_only = _grant(client, env["he"], scope="read") + assert client.post(PUSH, headers=app_bearer(read_only), json={"text": "x"}).status_code == 403 diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index af490230d..210bc7fc2 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -1426,6 +1426,10 @@ export type { ImportPreviewRequest, ImportRequest, ImportResult, + ImportTarget, + ImportTargetData, + ImportTargetResponse, + ImportTargetResponses, ImportTranscriptData, ImportTranscriptError, ImportTranscriptErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index e3bfc4787..9dc358223 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -704,6 +704,8 @@ import type { ImportNamespaceRdfData, ImportNamespaceRdfErrors, ImportNamespaceRdfResponses, + ImportTargetData, + ImportTargetResponses, ImportTranscriptData, ImportTranscriptErrors, ImportTranscriptResponses, @@ -10337,10 +10339,28 @@ export class ImportHooks { }); } + /** + * Import Target + * + * Where pushes with this bearer token go, without importing anything: an import webhook's token goes into its + * namespace; a person's own token (an API token, an app's, or the web app's) into any namespace they can import into, + * named with `?namespace=`. 401 for a token Lens doesn't know, saying which kind it is; 403 while a hook is paused. + */ + public static importTarget( + options?: Options, + ): RequestResult { + return (options?.client ?? client).get({ + url: "/api/v1/hooks/import", + ...options, + }); + } + /** * Push To Import Hook * - * Push something into an import webhook's namespace, with its token as a bearer token. Three ways: + * Push something into an import webhook's namespace, with its token as a bearer token, or into one of your own + * namespaces with your own token (an API token with write access, an app's, or the web app's) and `?namespace=`. + * Three ways: * * - a file as the raw body, named by `?filename=`, Content-Disposition or X-Filename: audio, video, documents and * images become recordings and resources as uploads do; .srt, .vtt, .json, .jsonl and .ics are imported as @@ -10349,8 +10369,9 @@ export class ImportHooks { * - JSON `{url}` (a page or PDF kept as a document) and/or `{text}` (imported as a transcript), with an optional * `title`. * - * Each runs the hook's pipeline, or the namespace's. 401 for an unknown token, 403 while the hook is paused, 413 over - * uploads.max_mb. Audited as `import.hook`. + * Each runs the hook's pipeline, or the namespace's. 401 for an unknown token, saying which kind it is; 403 while the + * hook is paused, for a read-only token, or without the import permission; 413 over uploads.max_mb. Audited as + * `import.hook` (by the hook) or `import.push` (by the person). */ public static pushToImportHook( options?: Options, @@ -10368,8 +10389,8 @@ export class ImportHooks { /** * Push To Import Hook Token * - * The same, with the token in the address, for services that can only be given a URL. Prefer the bearer token: - * addresses end up in logs. + * The same, with an import webhook's token in the address, for services that can only be given a URL. Prefer the + * bearer token: addresses end up in logs. A person's own token is never taken here. */ public static pushToImportHookToken( options: Options, diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index b0d2a2f70..3e172f02b 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -7088,6 +7088,51 @@ export type ImportResult = { [key: string]: unknown; }; +/** + * ImportTarget + * + * Where pushes with a token go (GET /hooks/import). + */ +export type ImportTarget = { + /** + * Kind + * + * an import webhook's token, or a person's own + */ + kind: "hook" | "person"; + /** + * Hook + * + * the hook's name + */ + hook?: string | null; + /** + * Email + * + * whom a person's own token acts as + */ + email?: string | null; + /** + * Namespace + * + * where a push goes without ?namespace=; null when there's a choice + */ + namespace?: string | null; + /** + * Namespaces + * + * every namespace this token can push into + */ + namespaces?: Array; + /** + * Write + * + * false for a read-only API token, which can't push + */ + write: boolean; + [key: string]: unknown; +}; + /** * ImportedAs */ @@ -36787,6 +36832,22 @@ export type NewImportHookTokenResponses = { export type NewImportHookTokenResponse = NewImportHookTokenResponses[keyof NewImportHookTokenResponses]; +export type ImportTargetData = { + body?: never; + path?: never; + query?: never; + url: "/api/v1/hooks/import"; +}; + +export type ImportTargetResponses = { + /** + * Successful Response + */ + 200: ImportTarget; +}; + +export type ImportTargetResponse = ImportTargetResponses[keyof ImportTargetResponses]; + export type PushToImportHookData = { /** * HookPush @@ -36825,6 +36886,12 @@ export type PushToImportHookData = { * its title (default: the file's name, or the page's) */ title?: string | null; + /** + * Namespace + * + * with your own token: the namespace it goes into (default: the only one you can import into) + */ + namespace?: string | null; }; url: "/api/v1/hooks/import"; }; diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 49f9e7ffe..ef4b92560 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -26349,12 +26349,32 @@ } }, "/api/v1/hooks/import": { + "get": { + "tags": [ + "import hooks" + ], + "summary": "Import Target", + "description": "Where pushes with this bearer token go, without importing anything: an import webhook's token goes into its\nnamespace; a person's own token (an API token, an app's, or the web app's) into any namespace they can import into,\nnamed with `?namespace=`. 401 for a token Lens doesn't know, saying which kind it is; 403 while a hook is paused.", + "operationId": "import_target", + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ImportTarget" + } + } + } + } + } + }, "post": { "tags": [ "import hooks" ], "summary": "Push To Import Hook", - "description": "Push something into an import webhook's namespace, with its token as a bearer token. Three ways:\n\n- a file as the raw body, named by `?filename=`, Content-Disposition or X-Filename: audio, video, documents and\n images become recordings and resources as uploads do; .srt, .vtt, .json, .jsonl and .ics are imported as\n transcripts;\n- multipart form files (field `file`, up to 20), with optional `title`, `url` and `text` fields;\n- JSON `{url}` (a page or PDF kept as a document) and/or `{text}` (imported as a transcript), with an optional\n `title`.\n\nEach runs the hook's pipeline, or the namespace's. 401 for an unknown token, 403 while the hook is paused, 413 over\nuploads.max_mb. Audited as `import.hook`.", + "description": "Push something into an import webhook's namespace, with its token as a bearer token, or into one of your own\nnamespaces with your own token (an API token with write access, an app's, or the web app's) and `?namespace=`.\nThree ways:\n\n- a file as the raw body, named by `?filename=`, Content-Disposition or X-Filename: audio, video, documents and\n images become recordings and resources as uploads do; .srt, .vtt, .json, .jsonl and .ics are imported as\n transcripts;\n- multipart form files (field `file`, up to 20), with optional `title`, `url` and `text` fields;\n- JSON `{url}` (a page or PDF kept as a document) and/or `{text}` (imported as a transcript), with an optional\n `title`.\n\nEach runs the hook's pipeline, or the namespace's. 401 for an unknown token, saying which kind it is; 403 while the\nhook is paused, for a read-only token, or without the import permission; 413 over uploads.max_mb. Audited as\n`import.hook` (by the hook) or `import.push` (by the person).", "operationId": "push_to_import_hook", "parameters": [ { @@ -26394,6 +26414,25 @@ "title": "Title" }, "description": "its title (default: the file's name, or the page's)" + }, + { + "name": "namespace", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string", + "maxLength": 40 + }, + { + "type": "null" + } + ], + "description": "with your own token: the namespace it goes into (default: the only one you can import into)", + "title": "Namespace" + }, + "description": "with your own token: the namespace it goes into (default: the only one you can import into)" } ], "responses": { @@ -26509,7 +26548,7 @@ "import hooks" ], "summary": "Push To Import Hook Token", - "description": "The same, with the token in the address, for services that can only be given a URL. Prefer the bearer token:\naddresses end up in logs.", + "description": "The same, with an import webhook's token in the address, for services that can only be given a URL. Prefer the\nbearer token: addresses end up in logs. A person's own token is never taken here.", "operationId": "push_to_import_hook_token", "parameters": [ { @@ -39632,6 +39671,76 @@ ], "title": "ImportResult" }, + "ImportTarget": { + "properties": { + "kind": { + "type": "string", + "enum": [ + "hook", + "person" + ], + "title": "Kind", + "description": "an import webhook's token, or a person's own" + }, + "hook": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Hook", + "description": "the hook's name" + }, + "email": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Email", + "description": "whom a person's own token acts as" + }, + "namespace": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Namespace", + "description": "where a push goes without ?namespace=; null when there's a choice" + }, + "namespaces": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Namespaces", + "description": "every namespace this token can push into" + }, + "write": { + "type": "boolean", + "title": "Write", + "description": "false for a read-only API token, which can't push" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "kind", + "write" + ], + "title": "ImportTarget", + "description": "Where pushes with a token go (GET /hooks/import)." + }, "ImportedAs": { "properties": { "recording": { diff --git a/threat-model.md b/threat-model.md index a5485eeeb..9e36a93e6 100644 --- a/threat-model.md +++ b/threat-model.md @@ -280,6 +280,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `/namespaces/{name}/sparql` | SPARQL text | yes, namespace readers | callback/code | none | *(documented, routes/rdf.py sparql docstring "SERVICE and FROM aren't allowed")* | | `/namespaces/{name}/rdf/import` | Turtle, N-Triples, JSON-LD | yes, editors | serialized state, size | none | *(documented, rdf_import.py docstring "RDF/XML isn't read")* | | `/uploads`, `/import`, `/recordings/{rid}/files` | file bytes, name, extension, size | yes, editors | data, type/class, size | none | *(documented, docs/configuration.md "Uploads accept only the types")* | +| `POST /hooks/import` | bearer (a hook's `lih_` token, or a person's `la_`, `lo_` or web token with `?namespace=`), file bytes, URL, text | yes: anyone holding a hook's token, for its namespace; a person's token, for namespaces where its person has the import permission and only with write scope | data, resource-name, size | none; the address form `/hooks/import/{token}` takes hook tokens only, so a person's token never sits in a logged path | *(documented, docs/api.md "Your own token only goes in the `Authorization` header")* | | watched folders, storage sources, IMAP, iCal | remote file names, bytes, mail, events | yes, whoever writes to the remote, mails the box or serves the feed | data, resource-name, size | admin creates sources | *(inferred, Q17)* | | the assistant's mailbox (`mailbox.py`) | mail headers, text, attachments, forwarded mail, `Authentication-Results` | yes, anyone who knows the address; From is forgeable | data, x-prompt-injection, size | answered only for `mailbox.users` whose provider vouched for From; forwarded text marked untrusted, and changes after it ask first | *(documented, docs/assistant-email.md "it answers only mail that is both")* | | `POST /imports/web`, extension `http` tools, notification targets | URL | yes: editors (web, tools), owners (targets) | resource-name, x-network-destination | none | *(maintainer, 2026-10)* | From dc56dd3d139399b7746bad5f6531ef1718b881d7 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 11:30:52 +0000 Subject: [PATCH 2/3] docs(changelog): the clipper signs in or takes your API token Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01NZxdXgdzoJfnSLju6B6Ygz --- CHANGELOG.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c80f9ecb3..4243fccdb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,15 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **The web clipper takes your API token, or just signs in.** It used to take only an import webhook's token, so an API + token was refused with "Lens didn't recognise this token: make a new import webhook and paste its token". Now its + options have **Sign in**: Lens asks you to allow the clipper, nothing to copy, and clips go into the namespace you + pick (the popup offers each one you can add to). **Use a token instead** takes an API token with write access or a + hook's token, and when Lens refuses one the clipper says why: unknown, revoked, read-only, or a Lens too old for + API tokens. Anything acting as you (an agent, a script) can push to `POST /api/v1/hooks/import?namespace=` with + your own token too, and `GET` there says where a token's pushes go ([Web clipper](docs/web-clipper.md#setting-it-up), + [API](docs/api.md#import-webhooks)). + ## 0.10.0 October 15, 2026 {id="0.10.0"} - **The assistant always answers.** When a question used up its tool steps (`ai.max_steps`), the assistant used to stop From dfd20003ab8e6ec56eb810d1b0d0f6dd049333cf Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Thu, 15 Oct 2026 11:34:57 +0000 Subject: [PATCH 3/3] docs(threat-model): the push address in the YAML and JSON companions Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01NZxdXgdzoJfnSLju6B6Ygz --- threat-model.json | 12 +++++++++++- threat-model.md | 2 +- threat-model.yaml | 11 ++++++++++- 3 files changed, 22 insertions(+), 3 deletions(-) diff --git a/threat-model.json b/threat-model.json index fd496b8d8..8acf0a5eb 100644 --- a/threat-model.json +++ b/threat-model.json @@ -84,7 +84,8 @@ { "name": "ingest", "entry_points": [ - "/uploads, /import, /recordings/{rid}/files" + "/uploads, /import, /recordings/{rid}/files", + "POST /hooks/import" ], "touches": [ "filesystem", @@ -385,6 +386,15 @@ "provenance": "documented", "source": "docs/configuration.md \"Uploads accept only the types\"" }, + { + "entry_point": "POST /hooks/import", + "parameter": "bearer (a hook's lih_ token, or a person's la_, lo_ or web token with ?namespace=), file bytes, URL, text", + "attacker_controllable": "yes", + "component": "ingest", + "caller_must_enforce": "none — P-UPLOAD; the address form /hooks/import/{token} takes hook tokens only, so a person's token never sits in a logged path", + "provenance": "documented", + "source": "docs/api.md \"the address form takes hook tokens only\"" + }, { "entry_point": "watched folders, storage sources, IMAP, iCal", "parameter": "remote file names, bytes, mail, events", diff --git a/threat-model.md b/threat-model.md index 9e36a93e6..296a68983 100644 --- a/threat-model.md +++ b/threat-model.md @@ -280,7 +280,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | `/namespaces/{name}/sparql` | SPARQL text | yes, namespace readers | callback/code | none | *(documented, routes/rdf.py sparql docstring "SERVICE and FROM aren't allowed")* | | `/namespaces/{name}/rdf/import` | Turtle, N-Triples, JSON-LD | yes, editors | serialized state, size | none | *(documented, rdf_import.py docstring "RDF/XML isn't read")* | | `/uploads`, `/import`, `/recordings/{rid}/files` | file bytes, name, extension, size | yes, editors | data, type/class, size | none | *(documented, docs/configuration.md "Uploads accept only the types")* | -| `POST /hooks/import` | bearer (a hook's `lih_` token, or a person's `la_`, `lo_` or web token with `?namespace=`), file bytes, URL, text | yes: anyone holding a hook's token, for its namespace; a person's token, for namespaces where its person has the import permission and only with write scope | data, resource-name, size | none; the address form `/hooks/import/{token}` takes hook tokens only, so a person's token never sits in a logged path | *(documented, docs/api.md "Your own token only goes in the `Authorization` header")* | +| `POST /hooks/import` | bearer (a hook's `lih_` token, or a person's `la_`, `lo_` or web token with `?namespace=`), file bytes, URL, text | yes: anyone holding a hook's token, for its namespace; a person's token, for namespaces where its person has the import permission and only with write scope | data, resource-name, size | none; the address form `/hooks/import/{token}` takes hook tokens only, so a person's token never sits in a logged path | *(documented, docs/api.md "the address form takes hook tokens only")* | | watched folders, storage sources, IMAP, iCal | remote file names, bytes, mail, events | yes, whoever writes to the remote, mails the box or serves the feed | data, resource-name, size | admin creates sources | *(inferred, Q17)* | | the assistant's mailbox (`mailbox.py`) | mail headers, text, attachments, forwarded mail, `Authentication-Results` | yes, anyone who knows the address; From is forgeable | data, x-prompt-injection, size | answered only for `mailbox.users` whose provider vouched for From; forwarded text marked untrusted, and changes after it ask first | *(documented, docs/assistant-email.md "it answers only mail that is both")* | | `POST /imports/web`, extension `http` tools, notification targets | URL | yes: editors (web, tools), owners (targets) | resource-name, x-network-destination | none | *(maintainer, 2026-10)* | diff --git a/threat-model.yaml b/threat-model.yaml index d0c9a3ac8..8f10e4923 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -2,7 +2,7 @@ # Authority order: prose > yaml > json. Regenerate whenever the prose changes. schema: threat-model-sidecar/v2 project: lens -prose_version: "threat-model.md@sha256:f430a4e9bedcf37ca6a5f5e2e782a3e72a9ac674b6dc78ebc91fc152af9e92a4" +prose_version: "threat-model.md@sha256:9d5f693c4d164a29816e39a914d62d72a40090930003ea2fc7949a049220819e" model_status: under-review triage_policy: strict confidence: {documented: 201, maintainer: 46, inferred: 91, assumption: 8} @@ -312,6 +312,15 @@ entry_points: control_kinds: [data, type-class, size] caller_must_enforce: "none — P-UPLOAD" provenance: {kind: documented, source: 'docs/configuration.md "Uploads accept only the types"'} + - id: "POST /hooks/import" + component: ingest + parameters: + - name: "bearer (a hook's lih_ token, or a person's la_, lo_ or web token with ?namespace=), file bytes, URL, text" + attacker_controllable: true + x-condition: "anyone holding a hook's token, for its namespace; a person's token, for namespaces where its person has the import permission and only with write scope" + control_kinds: [data, resource-name, size] + caller_must_enforce: "none — P-UPLOAD; the address form /hooks/import/{token} takes hook tokens only, so a person's token never sits in a logged path" + provenance: {kind: documented, source: 'docs/api.md "the address form takes hook tokens only"'} - id: "watched folders, storage sources, IMAP, iCal" component: outbound parameters: