From 843023fb9a5104064172b43d8cab7f72da2f6115 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sun, 18 Oct 2026 09:11:29 +0000 Subject: [PATCH] feat(feeds): a private podcast feed of a collection, from its page Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01QFUNzy1QZy5HhKAyJxTAYJ --- README.md | 4 +- changelog.d/collection-feed.md | 7 + docs/api.md | 9 +- docs/feeds.md | 33 ++- docs/mcp.md | 2 +- docs/requirements.md | 1 + fastapi_backend/app/api/mcp_tools.py | 18 +- .../app/api/v1/routes/collections.py | 56 ++++- fastapi_backend/app/domain/auth.py | 8 +- fastapi_backend/app/domain/store.py | 4 +- fastapi_backend/app/domain/syndication.py | 198 ++++++++++++++-- fastapi_backend/app/schemas/feeds.py | 14 ++ fastapi_backend/tests/api/test_feeds.py | 93 ++++++++ .../__tests__/collection-feed.test.ts | 21 ++ nextjs-frontend/app/openapi-client/index.ts | 16 ++ nextjs-frontend/app/openapi-client/sdk.gen.ts | 56 +++++ .../app/openapi-client/types.gen.ts | 153 ++++++++++++ .../collections/collection-detail.tsx | 7 + .../components/collections/feed-dialog.tsx | 91 ++++++++ .../components/collections/feed-model.ts | 14 ++ nextjs-frontend/openapi.json | 219 ++++++++++++++++++ threat-model.json | 8 +- threat-model.md | 4 +- threat-model.yaml | 10 +- 24 files changed, 1006 insertions(+), 40 deletions(-) create mode 100644 changelog.d/collection-feed.md create mode 100644 nextjs-frontend/__tests__/collection-feed.test.ts create mode 100644 nextjs-frontend/components/collections/feed-dialog.tsx create mode 100644 nextjs-frontend/components/collections/feed-model.ts diff --git a/README.md b/README.md index d647c2c42..21d1ec248 100644 --- a/README.md +++ b/README.md @@ -200,7 +200,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p confirm. Export an asset or a collection as an evidence PDF with everything Lens found about it, or the parts you pick. Publish a namespace's resources, episodes, timeline activities or text as a private RSS feed or a podcast feed that any feed reader or podcast app follows (Feeds, from the Podcasts page), and let a routine fill it on a schedule with - what's new, or make an episode about the week's new resources. ([Podcasts](docs/podcasts.md), + what's new, or make an episode about the week's new resources. Follow a saved collection in your podcast app: its + page gives your own private address for it, listing its assets with audio or video that you can read. + ([Podcasts](docs/podcasts.md), [specs from transcripts](docs/notes.md#specs-from-transcripts), [notes](docs/notes.md), [evidence PDFs](docs/api.md#evidence-pdfs), [published feeds](docs/feeds.md); [checks](docs/requirements.md#learn-from-it)) diff --git a/changelog.d/collection-feed.md b/changelog.d/collection-feed.md new file mode 100644 index 000000000..abb38b05d --- /dev/null +++ b/changelog.d/collection-feed.md @@ -0,0 +1,7 @@ +- **Follow a collection in your podcast app.** A saved collection's page has **Podcast feed**: your private address + for it, with Copy and New address. Its episodes are the collection's assets with audio or video that you can read, + with the same RSS and Apple Podcasts' tags, audio and private-address rules as a namespace's podcast feed. Everyone + who may read the collection gets an address of their own, and it stops when the collection is deleted or no longer + shared with them. Agents get it with `GET /api/v1/collections/{id}/feed` or the MCP tool `list_feeds` with + `collection_id` ([A collection's podcast feed](docs/feeds.md#a-collections-podcast-feed)). + diff --git a/docs/api.md b/docs/api.md index 21bfdc9ef..732f93df7 100644 --- a/docs/api.md +++ b/docs/api.md @@ -1799,9 +1799,16 @@ GET /api/v1/collections/{cid} PATCH /api/v1/collections/{cid} DELETE /api/v1/collections/{cid} GET /api/v1/collections/{cid}/evidence.pdf +GET /api/v1/collections/{cid}/feed +POST /api/v1/collections/{cid}/feed/token +DELETE /api/v1/collections/{cid}/feed ``` -`GET /{cid}/evidence.pdf` is the collection as one [evidence PDF](#evidence-pdfs). +`GET /{cid}/evidence.pdf` is the collection as one [evidence PDF](#evidence-pdfs). `GET /{cid}/feed` is your private +podcast feed of it: its address (`url`, read by podcast apps with no session), how many `episodes` it lists (its +assets with audio or video that you can read) and when an app last read it, made the first time you ask; +`POST /{cid}/feed/token` gives it a new address and `DELETE /{cid}/feed` stops it +([A collection's podcast feed](feeds.md#a-collections-podcast-feed)). ## views diff --git a/docs/feeds.md b/docs/feeds.md index 8710f8c17..b6aa9e4c2 100644 --- a/docs/feeds.md +++ b/docs/feeds.md @@ -1,7 +1,8 @@ # Published feeds (RSS and podcasts) **Status: built.** Feeds, their private addresses, the API, the MCP tools, the workflow nodes that let a routine fill a -feed or make an episode on a schedule, and the Feeds page in the web app. +feed or make an episode on a schedule, the Feeds page in the web app, and a saved collection's podcast feed, with its +address on the collection's page. A namespace can publish what's in it as a feed that any feed reader or podcast app follows: new resources, podcast episodes, things on the timeline, or text a person or an agent wrote. Each feed has a private address; add it to the @@ -66,6 +67,33 @@ new address (the old token is gone for good). What a feed listed stays in Lens e On the canvas, a graph workflow's palette has the publishing nodes under Data (Select assets, Select activities) and Keep (Publish to feed, Podcast episode); a feed is picked from a list, and the canvas names it on the node. +## A collection's podcast feed + +A saved collection ([collections](api.md#collections)) is a podcast of its own: its episodes are the assets in it +with audio or video, newest first (the newest 50), and a filter's new matches arrive as new episodes. On the +collection's page, **Podcast feed** shows your address for it with **Copy**: paste it into your podcast app as a show +added by its address. Nothing has to be made first, and nobody but you has to do anything. + +- **One address per person.** Everyone who may read the collection (its owner, and everyone when it's shared) has + their own address, and it lists only what they can read when their podcast app reads it, so sharing a collection + never shows anyone else's assets. A vault's assets are never in it. +- **The same private address as a namespace's feed.** It is an `lfd_` token under `/api/v1/feeds/public/`, kept + sealed and found by its hash, with the same RSS 2.0 and Apple Podcasts' tags, the same audio with byte ranges, and + the same requests to directories and search engines to keep out. **New address** (rotate) stops the old one at + once. +- **It stops when it should.** Deleting the collection deletes its feeds. When the collection is no longer shared + with you, or your account is disabled, your address stops working. +- Its title and description are the collection's, as they are when the app reads it. + +| | | +|---|---| +| `GET /api/v1/collections/{id}/feed` | your podcast feed of a collection you may read: its private address (`url`), how many `episodes` it lists now and when a podcast app last read it (`fetched_at`). Made the first time you ask (audited as `feed.create`), the same one after | +| `POST /api/v1/collections/{id}/feed/token` | a new address; the old one stops working. Audited as `feed.token` | +| `DELETE /api/v1/collections/{id}/feed` | stop it: its address stops working (asking again makes a new one). Audited as `feed.delete` | + +Agents get the address with the MCP tool `list_feeds` and `collection_id` (on the main MCP server, since a +collection spans namespaces). A collection's feed isn't in `GET /api/v1/feeds`, which lists namespaces' feeds. + ## From a routine A routine fills a feed on a schedule with a graph workflow ([Publishing from the graph](processing.md#publishing-from-the-graph)): @@ -92,7 +120,8 @@ what's new, and what the feed's readers couldn't read is refused. Agents use the MCP tools `list_feeds` (each feed's address, or one feed with its newest items), `create_feed` and `publish_to_feed` ([MCP](mcp.md)). -Refine later: artwork Lens makes for a feed without an `image`; Atom and JSON Feed alongside RSS; chapters and +Refine later: a collection feed's settings (artwork, how many episodes, its author) in the web app; undo for a new +address; artwork Lens makes for a feed without an `image`; Atom and JSON Feed alongside RSS; chapters and transcripts in a podcast feed (`podcast:transcript`); an automation action that publishes to a feed when something happens; a preview of the feed's XML on its page; Try on the canvas listing what Publish to feed would add, item by item. diff --git a/docs/mcp.md b/docs/mcp.md index 500b00ec3..bef629744 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -108,7 +108,7 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ | `schedule_project` | ask a prompt in a project on a schedule, in a new thread each time or the same one; change, switch off or remove a schedule. Needs a write-scope token | | `create_podcast` | make a two-host learning [episode](podcasts.md) about assets or lines in them, with an angle, a length and a style; it's made in the background. Needs a write-scope token and editor access where it goes | | `get_podcast` | an episode: how far it has got, its script with each line's speaker and the sources it cites (links), and what the fact-check changed | -| `list_feeds` | the RSS and podcast [feeds](feeds.md) your namespaces publish, each with its private address and how many items it holds; with `feed_id`, one feed and its newest items | +| `list_feeds` | the RSS and podcast [feeds](feeds.md) your namespaces publish, each with its private address and how many items it holds; with `feed_id`, one feed and its newest items; with `collection_id`, your private podcast feed of a saved collection (main server only) | | `create_feed` | make an RSS feed or a podcast feed in a namespace, for feed readers and podcast apps. Needs a write-scope token and editor access | | `publish_to_feed` | add resources, episodes, timeline activities and a text item to a feed; what's there already isn't added twice, and what may not show there comes back refused, with why. Needs a write-scope token and editor access | | `write_spec` | write a technical [spec](notes.md#specs-from-transcripts) from a recording's transcript as a note linked to it: every decision, no and constraint in the speaker's own words with links to when they said them, and the gaps left to decide later; it's written in the background. Needs a write-scope token and the right to process the recording | diff --git a/docs/requirements.md b/docs/requirements.md index dcd169914..fc9ffed22 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -245,6 +245,7 @@ request, to another check that proves the same thing; loosening a requirement or | A note shows the notes it's inside and the notes inside it, to people and agents | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_path_and_children`, `test_assistant_keeps_notes`; [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_note_tools`; [note-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/note-page.test.tsx) | | Undo and Recently deleted instead of a confirm | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_recently_deleted`; [notes-delete.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-delete.test.tsx) | | A namespace publishes a private RSS feed or a podcast feed (Apple Podcasts' tags, audio with byte ranges) of resources, episodes, timeline activities and text, holding only what its readers can already read; through the API and MCP | [test_feeds.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feeds.py) `test_making_filling_and_reading_a_feed`, `test_a_podcast_feed_lists_what_has_audio`, `test_timeline_activities_in_a_feed`, `test_vaults_and_settings`, `test_agents_use_feeds` | +| A saved collection is a private podcast feed of its assets with audio or video: one address per person who may read it, listing only what they can read, from the collection's page in one click with Copy, rotated or stopped like a namespace's feed; through the API and MCP | [test_feeds.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feeds.py) `test_a_collection_feed_lists_its_assets_with_media`, `test_a_collection_feed_token_rotates_and_stops`, `test_agents_get_a_collection_feed`; [collection-feed.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/collection-feed.test.ts) | | A routine fills a feed on a schedule, or makes a podcast episode about what's new: graph workflow nodes pick the resources or timeline activities since the last run, add them to a feed, or make an episode about them; Try says what they would do | [test_feed_nodes.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feed_nodes.py) `test_a_routine_publishes_new_resources_to_a_feed`, `test_a_routine_publishes_the_timeline_and_text`, `test_a_routine_makes_an_episode_and_the_feed_lists_it_once_made`, `test_try_says_what_it_would_do_and_bad_settings_are_refused` | | Feeds in the web app: a Feeds page, reached from the Podcasts page and under Explore once there is a feed; New feed with the title focused and Enter making it in the current namespace; a feed's page copies the address in one click, gets a new one, edits what readers show, and takes items out or deletes the feed with Undo instead of a confirm; the publishing nodes on the canvas, checked before Publish | [feeds-ui.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/feeds-ui.test.tsx) `the Feeds page lists each feed with its kind, namespace, items and whether an app reads it`, `New feed opens on the title, defaults to the current namespace, and Enter makes it and opens its page`, `a feed's page copies the address, gets a new one, and takes an item out with Undo`, `deleting a feed is undone by making it again with its items, at a new address`, `editors change what readers show, and Enter in a field saves`; [workflow-flow.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-flow.test.ts) `a Select node into Publish to feed or Podcast episode is a whole graph workflow`, `says what to fix: a feed to choose, the limits, and a graph that publishes nothing`, `names the selection and the feed on the canvas`; [nav-groups.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/nav-groups.test.tsx) `feeds are under Explore, after Podcasts, for everyone`, `Feeds is in the nav only once there is a feed, so a phone's nav stays as short as it was` | | An asset or a collection exported as an evidence PDF, whole or the parts you pick | [test_evidence.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_evidence.py) `test_a_collection_as_one_evidence_pdf`, `test_everything_lens_found_goes_in`, `test_picking_the_parts`; [evidence-parts.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/evidence-parts.test.tsx) | diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py index cbf6c99d1..562285d1e 100644 --- a/fastapi_backend/app/api/mcp_tools.py +++ b/fastapi_backend/app/api/mcp_tools.py @@ -2364,13 +2364,27 @@ def _feed_out(ctx: Context, row: dict[str, Any]) -> dict[str, Any]: "List published feeds", "The RSS and podcast feeds the person's namespaces publish (docs/feeds.md): each one's kind, title, namespace, how " "many items it holds, and its private address, which feed readers and podcast apps subscribe to. Anyone with the " - "address can read the feed, so give it only to the person. With feed_id, one feed and its newest items.", + "address can read the feed, so give it only to the person. With feed_id, one feed and its newest items. With " + "collection_id, the person's own podcast feed of a saved collection they may read (made the first time they ask): " + "its episodes are the collection's assets with audio or video that they can read.", Arg("namespace", "string", "only this namespace's feeds"), Arg("feed_id", "integer", "one feed, with its newest items"), + Arg("collection_id", "integer", "a saved collection: the person's private podcast feed of it"), ) -def list_feeds(ctx: Context, namespace: str | None, feed_id: int | None) -> dict[str, Any]: +def list_feeds(ctx: Context, namespace: str | None, feed_id: int | None, collection_id: int | None) -> dict[str, Any]: + from app.api.v1.routes import collections as collection_routes from app.domain import syndication + if collection_id is not None: + if ctx.ns: # a saved collection spans namespaces; this server sees one + raise ToolError("a collection's feed is on the main MCP server (/mcp), not a namespace's own") + collection_routes.own_collection(ctx.db, collection_id, ctx.user) # 404: not found + fid, made = syndication.for_collection(ctx.db, ctx.cfg, collection_id, ctx.user.id, by=ctx.user.email) + if made: + auth.audit(ctx.db, ctx.user.as_audit(), "feed.create", f"feed:{fid}", {"collection": collection_id, "via": "mcp"}) + f = syndication.collection_feed(ctx.db, ctx.cfg, fid, ctx.web) + keep = ("id", "kind", "collection", "url", "page", "episodes", "fetched_at") + return store.clean({("feed_id" if k == "id" else k): f.get(k) for k in keep}) if feed_id is not None: row = syndication.get(ctx.db, feed_id) # KeyError: not found ctx.acl.need(row["space"]) diff --git a/fastapi_backend/app/api/v1/routes/collections.py b/fastapi_backend/app/api/v1/routes/collections.py index 1384d2669..e04ace4d2 100644 --- a/fastapi_backend/app/api/v1/routes/collections.py +++ b/fastapi_backend/app/api/v1/routes/collections.py @@ -1,21 +1,23 @@ """Collections: a saved filter over the archive, or a fixed list of recordings. Yours, and others' shared ones. -A collection only ever lists recordings the viewer can read, whoever saved it. +A collection only ever lists recordings the viewer can read, whoever saved it. So does its podcast feed: each person +who may read it has an address of their own (app/domain/syndication.py, docs/feeds.md). """ from __future__ import annotations from typing import Any -from fastapi import APIRouter, HTTPException +from fastapi import APIRouter, HTTPException, Request from fastapi.responses import Response from app.api import evidence as evidence_api -from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer -from app.domain import evidence, recsets, store +from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer, web_origin +from app.domain import auth, evidence, recsets, store, syndication from app.domain.store import DB from app.schemas.collections import Collection, CollectionCreate, CollectionDetail, CollectionUpdate from app.schemas.common import Created, Ok +from app.schemas.feeds import CollectionFeed from app.schemas.recordings import EvidencePart router = APIRouter(prefix="/collections", tags=["collections"]) @@ -102,4 +104,50 @@ def update_collection(cid: int, body: CollectionUpdate, user: Writer, db: Db) -> def delete_collection(cid: int, user: Writer, db: Db) -> Ok: own_collection(db, cid, user, write=True) db.q("DELETE $r", r=store.R("saved_collection", cid)) + syndication.drop_collection(db, cid) # its podcast feeds' addresses stop working + return Ok() + + +# ---------- its podcast feed (docs/feeds.md#a-collections-podcast-feed) ---------- +def _feed_id(db: DB, cfg: dict[str, Any], cid: int, user: Principal) -> int: + """This person's feed of the collection, made (and audited as `feed.create`) the first time they ask.""" + fid, made = syndication.for_collection(db, cfg, cid, user.id, by=user.email) + if made: + auth.audit(db, user.as_audit(), "feed.create", f"feed:{fid}", {"kind": "podcast", "collection": cid}) + return fid + + +def _feed(db: DB, cfg: dict[str, Any], request: Request, cid: int, user: Principal) -> CollectionFeed: + fid = _feed_id(db, cfg, cid, user) + return CollectionFeed.model_validate(syndication.collection_feed(db, cfg, fid, web_origin(request))) + + +@router.get("/{cid}/feed") +def get_collection_feed(cid: int, request: Request, user: CurrentUser, db: Db, cfg: Cfg) -> CollectionFeed: + """Your private podcast feed of a collection you may read: its address, which a podcast app subscribes to with no + session (the token in it is the credential), made the first time you ask (audited as `feed.create`) and the same + one after. Its episodes are + the collection's assets with audio or video that you can read, as they are when the app reads it.""" + own_collection(db, cid, user) + return _feed(db, cfg, request, cid, user) + + +@router.post("/{cid}/feed/token") +def rotate_collection_feed(cid: int, request: Request, user: Writer, db: Db, cfg: Cfg) -> CollectionFeed: + """A new address for your podcast feed of the collection: the old one stops working at once, so a podcast app + subscribed to it needs the new one. Audited as `feed.token`.""" + own_collection(db, cid, user) + fid = _feed_id(db, cfg, cid, user) + syndication.rotate(db, cfg, fid) + auth.audit(db, user.as_audit(), "feed.token", f"feed:{fid}", {"collection": cid}) + return _feed(db, cfg, request, cid, user) + + +@router.delete("/{cid}/feed") +def delete_collection_feed(cid: int, user: Writer, db: Db) -> Ok: + """Stop your podcast feed of the collection: its address stops working at once (asking for it again makes a new + one). Audited as `feed.delete`.""" + own_collection(db, cid, user) + syndication.drop_collection(db, cid, user.id) + auth.audit(db, user.as_audit(), "feed.delete", f"saved_collection:{cid}") return Ok() diff --git a/fastapi_backend/app/domain/auth.py b/fastapi_backend/app/domain/auth.py index 9abba918c..2ac29e5c2 100644 --- a/fastapi_backend/app/domain/auth.py +++ b/fastapi_backend/app/domain/auth.py @@ -158,9 +158,10 @@ def update_account(db, uid, name=None, admin=None, disabled=None, password=None) def remove_account(db, uid, lose_vaults=False): """Remove someone for good: every way they sign in (sessions, passkeys, sign-in links, connected accounts, API - tokens, apps they gave access to), every role and permission they had, and what only they could see (their chats, - their own notes, highlights, saved views and collections, circles, what was shared with them and what agents asked, unfinished - uploads). What they shared stays (notes and views for everyone, comments, shares with circles), under their name: the account is kept as a marker without an email, so their + tokens, apps they gave access to, their collections' podcast feed addresses), every role and permission they had, + and what only they could see (their chats, their own notes, highlights, saved views and collections, circles, what + was shared with them and what agents asked, unfinished uploads). What they shared stays (notes and views for + everyone, comments, shares with circles), under their name: the account is kept as a marker without an email, so their address can be invited again. Refused (ValueError) when their passkeys are the only way into a vault, unless `lose_vaults`.""" from . import passkeys @@ -208,6 +209,7 @@ def remove_account(db, uid, lose_vaults=False): "DELETE note WHERE account = $a AND (shared ?? false) = false", "DELETE saved_view WHERE account = $a AND (shared ?? false) = false", "DELETE saved_collection WHERE account = $a AND (shared ?? false) = false", + "DELETE feed WHERE account = $a", # their collections' podcast feeds (a namespace's feed has no account) "DELETE upload WHERE account = $a AND state = 'receiving'", "UPDATE $r SET email = $gone, admin = false, disabled = true, removed_at = $t, pw = NONE", ], diff --git a/fastapi_backend/app/domain/store.py b/fastapi_backend/app/domain/store.py index 29cb5a59e..a5cc36abb 100644 --- a/fastapi_backend/app/domain/store.py +++ b/fastapi_backend/app/domain/store.py @@ -1634,10 +1634,12 @@ def close(self): "DEFINE TABLE IF NOT EXISTS podcast SCHEMALESS", "DEFINE INDEX IF NOT EXISTS podcast_space ON podcast FIELDS space", # published RSS and podcast feeds (app/domain/syndication.py): feed:, its token kept sealed and as a hash, and - # its items, one per feed for what each is about (key: ":") + # its items, one per feed for what each is about (key: ":"); a saved collection's feed, one per person, + # has `collection` and `account` instead of a namespace "DEFINE TABLE IF NOT EXISTS feed SCHEMALESS", "DEFINE INDEX IF NOT EXISTS feed_token ON feed FIELDS token_hash UNIQUE", "DEFINE INDEX IF NOT EXISTS feed_space ON feed FIELDS space", + "DEFINE INDEX IF NOT EXISTS feed_collection ON feed FIELDS collection, account", "DEFINE TABLE IF NOT EXISTS feed_item SCHEMALESS", "DEFINE INDEX IF NOT EXISTS feed_item_feed ON feed_item FIELDS feed", "DEFINE INDEX IF NOT EXISTS feed_item_key ON feed_item FIELDS key UNIQUE", diff --git a/fastapi_backend/app/domain/syndication.py b/fastapi_backend/app/domain/syndication.py index 87f12aa00..8d53ee34c 100644 --- a/fastapi_backend/app/domain/syndication.py +++ b/fastapi_backend/app/domain/syndication.py @@ -8,6 +8,12 @@ old address stops working at once. A feed asks podcast directories not to list it (itunes:block, podcast:locked) and search engines not to index it. +A saved collection (recsets.py) has a podcast feed of its own for each person who may read it (`collection` and +`account` on the feed's row, no namespace): its episodes are the collection's assets with audio or video, as that +person may read them now (the newest `max_items`), so each person's address lists only what they can read, and it +stops when the collection is deleted or no longer shared with them, or their account is disabled. Its token is made, +sealed, found and rotated as any feed's; the feeds' management (get, list_feeds) never sees these. + An item (`feed_item:`) is one of: - `asset`: a resource (a recording, a document, a page...): its title and summary, a link to its page in the web app, @@ -39,7 +45,7 @@ import urllib.parse from xml.sax.saxutils import escape, quoteattr -from . import events, keyring, podcasts, render, settings, store, video +from . import events, keyring, podcasts, recsets, render, settings, store, video R = store.R KINDS = ("rss", "podcast") @@ -183,18 +189,28 @@ def remove(db, fid): db.run(["DELETE feed_item WHERE feed = $f", "DELETE $r"], f=int(fid), r=R("feed", int(fid))) -def get(db, fid): - """A feed's row (without its token); KeyError when there's none.""" - row = db.one(f"SELECT {FIELDS} FROM $r", r=R("feed", int(fid))) +def _row(db, fid): + """Any feed's row (a collection's too, with its `collection` and `account`), without its token; KeyError.""" + row = db.one(f"SELECT {FIELDS}, collection, account FROM $r", r=R("feed", int(fid))) if not row: raise KeyError(fid) return row +def get(db, fid): + """A namespace's feed's row (without its token); KeyError when there's none (or it's a collection's).""" + row = _row(db, fid) + if row.pop("collection", None) is not None: + raise KeyError(fid) + row.pop("account", None) + return row + + def list_feeds(db, spaces=None): - """Feeds in namespaces `spaces` (None: every one), oldest first, each with how many items it holds.""" + """Namespaces' feeds in `spaces` (None: every one), oldest first, each with how many items it holds.""" rows = db.rows( - f"SELECT {FIELDS} FROM feed" + (" WHERE space IN $s" if spaces is not None else "") + " ORDER BY id", s=sorted(spaces or []) + f"SELECT {FIELDS} FROM feed WHERE collection = NONE" + (" AND space IN $s" if spaces is not None else "") + " ORDER BY id", + s=sorted(spaces or []), ) return [{**r, **_fill(db, r["id"])} for r in rows] @@ -219,11 +235,18 @@ def view(db, cfg, row, base): "max_items": row.get("max_items") or DEFAULTS["max_items"], "url": address(base, token), "path": PATH + token, - "page": f"{base}/feeds/{row['id']}", + "page": _page(base, row), } ) +def _page(base, feed): + """Where a feed shows in the web app: its collection's page, or the feed's own.""" + if feed.get("collection") is not None: + return f"{base}/collections/{feed['collection']}" + return f"{base}/feeds/{feed['id']}" + + def token_of(db, cfg, fid): row = db.one("SELECT token FROM $r", r=R("feed", int(fid))) if not row: @@ -232,8 +255,8 @@ def token_of(db, cfg, fid): def rotate(db, cfg, fid): - """A new token for a feed: the old address stops working at once. The new token.""" - get(db, fid) + """A new token for a feed (a collection's too): the old address stops working at once. The new token.""" + _row(db, fid) token = TOKEN_PREFIX + secrets.token_urlsafe(32) db.q( "UPDATE $r SET token_hash = $h, token = $t, updated_at = $u", @@ -253,8 +276,13 @@ def by_token(db, token): """The feed whose token this is; KeyError for any other.""" if not isinstance(token, str) or not TOKEN_RX.match(token): raise KeyError("token") - row = db.one(f"SELECT {FIELDS} FROM feed WHERE token_hash = $h LIMIT 1", h=_hash(token)) - if not row or podcasts._vault(db, row["space"]): # a namespace made a vault since: its feeds stop + row = db.one(f"SELECT {FIELDS}, collection, account FROM feed WHERE token_hash = $h LIMIT 1", h=_hash(token)) + if not row: + raise KeyError("token") + if row.get("collection") is not None: # its collection's name and description, as they are now + col, _ = _reader(db, row) + return {**row, "title": _line(col["name"], TITLE_MAX) or "Collection", "description": col.get("description")} + if podcasts._vault(db, row["space"]): # a namespace made a vault since: its feeds stop raise KeyError("token") return row @@ -268,6 +296,139 @@ def seen(db, feed, now=None): db.q("UPDATE $r SET fetched_at = $t", r=R("feed", feed["id"]), t=now.isoformat(timespec="seconds")) +# ---------- a saved collection's podcast feed ---------- +def for_collection(db, cfg, cid, account, by=None): + """(the id of person `account`'s podcast feed of saved collection `cid`, whether it was made now): made the first + time they ask. Whether they may read the collection is the caller's to check.""" + q = "SELECT VALUE record::id(id) FROM feed WHERE collection = $c AND account = $a" + have = sorted(db.values(q, c=int(cid), a=int(account))) + if have: + return have[0], False + fid, token, t = db.next_id("feed"), TOKEN_PREFIX + secrets.token_urlsafe(32), store.now() + db.q( + "CREATE $r CONTENT $d", + r=R("feed", fid), + d=store.clean( + { + **DEFAULTS, + "kind": "podcast", + "title": "Collection", # read from the collection whenever the feed is + "collection": int(cid), + "account": int(account), + "token_hash": _hash(token), + "token": settings.seal(cfg, token, _context(fid)), + "created_by": by, + "created_at": t, + "updated_at": t, + } + ), + ) + have = sorted(db.values(q, c=int(cid), a=int(account))) + for extra in have[1:]: # two first asks at once: the first one made stays + db.q("DELETE $r", r=R("feed", extra)) + return have[0], have[0] == fid + + +def collection_feed(db, cfg, fid, base): + """A collection's feed as its person sees it: its address, how many episodes it lists now, and when a podcast app + last read it.""" + row = _row(db, fid) + token = token_of(db, cfg, fid) + return store.clean( + { + "id": row["id"], + "collection": row["collection"], + "kind": row["kind"], + "url": address(base, token), + "path": PATH + token, + "page": _page(base, row), + "episodes": len(_episodes(db, cfg, row)), + "max_items": row.get("max_items") or DEFAULTS["max_items"], + "fetched_at": row.get("fetched_at"), + "created_at": row.get("created_at"), + "updated_at": row.get("updated_at"), + } + ) + + +def drop_collection(db, cid, account=None): + """Delete a collection's feeds (one person's, with `account`): their addresses stop working at once.""" + if account is None: + db.q("DELETE feed WHERE collection = $c", c=int(cid)) + else: + db.q("DELETE feed WHERE collection = $c AND account = $a", c=int(cid), a=int(account)) + + +def _reader(db, feed): + """(the collection, the namespaces its person may read now, vaults left out) for a collection's feed; KeyError + when the collection is gone or no longer shared with them, or their account is disabled.""" + from . import auth # auth reaches what imports this + + who = auth.active_account(db, feed.get("account")) + if not who: + raise KeyError("account") + col = recsets.get(db, feed["collection"]) + if col["account"] != who["id"] and not col.get("shared"): + raise KeyError("collection") + spaces = {s for s in auth.permissions(db, who) if not podcasts._vault(db, s)} + return col, spaces + + +def _episodes(db, cfg, feed): + """A collection feed's episodes, newest first: [(the recording's row, (content type, size))] of its assets with + audio or video, the newest `max_items`. KeyError as _reader.""" + col, spaces = _reader(db, feed) + ids = recsets.members(db, col, spaces) + if not ids: + return [] + rows = db.rows( + "SELECT record::id(id) AS id, space, title, summary, duration_ms, recorded_at, created_at FROM recording " + "WHERE id IN $ids AND source = 'audio'", + ids=[R("recording", i) for i in ids], + ) + rows.sort(key=lambda r: (str(r.get("recorded_at") or r.get("created_at") or ""), r["id"]), reverse=True) + out, limit = [], feed.get("max_items") or DEFAULTS["max_items"] + for r in rows: + if len(out) >= limit: + break + got = media(db, cfg, r["id"]) + if got: + out.append((r, got)) + return out + + +def _when(*values): + """The first of these that is a time, as ISO 8601 in UTC; now when none is.""" + for v in values: + if not v: + continue + try: + return events._parse(v).isoformat(timespec="seconds") + except (TypeError, ValueError): + continue + return store.now() + + +def _collection_shown(db, cfg, feed, base, token): + """_shown for a collection's feed: each episode an asset of it with audio or video.""" + out = [] + for rec, (ctype, size) in _episodes(db, cfg, feed): + summary = rec.get("summary") + out.append( + { + "title": rec.get("title") or "Untitled", + "guid": f"lens:feed-{feed['id']}:recording/{rec['id']}", + "published": _when(rec.get("recorded_at"), rec.get("created_at"), feed.get("created_at")), + "categories": [], + "description": (summary.get("summary") if isinstance(summary, dict) else None) or "", + "link": f"{base}/resources/{rec['id']}", + "enclosure": {"url": f"{address(base, token)}/items/{rec['id']}/audio", "type": ctype, "length": size}, + "duration_ms": rec.get("duration_ms"), + } + ) + return out + + # ---------- what may go in ---------- class Fit: """Whether what's in a namespace may show in a feed of namespace `space`: the same namespace, or one everyone who @@ -473,7 +634,14 @@ def media(db, cfg, rid): def item_recording(db, feed, iid): - """The recording behind item `iid` of `feed`, for its enclosure; KeyError when it has none (or may not show).""" + """The recording behind item `iid` of `feed`, for its enclosure; KeyError when it has none (or may not show). A + collection's feed names its assets by their ids.""" + if feed.get("collection") is not None: + col, spaces = _reader(db, feed) + rec = db.one("SELECT * FROM $r", r=R("recording", int(iid))) + if not rec or int(iid) not in recsets.members(db, col, spaces): + raise KeyError(iid) + return int(iid), rec it = db.one("SELECT recording FROM $r WHERE feed = $f", r=R("feed_item", int(iid)), f=feed["id"]) if not it or it.get("recording") is None: raise KeyError(iid) @@ -519,6 +687,8 @@ def _shown(db, cfg, feed, base, token): """The items a reader sees, newest first, each as {title, link, guid, published, description, categories, enclosure}: resources read as they are now, episodes once they're made, and only items with audio in a podcast feed.""" + if feed.get("collection") is not None: + return _collection_shown(db, cfg, feed, base, token) fits, out = Fit(db, feed["space"]), [] limit = feed.get("max_items") or DEFAULTS["max_items"] rows = items(db, feed["id"], limit=min(KEEP, limit * 3 if feed["kind"] == "podcast" else limit)) @@ -600,7 +770,7 @@ def render_xml(db, cfg, feed, base, token): f'', "", _el("title", feed["title"]), - _el("link", f"{base}/feeds/{feed['id']}"), + _el("link", _page(base, feed)), _el("description", _html(feed.get("description") or feed["title"])), _el("language", feed.get("language") or DEFAULTS["language"]), _el("lastBuildDate", _rfc822(built)), @@ -609,7 +779,7 @@ def render_xml(db, cfg, feed, base, token): _el("atom:link", href=me, rel="self", type="application/rss+xml"), ] if feed.get("image"): - parts.append(f"{_el('url', feed['image'])}{_el('title', feed['title'])}{_el('link', f'{base}/feeds/{feed["id"]}')}") + parts.append(f"{_el('url', feed['image'])}{_el('title', feed['title'])}{_el('link', _page(base, feed))}") if pod: parts += [ _el("itunes:author", feed.get("author") or "Lens"), diff --git a/fastapi_backend/app/schemas/feeds.py b/fastapi_backend/app/schemas/feeds.py index cfb211b11..13f3af829 100644 --- a/fastapi_backend/app/schemas/feeds.py +++ b/fastapi_backend/app/schemas/feeds.py @@ -93,3 +93,17 @@ class FeedItem(ResponseModel): origin: dict[str, Any] | None = Field(None, description="the routine run and workflow node that added it") by: str | None = None created_at: str | None = None + + +class CollectionFeed(ResponseModel): + id: int + collection: int + kind: FeedKind = "podcast" + url: str = Field(description="your private address for the collection's podcast feed: share it like a password") + path: str = Field(description="the address without the server") + page: str = Field(description="the collection's page in the web app") + episodes: int = Field(description="how many episodes it lists now: the collection's assets with audio or video you can read") + max_items: int = Field(description="the most episodes it lists, newest first") + fetched_at: str | None = Field(None, description="when a podcast app last read it") + created_at: str | None = None + updated_at: str | None = None diff --git a/fastapi_backend/tests/api/test_feeds.py b/fastapi_backend/tests/api/test_feeds.py index a2b282481..05bf6c2e9 100644 --- a/fastapi_backend/tests/api/test_feeds.py +++ b/fastapi_backend/tests/api/test_feeds.py @@ -212,3 +212,96 @@ def test_agents_use_feeds(client, db, cfg, env): assert {i["kind"] for i in one["newest"]} == {"asset", "text"} and any(i.get("url") for i in one["newest"]) assert tool(client, env["ca"], "list_feeds")["feeds"] == [] assert "not found" in tool_error(client, env["ca"], "list_feeds", feed_id=feed["feed_id"]) + + +def _audio(db, folder, rid, name): + """Give a resource audio podcast apps can play.""" + f = folder / name + f.write_bytes(b"ID3" + bytes(range(256)) * 8) + db.q("UPDATE $r SET source = 'audio', path = $p, duration_ms = 61000", r=R("recording", rid), p=str(f)) + return f + + +def _titles(client, path): + return sorted(i.findtext("title") for i in _read(client, path)[0].findall("item")) + + +def test_a_collection_feed_lists_its_assets_with_media(client, db, cfg, env, folder): + a, b, c = env["ids"] + _audio(db, folder, a, "a.mp3") + _audio(db, folder, c, "c.mp3") + cid = client.post( + "/api/v1/collections", + headers=env["root"], + json={"name": "Commute", "description": "For the train", "recordings": [a, b, c], "shared": True}, + ).json()["id"] + assert client.get(f"/api/v1/collections/{cid}/feed").status_code == 401 + mine = client.get(f"/api/v1/collections/{cid}/feed", headers=env["root"]).json() + assert mine["collection"] == cid and mine["kind"] == "podcast" and mine["episodes"] == 2 + assert mine["path"].startswith("/api/v1/feeds/public/lfd_") and mine["url"].endswith(mine["path"]) + assert mine["page"].endswith(f"/collections/{cid}") + assert client.get(f"/api/v1/collections/{cid}/feed", headers=env["root"]).json()["url"] == mine["url"] # the same one after + assert not db.one("SELECT token FROM $r", r=R("feed", mine["id"]))["token"].startswith("lfd_") # sealed + # with no session, a podcast app reads the assets with audio or video (not b, a transcript), as their reader may + ch, r = _read(client, mine["path"]) + assert ch.findtext("title") == "Commute" and ch.findtext("description") == "For the train" + assert ch.findtext(f"{ITUNES}block") == "Yes" and ch.findtext("link").endswith(f"/collections/{cid}") + assert _titles(client, mine["path"]) == ["call", "ep1"] + enc = next(i for i in ch.findall("item") if i.findtext("title") == "ep1").find("enclosure") + assert enc.get("url") == f"{mine['url']}/items/{a}/audio" and enc.get("type") == "audio/mpeg" + got = client.get(enc.get("url").removeprefix("http://127.0.0.1"), headers={"Range": "bytes=0-2"}) + assert got.status_code == 206 and got.content == b"ID3" and got.headers["referrer-policy"] == "no-referrer" + assert client.get(f"{mine['path']}/items/{b}/audio").status_code == 404 # no media: not an episode, not served + assert client.get(mine["path"], headers={"If-None-Match": r.headers["etag"]}).status_code == 304 + # each person's address lists what they may read: a viewer of pods hears ep1 alone, and can't fetch the call + theirs = client.get(f"/api/v1/collections/{cid}/feed", headers=env["vi"]).json() + assert theirs["url"] != mine["url"] and theirs["episodes"] == 1 and _titles(client, theirs["path"]) == ["ep1"] + assert client.get(f"{theirs['path']}/items/{c}/audio").status_code == 404 + # namespaces' feeds don't show collections' feeds + assert client.get("/api/v1/feeds", headers=env["root"]).json() == [] + assert client.get(f"/api/v1/feeds/{mine['id']}", headers=env["root"]).status_code == 404 + assert client.post(f"/api/v1/feeds/{mine['id']}/token", headers=env["root"]).status_code == 404 + # a collection no longer shared stops its other readers' feeds; one never shared has none to give + client.patch(f"/api/v1/collections/{cid}", headers=env["root"], json={"shared": False}) + assert client.get(theirs["path"]).status_code == 404 and _titles(client, mine["path"]) == ["call", "ep1"] + assert client.get(f"/api/v1/collections/{cid}/feed", headers=env["vi"]).status_code == 404 + # a namespace made a vault drops out of the feed + db.q("UPSERT $r SET vault = true", r=R("data_key", store.ns_id(db, "calls"))) + assert _titles(client, mine["path"]) == ["ep1"] + + +def test_a_collection_feed_token_rotates_and_stops(client, db, cfg, env, folder): + a, _b, _c = env["ids"] + _audio(db, folder, a, "a.mp3") + cid = client.post("/api/v1/collections", headers=env["ed"], json={"name": "Mine", "recordings": [a]}).json()["id"] + feed = client.get(f"/api/v1/collections/{cid}/feed", headers=env["ed"]).json() + audio = f"{feed['path']}/items/{a}/audio" + assert client.get(audio).status_code == 200 + # a new address: the old one, and its audio, stop working at once + new = client.post(f"/api/v1/collections/{cid}/feed/token", headers=env["ed"]).json() + assert new["id"] == feed["id"] and new["url"] != feed["url"] + assert client.get(feed["path"]).status_code == 404 and client.get(audio).status_code == 404 + assert _titles(client, new["path"]) == ["ep1"] + assert set(db.values("SELECT VALUE action FROM audit_log")) >= {"feed.create", "feed.token"} + # stopped: the address goes; asking again makes another + assert client.delete(f"/api/v1/collections/{cid}/feed", headers=env["ed"]).json() == {"ok": True} + assert client.get(new["path"]).status_code == 404 + again = client.get(f"/api/v1/collections/{cid}/feed", headers=env["ed"]).json() + assert again["url"] not in (feed["url"], new["url"]) + # a disabled account's feeds stop; a deleted collection's feeds go + db.q("UPDATE $r SET disabled = true", r=R("account", db.values("SELECT VALUE record::id(id) FROM account WHERE email = 'ed@x.io'")[0])) + assert client.get(again["path"]).status_code == 404 + db.q("UPDATE account SET disabled = false WHERE email = 'ed@x.io'") + assert client.get(again["path"]).status_code == 200 + assert client.delete(f"/api/v1/collections/{cid}", headers=env["ed"]).json() == {"ok": True} + assert client.get(again["path"]).status_code == 404 and not db.rows("SELECT id FROM feed WHERE collection = $c", c=cid) + + +def test_agents_get_a_collection_feed(client, db, cfg, env, folder): + a, _b, _c = env["ids"] + _audio(db, folder, a, "a.mp3") + cid = client.post("/api/v1/collections", headers=env["ed"], json={"name": "Mine", "recordings": [a]}).json()["id"] + got = tool(client, env["ed"], "list_feeds", collection_id=cid) + assert got["collection"] == cid and got["episodes"] == 1 and got["url"].startswith("http") + assert got["url"].endswith(client.get(f"/api/v1/collections/{cid}/feed", headers=env["ed"]).json()["path"]) # the same one + assert "not found" in tool_error(client, env["vi"], "list_feeds", collection_id=cid) # not shared with them diff --git a/nextjs-frontend/__tests__/collection-feed.test.ts b/nextjs-frontend/__tests__/collection-feed.test.ts new file mode 100644 index 000000000..e2fe07369 --- /dev/null +++ b/nextjs-frontend/__tests__/collection-feed.test.ts @@ -0,0 +1,21 @@ +import { feedStatus } from "@/components/collections/feed-model"; + +describe("a collection's podcast feed", () => { + const now = Date.parse("2026-10-18T12:00:00Z"); + + it("says how many episodes it lists and whether a podcast app reads it", () => { + expect(feedStatus({ episodes: 3, max_items: 50, fetched_at: "2026-10-18T11:55:00Z" }, now)).toBe( + "3 episodes · a podcast app read it 5 minutes ago", + ); + expect(feedStatus({ episodes: 1, max_items: 50, fetched_at: null }, now)).toBe( + "1 episode · no podcast app has read it yet", + ); + }); + + it("says why it's empty, and when it lists only the newest", () => { + expect(feedStatus({ episodes: 0, max_items: 50 }, now)).toBe( + "No episodes yet: assets with audio or video become episodes · no podcast app has read it yet", + ); + expect(feedStatus({ episodes: 50, max_items: 50, fetched_at: null }, now)).toMatch(/^The newest 50 episodes/); + }); +}); diff --git a/nextjs-frontend/app/openapi-client/index.ts b/nextjs-frontend/app/openapi-client/index.ts index 929ec752b..dd0f1499f 100644 --- a/nextjs-frontend/app/openapi-client/index.ts +++ b/nextjs-frontend/app/openapi-client/index.ts @@ -458,6 +458,7 @@ export type { CollectionEvidenceErrors, CollectionEvidenceResponse, CollectionEvidenceResponses, + CollectionFeed, CollectionMember, CollectionMemberSet, CollectionNode, @@ -837,6 +838,11 @@ export type { DeleteCollectionData, DeleteCollectionError, DeleteCollectionErrors, + DeleteCollectionFeedData, + DeleteCollectionFeedError, + DeleteCollectionFeedErrors, + DeleteCollectionFeedResponse, + DeleteCollectionFeedResponses, DeleteCollectionResponse, DeleteCollectionResponses, DeleteCommentData, @@ -1363,6 +1369,11 @@ export type { GetCollectionData, GetCollectionError, GetCollectionErrors, + GetCollectionFeedData, + GetCollectionFeedError, + GetCollectionFeedErrors, + GetCollectionFeedResponse, + GetCollectionFeedResponses, GetCollectionFieldsData, GetCollectionFieldsError, GetCollectionFieldsErrors, @@ -3431,6 +3442,11 @@ export type { Roles, RoleUpdate, RollbackAsk, + RotateCollectionFeedData, + RotateCollectionFeedError, + RotateCollectionFeedErrors, + RotateCollectionFeedResponse, + RotateCollectionFeedResponses, RotateFeedTokenData, RotateFeedTokenError, RotateFeedTokenErrors, diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index d394ef7c0..db343b129 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -367,6 +367,9 @@ import type { DeleteCircleResponses, DeleteCollectionData, DeleteCollectionErrors, + DeleteCollectionFeedData, + DeleteCollectionFeedErrors, + DeleteCollectionFeedResponses, DeleteCollectionResponses, DeleteCommentData, DeleteCommentErrors, @@ -628,6 +631,9 @@ import type { GetCircleResponses, GetCollectionData, GetCollectionErrors, + GetCollectionFeedData, + GetCollectionFeedErrors, + GetCollectionFeedResponses, GetCollectionFieldsData, GetCollectionFieldsErrors, GetCollectionFieldsResponses, @@ -1655,6 +1661,9 @@ import type { RevokeTokenData, RevokeTokenErrors, RevokeTokenResponses, + RotateCollectionFeedData, + RotateCollectionFeedErrors, + RotateCollectionFeedResponses, RotateFeedTokenData, RotateFeedTokenErrors, RotateFeedTokenResponses, @@ -12043,6 +12052,53 @@ export class Collections { ...options, }); } + + /** + * Delete Collection Feed + * + * Stop your podcast feed of the collection: its address stops working at once (asking for it again makes a new + * one). Audited as `feed.delete`. + */ + public static deleteCollectionFeed( + options: Options, + ): RequestResult { + return (options.client ?? client).delete({ + url: "/api/v1/collections/{cid}/feed", + ...options, + }); + } + + /** + * Get Collection Feed + * + * Your private podcast feed of a collection you may read: its address, which a podcast app subscribes to with no + * session (the token in it is the credential), made the first time you ask (audited as `feed.create`) and the same + * one after. Its episodes are + * the collection's assets with audio or video that you can read, as they are when the app reads it. + */ + public static getCollectionFeed( + options: Options, + ): RequestResult { + return (options.client ?? client).get({ + url: "/api/v1/collections/{cid}/feed", + ...options, + }); + } + + /** + * Rotate Collection Feed + * + * A new address for your podcast feed of the collection: the old one stops working at once, so a podcast app + * subscribed to it needs the new one. Audited as `feed.token`. + */ + public static rotateCollectionFeed( + options: Options, + ): RequestResult { + return (options.client ?? client).post({ + url: "/api/v1/collections/{cid}/feed/token", + ...options, + }); + } } export class Batches { diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 52e09d2cf..83343054c 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -4924,6 +4924,69 @@ export type CollectionDetail = { [key: string]: unknown; }; +/** + * CollectionFeed + */ +export type CollectionFeed = { + /** + * Id + */ + id: number; + /** + * Collection + */ + collection: number; + /** + * Kind + */ + kind?: "rss" | "podcast"; + /** + * Url + * + * your private address for the collection's podcast feed: share it like a password + */ + url: string; + /** + * Path + * + * the address without the server + */ + path: string; + /** + * Page + * + * the collection's page in the web app + */ + page: string; + /** + * Episodes + * + * how many episodes it lists now: the collection's assets with audio or video you can read + */ + episodes: number; + /** + * Max Items + * + * the most episodes it lists, newest first + */ + max_items: number; + /** + * Fetched At + * + * when a podcast app last read it + */ + fetched_at?: string | null; + /** + * Created At + */ + created_at?: string | null; + /** + * Updated At + */ + updated_at?: string | null; + [key: string]: unknown; +}; + /** * CollectionMember */ @@ -49058,6 +49121,96 @@ export type CollectionEvidenceResponses = { export type CollectionEvidenceResponse = CollectionEvidenceResponses[keyof CollectionEvidenceResponses]; +export type DeleteCollectionFeedData = { + body?: never; + path: { + /** + * Cid + */ + cid: number; + }; + query?: never; + url: "/api/v1/collections/{cid}/feed"; +}; + +export type DeleteCollectionFeedErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type DeleteCollectionFeedError = DeleteCollectionFeedErrors[keyof DeleteCollectionFeedErrors]; + +export type DeleteCollectionFeedResponses = { + /** + * Successful Response + */ + 200: Ok; +}; + +export type DeleteCollectionFeedResponse = DeleteCollectionFeedResponses[keyof DeleteCollectionFeedResponses]; + +export type GetCollectionFeedData = { + body?: never; + path: { + /** + * Cid + */ + cid: number; + }; + query?: never; + url: "/api/v1/collections/{cid}/feed"; +}; + +export type GetCollectionFeedErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type GetCollectionFeedError = GetCollectionFeedErrors[keyof GetCollectionFeedErrors]; + +export type GetCollectionFeedResponses = { + /** + * Successful Response + */ + 200: CollectionFeed; +}; + +export type GetCollectionFeedResponse = GetCollectionFeedResponses[keyof GetCollectionFeedResponses]; + +export type RotateCollectionFeedData = { + body?: never; + path: { + /** + * Cid + */ + cid: number; + }; + query?: never; + url: "/api/v1/collections/{cid}/feed/token"; +}; + +export type RotateCollectionFeedErrors = { + /** + * Validation Error + */ + 422: HttpValidationError; +}; + +export type RotateCollectionFeedError = RotateCollectionFeedErrors[keyof RotateCollectionFeedErrors]; + +export type RotateCollectionFeedResponses = { + /** + * Successful Response + */ + 200: CollectionFeed; +}; + +export type RotateCollectionFeedResponse = RotateCollectionFeedResponses[keyof RotateCollectionFeedResponses]; + export type EstimateBatchData = { body: BatchPlan; path?: never; diff --git a/nextjs-frontend/components/collections/collection-detail.tsx b/nextjs-frontend/components/collections/collection-detail.tsx index 29b1f4616..a5ee0575a 100644 --- a/nextjs-frontend/components/collections/collection-detail.tsx +++ b/nextjs-frontend/components/collections/collection-detail.tsx @@ -8,6 +8,7 @@ import { MessagesSquare, Pencil, Play, + Podcast, SlidersHorizontal, Trash2, } from "lucide-react"; @@ -19,6 +20,7 @@ import { Collections } from "@/app/openapi-client"; import type { Collection } from "@/app/openapi-client/types.gen"; import { CollectionDialog } from "@/components/collections/collection-dialog"; import { describeCollection } from "@/components/collections/describe"; +import { CollectionFeedDialog } from "@/components/collections/feed-dialog"; import { EvidencePartsDialog, type EvidencePart } from "@/components/evidence/parts-dialog"; import { saveBlob, slug } from "@/components/recording/hooks"; import { recordingHref } from "@/components/search/links"; @@ -59,6 +61,7 @@ export function CollectionDetail({ id }: { id: number }) { }, }); const [picking, setPicking] = useState(false); + const [feeding, setFeeding] = useState(false); const evidence = useMutation({ mutationFn: async ({ name, include }: { name: string; include?: EvidencePart[] }) => { const blob = (await data( @@ -172,6 +175,10 @@ export function CollectionDetail({ id }: { id: number }) { > + + void; +}) { + const client = useApiClient(); + const qc = useQueryClient(); + const toast = useToast(); + const key = ["collection-feed", id]; + const feed = useQuery({ + queryKey: key, + queryFn: () => data(Collections.getCollectionFeed({ client, path: { cid: id } })), + enabled: open, + }); + const rotate = useMutation({ + mutationFn: () => data(Collections.rotateCollectionFeed({ client, path: { cid: id } })), + onSuccess: (f) => { + qc.setQueryData(key, f); + toast({ + title: "New address made", + body: "The old one stopped working: add this one to your podcast app.", + tone: "intent", + }); + }, + onError: (e) => + toast({ + title: "Couldn’t make a new address", + body: e instanceof ApiError ? e.message : "Please try again.", + tone: "red", + }), + }); + return ( + + + + + } + > + {feed.isLoading ? ( + + ) : feed.isError ? ( + {feed.error.message} + ) : feed.data ? ( +
+ +

{feedStatus(feed.data)}

+

+ Anyone with this address can listen, so share it like a password. New address stops this one at once. +

+
+ ) : null} +
+ ); +} diff --git a/nextjs-frontend/components/collections/feed-model.ts b/nextjs-frontend/components/collections/feed-model.ts new file mode 100644 index 000000000..bd7dc4946 --- /dev/null +++ b/nextjs-frontend/components/collections/feed-model.ts @@ -0,0 +1,14 @@ +import type { CollectionFeed } from "@/app/openapi-client/types.gen"; +import { count, plural, relative } from "@/lib/format"; + +/** "3 episodes · a podcast app read it 5 minutes ago", or why it lists nothing yet. */ +export function feedStatus(f: Pick, now = Date.now()): string { + const episodes = + f.episodes === 0 + ? "No episodes yet: assets with audio or video become episodes" + : f.episodes >= f.max_items + ? `The newest ${count(f.max_items)} episodes` + : plural(f.episodes, "episode"); + const read = f.fetched_at ? `a podcast app read it ${relative(f.fetched_at, now)}` : "no podcast app has read it yet"; + return `${episodes} · ${read}`; +} diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 5cdf772d0..7d680a369 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -29727,6 +29727,133 @@ } } }, + "/api/v1/collections/{cid}/feed": { + "get": { + "tags": [ + "collections" + ], + "summary": "Get Collection Feed", + "description": "Your private podcast feed of a collection you may read: its address, which a podcast app subscribes to with no\nsession (the token in it is the credential), made the first time you ask (audited as `feed.create`) and the same\none after. Its episodes are\nthe collection's assets with audio or video that you can read, as they are when the app reads it.", + "operationId": "get_collection_feed", + "parameters": [ + { + "name": "cid", + "in": "path", + "required": true, + "schema": { + "type": "integer", + "title": "Cid" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/CollectionFeed" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "delete": { + "tags": [ + "collections" + ], + "summary": "Delete Collection Feed", + "description": "Stop your podcast feed of the collection: its address stops working at once (asking for it again makes a new\none). Audited as `feed.delete`.", + "operationId": "delete_collection_feed", + "parameters": [ + { + "name": "cid", + "in": "path", + "required": true, + "schema": { + "type": "integer", + "title": "Cid" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Ok" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/api/v1/collections/{cid}/feed/token": { + "post": { + "tags": [ + "collections" + ], + "summary": "Rotate Collection Feed", + "description": "A new address for your podcast feed of the collection: the old one stops working at once, so a podcast app\nsubscribed to it needs the new one. Audited as `feed.token`.", + "operationId": "rotate_collection_feed", + "parameters": [ + { + "name": "cid", + "in": "path", + "required": true, + "schema": { + "type": "integer", + "title": "Cid" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/CollectionFeed" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, "/api/v1/batches/estimate": { "post": { "tags": [ @@ -45791,6 +45918,98 @@ ], "title": "CollectionDetail" }, + "CollectionFeed": { + "properties": { + "id": { + "type": "integer", + "title": "Id" + }, + "collection": { + "type": "integer", + "title": "Collection" + }, + "kind": { + "type": "string", + "enum": [ + "rss", + "podcast" + ], + "title": "Kind", + "default": "podcast" + }, + "url": { + "type": "string", + "title": "Url", + "description": "your private address for the collection's podcast feed: share it like a password" + }, + "path": { + "type": "string", + "title": "Path", + "description": "the address without the server" + }, + "page": { + "type": "string", + "title": "Page", + "description": "the collection's page in the web app" + }, + "episodes": { + "type": "integer", + "title": "Episodes", + "description": "how many episodes it lists now: the collection's assets with audio or video you can read" + }, + "max_items": { + "type": "integer", + "title": "Max Items", + "description": "the most episodes it lists, newest first" + }, + "fetched_at": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Fetched At", + "description": "when a podcast app last read it" + }, + "created_at": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Created At" + }, + "updated_at": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Updated At" + } + }, + "additionalProperties": true, + "type": "object", + "required": [ + "id", + "collection", + "url", + "path", + "page", + "episodes", + "max_items" + ], + "title": "CollectionFeed" + }, "CollectionMember": { "properties": { "expires_at": { diff --git a/threat-model.json b/threat-model.json index 9be6faac9..07de91e6c 100644 --- a/threat-model.json +++ b/threat-model.json @@ -144,7 +144,7 @@ "name": "optional", "entry_points": [ "POST /automations/hooks/{token} (an automation's webhook)", - "GET /feeds/public/{token}, /feeds/public/{token}/items/{item}/audio (a published feed)", + "GET /feeds/public/{token}, /feeds/public/{token}/items/{item}/audio (a published feed, or a saved collection's podcast feed)", "automations' params and prompts, POST /automations/preview and /automations/variables", "MQTT 1883, syslog 5514, POST /sensors/push/{token}/{stream}", "SMB share (port 1445 on the host, 4445 in the worker)", @@ -588,11 +588,11 @@ ] }, { - "entry_point": "GET /feeds/public/{token}, /feeds/public/{token}/items/{item}/audio (a published feed)", + "entry_point": "GET /feeds/public/{token}, /feeds/public/{token}/items/{item}/audio (a published feed, or a saved collection's podcast feed)", "parameter": "token in the path, Range, If-None-Match", "attacker_controllable": "yes", "component": "optional", - "caller_must_enforce": "the person shares the address as a password; Lens finds the token by its hash, stops the old address when a new one is made, and lists only what the namespace's readers can read", + "caller_must_enforce": "the person shares the address as a password; Lens finds the token by its hash, stops the old address when a new one is made, and lists only what the namespace's readers can read, or for a collection's feed what its person may read now", "provenance": "documented", "source": "docs/feeds.md \"Private by default\"" }, @@ -1204,7 +1204,7 @@ "property": "P-FEEDS", "violation_symptom": "info-leak", "severity_tier": "security", - "conditions": "a published feed's address opens that feed alone, read-only, with no session; only an editor of the namespace makes one; it lists only items whose resource or activity the readers of the feed's namespace can read, never a vault's, and a namespace made a vault stops its feeds; the token is found by its hash, and rotating or deleting the feed stops the old address at once; text in an item cannot add markup to a reader", + "conditions": "a published feed's address opens that feed alone, read-only, with no session; only an editor of the namespace makes one; it lists only items whose resource or activity the readers of the feed's namespace can read, never a vault's, and a namespace made a vault stops its feeds; a saved collection's podcast feed is one per person who may read the collection, lists the collection's assets with audio or video that person may read when it is read, never a vault's, and stops when the collection is deleted or no longer shared with them or their account is disabled; the token is found by its hash, and rotating or deleting the feed stops the old address at once; text in an item cannot add markup to a reader", "provenance": "documented", "source": "docs/feeds.md \"Private by default\"" }, diff --git a/threat-model.md b/threat-model.md index f6a2e8fd6..7282895ef 100644 --- a/threat-model.md +++ b/threat-model.md @@ -313,7 +313,7 @@ classified by gate only *(inferred, Q8)*; unread modules are listed in Q34. | Python code tools, shipped and admin-created templates, workflows, pipelines, routines | Python, Jinja source, regexes | admin only | callback/code | admin is trusted | *(maintainer, 2026-10)* | | MQTT 1883, syslog 5514, `POST /sensors/push/{token}/{stream}` | payloads, hub logins, token in the path | yes, once enabled | data, size/rate, x-credential, x-prompt-injection | admin enables; operator firewalls; readings, alert patterns and new devices reach automations as outside text (`lens.untrusted`) | *(documented, docs/sensors.md "Sensors are off until an admin turns them on")* | | `POST /automations/hooks/{token}` (an automation's webhook) | body (JSON, a form's text fields or text), query, Standard Webhooks headers, token in the path | yes, anyone given the address | data, size/rate, x-credential, x-prompt-injection | no hook exists until an automation with a webhook trigger is saved; the token is 256 random bits kept only as a hash, and revoked with its trigger; 64 KB and 60 calls a minute per hook; a signature, when sent (or required by a `signed` hook), must match the hook's sealed secret within five minutes; a retried message id is logged once; what was sent is outside text (`lens.untrusted`), wrapped as data in prompts | *(documented, docs/automations.md "No webhook exists until an automation with such a trigger is saved")*. Tested by `fastapi_backend/tests/api/test_automation_hooks.py` `test_a_hook_is_made_for_a_trigger_and_only_its_hash_kept`, `test_a_call_is_a_webhook_received_activity`, `test_size_and_rate_caps`, `test_signed_calls_and_retries`; `fastapi_backend/tests/api/test_automations.py` `test_a_webhook_trigger_gets_its_address_when_saved_and_loses_it_when_dropped` | -| `GET /feeds/public/{token}`, `/feeds/public/{token}/items/{item}/audio` (a published feed) | token in the path, `Range`, `If-None-Match` | yes, anyone given the address | x-credential, resource-name | no feed exists until an editor of a namespace makes one; the token is 256 random bits, found by its hash and sealed with the server key so the namespace's readers see the address again, and a new one stops the old at once; the address reads that feed alone, whose items are bounded by who reads its namespace (P-FEEDS), and `Range` reads only the audio of an item the feed lists | *(documented, docs/feeds.md "Private by default")* | +| `GET /feeds/public/{token}`, `/feeds/public/{token}/items/{item}/audio` (a published feed, or a saved collection's podcast feed) | token in the path, `Range`, `If-None-Match` | yes, anyone given the address | x-credential, resource-name | no feed exists until an editor of a namespace makes one, or someone who may read a saved collection asks for its feed (`GET /collections/{cid}/feed`: their own address); the token is 256 random bits, found by its hash and sealed with the server key so the namespace's readers see the address again, and a new one stops the old at once; the address reads that feed alone, whose items are bounded by who reads its namespace, or for a collection's feed by what its person may read now (P-FEEDS), and `Range` reads only the audio of an item the feed lists | *(documented, docs/feeds.md "Private by default")* | | `POST /circles`, `POST /circles/shares`, MCP `save_circle` and `share_with_circle`, the assistant's and automations' `share_with_circle` (circles and sharing with them) | circle members (account ids or emails), an activity or resource id, a note, `with_item` | yes, anyone signed in; the note and the snapshot reach the recipients | data, resource-name, size, x-prompt-injection | members must be people the sharer may add (P-CIRCLES); Lens checks the sharer can read the item, refuses vaults, caps the note at 2000 characters and circles at 50 of 200 people, and needs `publish` to give the item; the assistant asks first | *(documented, docs/circles.md "What keeps it safe")* | | `POST /circles/when-free`, MCP `when_free`, the assistant's `when_free` (when people in your circles are free) | people (account ids or emails), a window, a time zone, minutes, the hours of the day | yes, anyone signed in | resource-name, size | the person asked must have let a circle with the asker see (P-FREE-BUSY); Lens answers only start and end times, caps the window at 31 days and 20 people, and records each ask in the asked person's timeline | *(documented, docs/circles.md "When they're free")* | | `POST /circles/ask`, MCP `ask_their_agent`, the assistant's `ask_their_agent` (asking someone's agent a question) | the person asked, a question (1000 characters) | yes, anyone signed in; the question reaches a model | data, size, x-prompt-injection | the person asked must have let a circle with the asker ask, and named the namespaces (P-ASK-AGENT); Lens answers from those namespaces only, wraps the question as outside text, and records the question and answer in their timeline | *(documented, docs/circles.md "Ask their agent")* | @@ -557,7 +557,7 @@ Paths in the Voided by column are relative to `fastapi_backend/app/`. Searches c | P-SHARE | A share link opens one asset, read-only. It expires and can be revoked. Only hashes are stored. A dead link answers the same 410 whatever went wrong. | integrity-bypass | security-critical | *(documented, docs/authentication.md "give read-only access to one asset's player and embed")* | Voided by: Liveness plus asset match only — `domain/auth.py:441-444`; `?s=` routes — `api/deps.py:252` (Q7). | | P-GRANT-END | A role in a namespace, a role on a collection or permission on an asset given with an end date opens nothing after it, on every route. Every check reads grants through `auth.permissions`, `hierarchy.roles_of` and `access.granted` / `has_permission`, which leave ended ones out; only someone who may give the grant changes its end. | integrity-bypass | security-critical | *(documented, docs/access.md "After its end a grant opens nothing")* | Voided by: links signed before the end keep opening their path until they expire (N-SIGNED-BEARER); a read of `membership`, `collection_role` or `permission` that skips `auth.live` — `domain/auth.py` `live`. | | P-SAVE-OFF | With an asset's downloads off, its own file (`/media`, the file tree), the PDF made of it and its supplementary files (the API's and IIIF's) answer 403 to anyone below its owners, whatever else lets them see it (a role, a collection role, permission, an IP group, a share link, a signed link), and IIIF manifests stop offering them. A link signed for an owner carries `save=1`, signed with the rest; only owners change the switch. | info-leak | security-critical | *(documented, docs/access.md "saving a copy of the asset's files is for its owners")* | Voided by: what plays (`/audio`, frames, pages, captions, text) stays open by design (N-SAVE-CAPTURE); links signed with `save=1` for an owner open the file for whoever holds them until they expire (N-SIGNED-BEARER) — `api/deps.py` `Access.saves`, `api/iiif.py` `_may_save`, `domain/filetree.py` `Tree._assets`. | -| P-FEEDS | A published feed's address opens that feed alone, read-only, with no session: the feed's XML and the audio of items it lists. Only an editor of the namespace makes a feed, and what it lists is bounded by who already reads that namespace: an item's resource or activity is in the namespace, or in one every reader of the feed's namespace reads too, never a vault's, and a namespace made a vault stops its feeds. The token is found by its hash; rotating or deleting the feed stops the old address at once. Text in an item cannot add markup to a reader. | info-leak | security-critical | *(documented, docs/feeds.md "Private by default")* | Voided by: the token is sealed with the server key, not only hashed, so the namespace's readers (and whoever holds `secret.key`) see the address again — `domain/syndication.py` `token_of`; anyone given the address reads the feed until it is rotated; editors choose what goes in, as does a routine (admins make them) whose workflow nodes publish what they pick, bounded the same way; a resource's title and summary show as they are now, so an edit after publishing shows; the fit is by namespace roles (`domain/syndication.py` `Fit`), not per-asset grants, so an asset later closed by access level still shows while its namespace fits. | +| P-FEEDS | A published feed's address opens that feed alone, read-only, with no session: the feed's XML and the audio of items it lists. Only an editor of the namespace makes a feed, and what it lists is bounded by who already reads that namespace: an item's resource or activity is in the namespace, or in one every reader of the feed's namespace reads too, never a vault's, and a namespace made a vault stops its feeds. A saved collection's podcast feed is one per person who may read the collection: it lists the collection's assets with audio or video that person may read when the feed is read, never a vault's, and stops when the collection is deleted or no longer shared with them, or their account is disabled. The token is found by its hash; rotating or deleting the feed stops the old address at once. Text in an item cannot add markup to a reader. | info-leak | security-critical | *(documented, docs/feeds.md "Private by default")* | Voided by: the token is sealed with the server key, not only hashed, so the namespace's readers (and whoever holds `secret.key`) see the address again — `domain/syndication.py` `token_of`; anyone given the address reads the feed until it is rotated; editors choose what goes in, as does a routine (admins make them) whose workflow nodes publish what they pick, bounded the same way; a resource's title and summary show as they are now, so an edit after publishing shows; the fit is by namespace roles (`domain/syndication.py` `Fit`), not per-asset grants, so an asset later closed by access level still shows while its namespace fits; a collection's feed follows its person's namespace roles and the collection as they are when it is read, so whoever holds the address hears what that person may read now, assets added later included, and an app or read-only API token acting for the person can ask for the address, which outlives the app's or token's revocation until it is rotated (`domain/syndication.py` `_reader`). | | P-CIRCLES | A share reaches only people in the sharer's circles, and a circle holds only people the sharer shares a namespace with, who have them in a circle, or anyone for an admin, so a circle doesn't reveal who has an account. A share carries only an activity or resource the sharer can read, never one in a vault; it gives the item itself (permission on that one resource) only when the sharer holds `publish` in its namespace. A received share is seen by its recipient alone and belongs to no namespace. Undo deletes the share from every timeline and takes back the permissions it gave. The assistant always asks before sharing, an automation's `share_with_circle` asks too unless its person let that node run on its own, and read-only tokens can't share. | info-leak | security-critical | *(documented, docs/circles.md "What keeps it safe")* | Voided by: the snapshot is kept as it was when shared, so a later edit or deletion of the item doesn't reach it; deleting a circle doesn't take back what was shared with it; a recipient keeps what they read before an undo — `domain/circles.py`. | | P-SCALING | Auto scaling launches nothing that costs money unless a person, signed in to the web app, approved the group's terms: its provider, what each unit is, its max, its budget and what happens at its cap. An API key, an app, MCP and the assistant acting on its own never approve, and a change that widens the terms takes the approval away. Only a person signed in to the web app lets go a unit its provider runs with Lens's tag that Lens has no unit for (a stray), since Lens may not have started it. A provider's credentials are sealed per provider and field, opened only for a call to it, and never shown. A unit is handed no long-lived Lens secret to start, and when it goes it leaves the mesh: its unspent join token is withdrawn and the node that joined with it is removed. | integrity-bypass | security-critical | *(documented, docs/scaling.md "The person approves")* | Voided by: an admin's web session approves, and an admin is trusted; a yes on the Approvals page to the assistant's proposal approves the terms that proposal names; within the terms agents may raise the desired count up to max, spending up to the budget; costs are estimates from list prices (`domain/scaling.py` `accrue`), not bills, so a provider can charge more than the budget counts; a cloud unit's user-data holds a join token for that unit, usable once within an hour, which anyone with the cloud account or a process on that machine reading its metadata can see until it is spent (`domain/scaling_drivers.py` `user_data`); a cloud provider's API token can do whatever its account can, and Lens calls only the provider's fixed API host with it (`domain/scaling_clouds.py`); a Lambda unit carries an SSH key of the account's, so whoever holds that key's private half can sign in to the instance (`domain/scaling_clouds.py` `Lambda`); a Proxmox provider's address is the admin's to give, and turning `verify_tls` off trusts any server answering there with the API token; a Proxmox unit's first-boot script, with its join token, is passed to the clone through the guest agent (`domain/scaling_proxmox.py`); a RunPod group's vLLM API key (one for all its units, so model calls can go to any of them) and the provider's Hugging Face token are in each pod's environment, which anyone with the RunPod account can read, and the unit's model server is reachable from the internet through RunPod's proxy behind that key (`domain/scaling_models.py`); a Hugging Face unit is a public Inference Endpoint behind its group's vLLM key, which Hugging Face keeps as an encrypted endpoint secret, and the provider's Hugging Face token can manage every endpoint of its account (`domain/scaling_models.py` `HuggingFace`); a SageMaker unit's model calls are signed with the provider's AWS key, which is kept sealed on the unit and, while the unit serves, as the LLM provider's key, so whoever can read Lens's sealed settings can do with it whatever those keys can; the provider's Hugging Face token is in the SageMaker model's environment, which anyone who may describe the account's models can read (`domain/scaling_sagemaker.py`, `domain/llm.py` `chat_request`); an AWS key given by hand as the LLM provider's key, for a SageMaker endpoint or Amazon Bedrock, only signs calls to that runtime and is never sent, but can do whatever its IAM policy allows, so it should allow only invoking the model (`domain/llm.py`, `domain/bedrock.py`); a Bedrock unit's provisioned throughput is billed by the hour with no commitment until Lens deletes it, at the price the admin gives, and its model calls are signed with the provider's AWS key, kept sealed on the unit and, while it serves, as the LLM provider's key (`domain/scaling_bedrock.py`); while a RunPod, Hugging Face, SageMaker or Bedrock unit serves model calls, they go, with the text they carry, to that rented GPU through the host's address (`domain/scaling.py` `_serve`); a Lambda model unit's first-boot script holds its certificate's key and its group's vLLM key, which anyone with the Lambda account can read, its port 8000 is open to the internet through the firewall ruleset Lens makes (`-models`) behind that key, and while it serves `llm.tls_ca` trusts the group's own CA for any name, so a certificate that CA signed for one unit passes for another of the group (`domain/scaling_models.py` `LambdaModels`, `domain/scaling.py` `unit_tls`); an SSH provider's host keys are trusted on first use, when an admin saves or checks it, and held to after (a host that answered first with another key is trusted until taken off), and its private key signs in as the given users on the given hosts (`domain/scaling_ssh.py`). | | P-FREE-BUSY | A person's busy times reach only themselves and people in a circle of theirs that lets `free_busy`, which nothing but they turn on: only each event's start and end, never what it is, where or with whom, and never from a vault. A window is at most 31 days, and every ask shows in the asked person's own timeline (`agent.asked`). | info-leak | security-critical | *(documented, docs/circles.md "When they're free")* | Voided by: an event they organise or are invited to by their account's email counts wherever it is held, if they can read it; asks repeated over time show the shape of their days to those they let see; a repeating event counts by its first time only — `domain/circles_ask.py`. | diff --git a/threat-model.yaml b/threat-model.yaml index 82f8a76b5..58a31449f 100644 --- a/threat-model.yaml +++ b/threat-model.yaml @@ -501,15 +501,15 @@ entry_points: - "fastapi_backend/tests/api/test_automation_hooks.py::test_size_and_rate_caps" - "fastapi_backend/tests/api/test_automation_hooks.py::test_signed_calls_and_retries" - "fastapi_backend/tests/api/test_automations.py::test_a_webhook_trigger_gets_its_address_when_saved_and_loses_it_when_dropped" - - id: "GET /feeds/public/{token}, /feeds/public/{token}/items/{item}/audio (a published feed)" + - id: "GET /feeds/public/{token}, /feeds/public/{token}/items/{item}/audio (a published feed, or a saved collection's podcast feed)" component: optional parameters: - name: "token in the path, Range, If-None-Match" attacker_controllable: true - x-condition: "anyone given the address, once an editor of a namespace makes a feed" + x-condition: "anyone given the address, once an editor of a namespace makes a feed or someone who may read a saved collection asks for its feed" control_kinds: [x-credential, resource-name] obligation_id: keep-feed-addresses-secret - caller_must_enforce: "the person shares the address as a password; Lens finds the token by its hash, stops the old address when a new one is made, and lists only what the namespace's readers can read" + caller_must_enforce: "the person shares the address as a password; Lens finds the token by its hash, stops the old address when a new one is made, and lists only what the namespace's readers can read, or for a collection's feed what its person may read now" provenance: {kind: documented, source: 'docs/feeds.md "Private by default"'} - id: "POST /circles and /circles/shares, MCP save_circle and share_with_circle, the assistant's and automations' share_with_circle (circles and sharing with them)" component: authz @@ -1718,9 +1718,9 @@ properties_claimed: kind: confidentiality components: [authz, optional] tier: security-critical - conditions: "a published feed's address opens that feed alone, read-only, with no session; only an editor of the namespace makes one; it lists only items whose resource or activity the readers of the feed's namespace can read, never a vault's, and a namespace made a vault stops its feeds; the token is found by its hash, and rotating or deleting the feed stops the old address at once; text in an item cannot add markup to a reader" + conditions: "a published feed's address opens that feed alone, read-only, with no session; only an editor of the namespace makes one; it lists only items whose resource or activity the readers of the feed's namespace can read, never a vault's, and a namespace made a vault stops its feeds; a saved collection's podcast feed is one per person who may read the collection, lists the collection's assets with audio or video that person may read when it is read, never a vault's, and stops when the collection is deleted or no longer shared with them or their account is disabled; the token is found by its hash, and rotating or deleting the feed stops the old address at once; text in an item cannot add markup to a reader" violation_symptoms: [info-leak] - x-voided-by: "the token is sealed with the server key, not only hashed, so the namespace's readers and whoever holds secret.key see the address again (domain/syndication.py token_of); anyone given the address reads the feed until it is rotated; editors choose what goes in, as does a routine (admins make them) whose workflow nodes publish what they pick, bounded the same way; a resource's title and summary show as they are now; the fit is by namespace roles (domain/syndication.py Fit), not per-asset grants" + x-voided-by: "the token is sealed with the server key, not only hashed, so the namespace's readers and whoever holds secret.key see the address again (domain/syndication.py token_of); anyone given the address reads the feed until it is rotated; editors choose what goes in, as does a routine (admins make them) whose workflow nodes publish what they pick, bounded the same way; a resource's title and summary show as they are now; the fit is by namespace roles (domain/syndication.py Fit), not per-asset grants; a collection's feed follows its person's namespace roles and the collection as they are when it is read, and an app or read-only API token acting for the person can ask for the address, which outlives the app's or token's revocation until rotated (domain/syndication.py _reader)" provenance: {kind: documented, source: 'docs/feeds.md "Private by default"'} - id: P-CIRCLES kind: confidentiality