Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
confirm. Export an asset or a collection as an evidence PDF with everything Lens found about it, or the parts you
pick. Publish a namespace's resources, episodes, timeline activities or text as a private RSS feed or a podcast
feed that any feed reader or podcast app follows (Feeds, from the Podcasts page), and let a routine fill it on a schedule with
what's new, or make an episode about the week's new resources. ([Podcasts](docs/podcasts.md),
what's new, or make an episode about the week's new resources. Follow a saved collection in your podcast app: its
page gives your own private address for it, listing its assets with audio or video that you can read.
([Podcasts](docs/podcasts.md),
[specs from transcripts](docs/notes.md#specs-from-transcripts), [notes](docs/notes.md),
[evidence PDFs](docs/api.md#evidence-pdfs), [published feeds](docs/feeds.md);
[checks](docs/requirements.md#learn-from-it))
Expand Down
7 changes: 7 additions & 0 deletions changelog.d/collection-feed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
- **Follow a collection in your podcast app.** A saved collection's page has **Podcast feed**: your private address
for it, with Copy and New address. Its episodes are the collection's assets with audio or video that you can read,
with the same RSS and Apple Podcasts' tags, audio and private-address rules as a namespace's podcast feed. Everyone
who may read the collection gets an address of their own, and it stops when the collection is deleted or no longer
shared with them. Agents get it with `GET /api/v1/collections/{id}/feed` or the MCP tool `list_feeds` with
`collection_id` ([A collection's podcast feed](docs/feeds.md#a-collections-podcast-feed)).

9 changes: 8 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1799,9 +1799,16 @@ GET /api/v1/collections/{cid}
PATCH /api/v1/collections/{cid}
DELETE /api/v1/collections/{cid}
GET /api/v1/collections/{cid}/evidence.pdf
GET /api/v1/collections/{cid}/feed
POST /api/v1/collections/{cid}/feed/token
DELETE /api/v1/collections/{cid}/feed
```

`GET /{cid}/evidence.pdf` is the collection as one [evidence PDF](#evidence-pdfs).
`GET /{cid}/evidence.pdf` is the collection as one [evidence PDF](#evidence-pdfs). `GET /{cid}/feed` is your private
podcast feed of it: its address (`url`, read by podcast apps with no session), how many `episodes` it lists (its
assets with audio or video that you can read) and when an app last read it, made the first time you ask;
`POST /{cid}/feed/token` gives it a new address and `DELETE /{cid}/feed` stops it
([A collection's podcast feed](feeds.md#a-collections-podcast-feed)).

## views

Expand Down
33 changes: 31 additions & 2 deletions docs/feeds.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Published feeds (RSS and podcasts)

**Status: built.** Feeds, their private addresses, the API, the MCP tools, the workflow nodes that let a routine fill a
feed or make an episode on a schedule, and the Feeds page in the web app.
feed or make an episode on a schedule, the Feeds page in the web app, and a saved collection's podcast feed, with its
address on the collection's page.

A namespace can publish what's in it as a feed that any feed reader or podcast app follows: new resources, podcast
episodes, things on the timeline, or text a person or an agent wrote. Each feed has a private address; add it to the
Expand Down Expand Up @@ -66,6 +67,33 @@ new address (the old token is gone for good). What a feed listed stays in Lens e
On the canvas, a graph workflow's palette has the publishing nodes under Data (Select assets, Select activities) and
Keep (Publish to feed, Podcast episode); a feed is picked from a list, and the canvas names it on the node.

## A collection's podcast feed

A saved collection ([collections](api.md#collections)) is a podcast of its own: its episodes are the assets in it
with audio or video, newest first (the newest 50), and a filter's new matches arrive as new episodes. On the
collection's page, **Podcast feed** shows your address for it with **Copy**: paste it into your podcast app as a show
added by its address. Nothing has to be made first, and nobody but you has to do anything.

- **One address per person.** Everyone who may read the collection (its owner, and everyone when it's shared) has
their own address, and it lists only what they can read when their podcast app reads it, so sharing a collection
never shows anyone else's assets. A vault's assets are never in it.
- **The same private address as a namespace's feed.** It is an `lfd_` token under `/api/v1/feeds/public/`, kept
sealed and found by its hash, with the same RSS 2.0 and Apple Podcasts' tags, the same audio with byte ranges, and
the same requests to directories and search engines to keep out. **New address** (rotate) stops the old one at
once.
- **It stops when it should.** Deleting the collection deletes its feeds. When the collection is no longer shared
with you, or your account is disabled, your address stops working.
- Its title and description are the collection's, as they are when the app reads it.

| | |
|---|---|
| `GET /api/v1/collections/{id}/feed` | your podcast feed of a collection you may read: its private address (`url`), how many `episodes` it lists now and when a podcast app last read it (`fetched_at`). Made the first time you ask (audited as `feed.create`), the same one after |
| `POST /api/v1/collections/{id}/feed/token` | a new address; the old one stops working. Audited as `feed.token` |
| `DELETE /api/v1/collections/{id}/feed` | stop it: its address stops working (asking again makes a new one). Audited as `feed.delete` |

Agents get the address with the MCP tool `list_feeds` and `collection_id` (on the main MCP server, since a
collection spans namespaces). A collection's feed isn't in `GET /api/v1/feeds`, which lists namespaces' feeds.

## From a routine

A routine fills a feed on a schedule with a graph workflow ([Publishing from the graph](processing.md#publishing-from-the-graph)):
Expand All @@ -92,7 +120,8 @@ what's new, and what the feed's readers couldn't read is refused.
Agents use the MCP tools `list_feeds` (each feed's address, or one feed with its newest items), `create_feed` and
`publish_to_feed` ([MCP](mcp.md)).

Refine later: artwork Lens makes for a feed without an `image`; Atom and JSON Feed alongside RSS; chapters and
Refine later: a collection feed's settings (artwork, how many episodes, its author) in the web app; undo for a new
address; artwork Lens makes for a feed without an `image`; Atom and JSON Feed alongside RSS; chapters and
transcripts in a podcast feed (`podcast:transcript`); an automation action that publishes to a feed when something
happens; a preview of the feed's XML on its page; Try on the canvas listing what Publish to feed would add, item by
item.
2 changes: 1 addition & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
| `schedule_project` | ask a prompt in a project on a schedule, in a new thread each time or the same one; change, switch off or remove a schedule. Needs a write-scope token |
| `create_podcast` | make a two-host learning [episode](podcasts.md) about assets or lines in them, with an angle, a length and a style; it's made in the background. Needs a write-scope token and editor access where it goes |
| `get_podcast` | an episode: how far it has got, its script with each line's speaker and the sources it cites (links), and what the fact-check changed |
| `list_feeds` | the RSS and podcast [feeds](feeds.md) your namespaces publish, each with its private address and how many items it holds; with `feed_id`, one feed and its newest items |
| `list_feeds` | the RSS and podcast [feeds](feeds.md) your namespaces publish, each with its private address and how many items it holds; with `feed_id`, one feed and its newest items; with `collection_id`, your private podcast feed of a saved collection (main server only) |
| `create_feed` | make an RSS feed or a podcast feed in a namespace, for feed readers and podcast apps. Needs a write-scope token and editor access |
| `publish_to_feed` | add resources, episodes, timeline activities and a text item to a feed; what's there already isn't added twice, and what may not show there comes back refused, with why. Needs a write-scope token and editor access |
| `write_spec` | write a technical [spec](notes.md#specs-from-transcripts) from a recording's transcript as a note linked to it: every decision, no and constraint in the speaker's own words with links to when they said them, and the gaps left to decide later; it's written in the background. Needs a write-scope token and the right to process the recording |
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ request, to another check that proves the same thing; loosening a requirement or
| A note shows the notes it's inside and the notes inside it, to people and agents | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_path_and_children`, `test_assistant_keeps_notes`; [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_note_tools`; [note-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/note-page.test.tsx) |
| Undo and Recently deleted instead of a confirm | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_recently_deleted`; [notes-delete.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-delete.test.tsx) |
| A namespace publishes a private RSS feed or a podcast feed (Apple Podcasts' tags, audio with byte ranges) of resources, episodes, timeline activities and text, holding only what its readers can already read; through the API and MCP | [test_feeds.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feeds.py) `test_making_filling_and_reading_a_feed`, `test_a_podcast_feed_lists_what_has_audio`, `test_timeline_activities_in_a_feed`, `test_vaults_and_settings`, `test_agents_use_feeds` |
| A saved collection is a private podcast feed of its assets with audio or video: one address per person who may read it, listing only what they can read, from the collection's page in one click with Copy, rotated or stopped like a namespace's feed; through the API and MCP | [test_feeds.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feeds.py) `test_a_collection_feed_lists_its_assets_with_media`, `test_a_collection_feed_token_rotates_and_stops`, `test_agents_get_a_collection_feed`; [collection-feed.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/collection-feed.test.ts) |
| A routine fills a feed on a schedule, or makes a podcast episode about what's new: graph workflow nodes pick the resources or timeline activities since the last run, add them to a feed, or make an episode about them; Try says what they would do | [test_feed_nodes.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feed_nodes.py) `test_a_routine_publishes_new_resources_to_a_feed`, `test_a_routine_publishes_the_timeline_and_text`, `test_a_routine_makes_an_episode_and_the_feed_lists_it_once_made`, `test_try_says_what_it_would_do_and_bad_settings_are_refused` |
| Feeds in the web app: a Feeds page, reached from the Podcasts page and under Explore once there is a feed; New feed with the title focused and Enter making it in the current namespace; a feed's page copies the address in one click, gets a new one, edits what readers show, and takes items out or deletes the feed with Undo instead of a confirm; the publishing nodes on the canvas, checked before Publish | [feeds-ui.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/feeds-ui.test.tsx) `the Feeds page lists each feed with its kind, namespace, items and whether an app reads it`, `New feed opens on the title, defaults to the current namespace, and Enter makes it and opens its page`, `a feed's page copies the address, gets a new one, and takes an item out with Undo`, `deleting a feed is undone by making it again with its items, at a new address`, `editors change what readers show, and Enter in a field saves`; [workflow-flow.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-flow.test.ts) `a Select node into Publish to feed or Podcast episode is a whole graph workflow`, `says what to fix: a feed to choose, the limits, and a graph that publishes nothing`, `names the selection and the feed on the canvas`; [nav-groups.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/nav-groups.test.tsx) `feeds are under Explore, after Podcasts, for everyone`, `Feeds is in the nav only once there is a feed, so a phone's nav stays as short as it was` |
| An asset or a collection exported as an evidence PDF, whole or the parts you pick | [test_evidence.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_evidence.py) `test_a_collection_as_one_evidence_pdf`, `test_everything_lens_found_goes_in`, `test_picking_the_parts`; [evidence-parts.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/evidence-parts.test.tsx) |
Expand Down
18 changes: 16 additions & 2 deletions fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -2364,13 +2364,27 @@ def _feed_out(ctx: Context, row: dict[str, Any]) -> dict[str, Any]:
"List published feeds",
"The RSS and podcast feeds the person's namespaces publish (docs/feeds.md): each one's kind, title, namespace, how "
"many items it holds, and its private address, which feed readers and podcast apps subscribe to. Anyone with the "
"address can read the feed, so give it only to the person. With feed_id, one feed and its newest items.",
"address can read the feed, so give it only to the person. With feed_id, one feed and its newest items. With "
"collection_id, the person's own podcast feed of a saved collection they may read (made the first time they ask): "
"its episodes are the collection's assets with audio or video that they can read.",
Arg("namespace", "string", "only this namespace's feeds"),
Arg("feed_id", "integer", "one feed, with its newest items"),
Arg("collection_id", "integer", "a saved collection: the person's private podcast feed of it"),
)
def list_feeds(ctx: Context, namespace: str | None, feed_id: int | None) -> dict[str, Any]:
def list_feeds(ctx: Context, namespace: str | None, feed_id: int | None, collection_id: int | None) -> dict[str, Any]:
from app.api.v1.routes import collections as collection_routes
from app.domain import syndication

if collection_id is not None:
if ctx.ns: # a saved collection spans namespaces; this server sees one
raise ToolError("a collection's feed is on the main MCP server (/mcp), not a namespace's own")
collection_routes.own_collection(ctx.db, collection_id, ctx.user) # 404: not found
fid, made = syndication.for_collection(ctx.db, ctx.cfg, collection_id, ctx.user.id, by=ctx.user.email)
if made:
auth.audit(ctx.db, ctx.user.as_audit(), "feed.create", f"feed:{fid}", {"collection": collection_id, "via": "mcp"})
f = syndication.collection_feed(ctx.db, ctx.cfg, fid, ctx.web)
keep = ("id", "kind", "collection", "url", "page", "episodes", "fetched_at")
return store.clean({("feed_id" if k == "id" else k): f.get(k) for k in keep})
if feed_id is not None:
row = syndication.get(ctx.db, feed_id) # KeyError: not found
ctx.acl.need(row["space"])
Expand Down
56 changes: 52 additions & 4 deletions fastapi_backend/app/api/v1/routes/collections.py
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
"""Collections: a saved filter over the archive, or a fixed list of recordings. Yours, and others' shared ones.

A collection only ever lists recordings the viewer can read, whoever saved it.
A collection only ever lists recordings the viewer can read, whoever saved it. So does its podcast feed: each person
who may read it has an address of their own (app/domain/syndication.py, docs/feeds.md).
"""

from __future__ import annotations

from typing import Any

from fastapi import APIRouter, HTTPException
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import Response

from app.api import evidence as evidence_api
from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer
from app.domain import evidence, recsets, store
from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer, web_origin
from app.domain import auth, evidence, recsets, store, syndication
from app.domain.store import DB
from app.schemas.collections import Collection, CollectionCreate, CollectionDetail, CollectionUpdate
from app.schemas.common import Created, Ok
from app.schemas.feeds import CollectionFeed
from app.schemas.recordings import EvidencePart

router = APIRouter(prefix="/collections", tags=["collections"])
Expand Down Expand Up @@ -102,4 +104,50 @@ def update_collection(cid: int, body: CollectionUpdate, user: Writer, db: Db) ->
def delete_collection(cid: int, user: Writer, db: Db) -> Ok:
own_collection(db, cid, user, write=True)
db.q("DELETE $r", r=store.R("saved_collection", cid))
syndication.drop_collection(db, cid) # its podcast feeds' addresses stop working
return Ok()


# ---------- its podcast feed (docs/feeds.md#a-collections-podcast-feed) ----------
def _feed_id(db: DB, cfg: dict[str, Any], cid: int, user: Principal) -> int:
"""This person's feed of the collection, made (and audited as `feed.create`) the first time they ask."""
fid, made = syndication.for_collection(db, cfg, cid, user.id, by=user.email)
if made:
auth.audit(db, user.as_audit(), "feed.create", f"feed:{fid}", {"kind": "podcast", "collection": cid})
return fid


def _feed(db: DB, cfg: dict[str, Any], request: Request, cid: int, user: Principal) -> CollectionFeed:
fid = _feed_id(db, cfg, cid, user)
return CollectionFeed.model_validate(syndication.collection_feed(db, cfg, fid, web_origin(request)))


@router.get("/{cid}/feed")
def get_collection_feed(cid: int, request: Request, user: CurrentUser, db: Db, cfg: Cfg) -> CollectionFeed:
"""Your private podcast feed of a collection you may read: its address, which a podcast app subscribes to with no
session (the token in it is the credential), made the first time you ask (audited as `feed.create`) and the same
one after. Its episodes are
the collection's assets with audio or video that you can read, as they are when the app reads it."""
own_collection(db, cid, user)
return _feed(db, cfg, request, cid, user)


@router.post("/{cid}/feed/token")
def rotate_collection_feed(cid: int, request: Request, user: Writer, db: Db, cfg: Cfg) -> CollectionFeed:
"""A new address for your podcast feed of the collection: the old one stops working at once, so a podcast app
subscribed to it needs the new one. Audited as `feed.token`."""
own_collection(db, cid, user)
fid = _feed_id(db, cfg, cid, user)
syndication.rotate(db, cfg, fid)
auth.audit(db, user.as_audit(), "feed.token", f"feed:{fid}", {"collection": cid})
return _feed(db, cfg, request, cid, user)


@router.delete("/{cid}/feed")
def delete_collection_feed(cid: int, user: Writer, db: Db) -> Ok:
"""Stop your podcast feed of the collection: its address stops working at once (asking for it again makes a new
one). Audited as `feed.delete`."""
own_collection(db, cid, user)
syndication.drop_collection(db, cid, user.id)
auth.audit(db, user.as_audit(), "feed.delete", f"saved_collection:{cid}")
return Ok()
Loading
Loading