Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
A query made inside another on the same thread now runs on that connection
([Concurrency](docs/database.md#concurrency)).

- **Published feeds.** A namespace publishes a private RSS feed, or a podcast feed with Apple Podcasts' tags, that any
feed reader or podcast app follows at its address: resources (with their audio as the enclosure), podcast episodes
once they're made (with the sources they cite), timeline activities and text. A feed holds only what everyone who
reads its namespace can already read, never a vault's; its token is all a reader needs, so it's kept sealed, a new
one stops the old address at once, and the feed asks directories and search engines not to list it. Editors make and
fill feeds with `POST /api/v1/feeds` and `POST /api/v1/feeds/{id}/items`; agents use the MCP tools `list_feeds`,
`create_feed` and `publish_to_feed` ([Published feeds](docs/feeds.md)).
## 0.14.0 <small>October 17, 2026</small> {id="0.14.0"}

- **Dates read as days, for agents.** With Settings → Analysis → Read dates on (off by default), analysis reads the
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,11 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
are your own) and each note shows the notes it's inside and the ones inside it, with today's note one click away,
templates to start from, a search by their words, a download as Markdown, and Undo and Recently deleted instead of a
confirm. Export an asset or a collection as an evidence PDF with everything Lens found about it, or the parts you
pick. ([Podcasts](docs/podcasts.md), [specs from transcripts](docs/notes.md#specs-from-transcripts),
[notes](docs/notes.md), [evidence PDFs](docs/api.md#evidence-pdfs); [checks](docs/requirements.md#learn-from-it))
pick. Publish a namespace's resources, episodes, timeline activities or text as a private RSS feed or a podcast
feed that any feed reader or podcast app follows. ([Podcasts](docs/podcasts.md),
[specs from transcripts](docs/notes.md#specs-from-transcripts), [notes](docs/notes.md),
[evidence PDFs](docs/api.md#evidence-pdfs), [published feeds](docs/feeds.md);
[checks](docs/requirements.md#learn-from-it))
- **Automate it.** Pipelines and workflows on a canvas, routines on a schedule, batch runs over a collection, and
notifications to webhooks, Slack, Discord and Matterbridge. Everything that happens in Lens (new assets, emails,
calendar events, files, feed posts, what people say to Lens, sensor readings, notes, graph changes, runs, budget
Expand Down
72 changes: 72 additions & 0 deletions docs/feeds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Published feeds (RSS and podcasts)

**Status: in progress.** Feeds, their private addresses, the API and the MCP tools are built. Routine workflow nodes
that pick part of the graph or the timeline and publish it, and the Feeds page in the web app, come next.

A namespace can publish what's in it as a feed that any feed reader or podcast app follows: new resources, podcast
episodes, things on the timeline, or text a person or an agent wrote. Each feed has a private address; add it to the
app on your phone and new items arrive there.

## Two kinds

| Kind | Lists | For |
|---|---|---|
| `rss` | everything added to it: resources, episodes, timeline activities and text | feed readers |
| `podcast` | items with audio: podcast episodes once their audio is made, and resources that have audio (or video) | podcast apps (Apple Podcasts, Pocket Casts, Overcast, AntennaPod and the rest) |

Both are RSS 2.0. A podcast feed adds Apple Podcasts' tags (author, category, explicit, artwork, each episode's
duration) and each item's audio as its enclosure, which podcast apps download with byte ranges.

## What an item shows

- **A resource**: its title and summary as they are now, a link to its page in the web app, and its audio when it
has some. A resource deleted since drops out of the feed.
- **A podcast episode** ([Podcasts](podcasts.md)): listed once it's made. Its description is the outline's summary,
then the sources it cites, each with a link, so a listener can check every claim. In a podcast feed it's listed
once its audio is made; in an RSS feed a script-only episode is listed as text.
- **A timeline activity** ([Timeline](timeline.md)): its one-line summary, when it happened, its kind as the
category, and a link to what it's about.
- **Text**: a title, text and a link (http or https), such as a digest a workflow or an agent wrote.

An item is listed once per feed: adding the same resource or activity again does nothing, so a routine that runs
again adds only what's new. A feed lists its newest items (50 by default, up to 500) and keeps its newest 1000.

## Private by default

- A feed's address carries a token (`/api/v1/feeds/public/lfd_...`). The token is all a reader needs, so share the
address as you would a password. Readers of the feed's namespace see the address in the web app and the API; a new
address (**rotate**) stops the old one at once.
- What a feed shows is bounded by who can already read its namespace: an item's resource or activity is in that
namespace, or in one that everyone who reads the feed's namespace can read too, and never in a
[vault](encryption.md). A vault can't publish a feed, since its resources open only with a passkey.
- A feed asks podcast directories not to list it (`itunes:block`, `podcast:locked`) and search engines not to index
it (`X-Robots-Tag`), and its address isn't sent on to other sites (`Referrer-Policy: no-referrer`).
- Text in an item is escaped for the XML and again for readers that show a description as HTML, so a title or a
transcript can't add markup or scripts to a reader.
- Nothing is published until someone makes a feed and adds to it, and a feed changes nothing else in Lens.

Feed readers poll; the feed answers `304 Not Modified` when nothing changed (its `ETag`), and notes when a reader
last read it (`fetched_at`, at most once a minute), so the web app can show that a podcast app is subscribed.

## API and MCP

| | |
|---|---|
| `GET /api/v1/feeds` | feeds in the namespaces you can read (or `?namespace=`), each with its private address (`url`), how many items it holds and when a reader last read it |
| `POST /api/v1/feeds` | make a feed (editors of its namespace): `namespace`, `title`, `kind` (`rss` or `podcast`), and optionally `description`, `author`, `language`, `category`, `explicit`, `image` (an `https://` address) and `max_items`. Audited as `feed.create` |
| `GET /api/v1/feeds/{id}` | one feed |
| `PATCH /api/v1/feeds/{id}` | change its settings (editors) |
| `DELETE /api/v1/feeds/{id}` | delete it and its items (editors); its address stops working. What the items were about stays |
| `POST /api/v1/feeds/{id}/token` | a new private address (editors); the old one stops working |
| `GET /api/v1/feeds/{id}/items` | its items, newest first |
| `POST /api/v1/feeds/{id}/items` | add `recordings` (resources and episodes), `activities` (timeline activity ids) and `texts` (`{title, summary, link, guid}`) (editors, who can read each one). Answers what was `added`, how many were there `already`, and what was `refused`, with why. Audited as `feed.publish` |
| `DELETE /api/v1/feeds/{id}/items/{item}` | take an item out (editors) |
| `GET /api/v1/feeds/public/{token}` | the feed itself, as feed readers and podcast apps read it: no session, the token is the credential |
| `GET /api/v1/feeds/public/{token}/items/{item}/audio` | an item's audio, with byte ranges (and `HEAD`) |

Agents use the MCP tools `list_feeds` (each feed's address, or one feed with its newest items), `create_feed` and
`publish_to_feed` ([MCP](mcp.md)).

Refine later: artwork Lens makes for a feed without an `image`; Atom and JSON Feed alongside RSS; chapters and
transcripts in a podcast feed (`podcast:transcript`); an automation action that publishes to a feed when something
happens.
6 changes: 5 additions & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,9 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
| `export_chat` | one of them as Markdown or JSON ([Copy and export](assistant.md#copy-and-export)): the version showing, one `message`, every version (`all_versions`) or OpenAI's `{"messages": [...]}` shape (`as_messages`) |
| `create_podcast` | make a two-host learning [episode](podcasts.md) about assets or lines in them, with an angle, a length and a style; it's made in the background. Needs a write-scope token and editor access where it goes |
| `get_podcast` | an episode: how far it has got, its script with each line's speaker and the sources it cites (links), and what the fact-check changed |
| `list_feeds` | the RSS and podcast [feeds](feeds.md) your namespaces publish, each with its private address and how many items it holds; with `feed_id`, one feed and its newest items |
| `create_feed` | make an RSS feed or a podcast feed in a namespace, for feed readers and podcast apps. Needs a write-scope token and editor access |
| `publish_to_feed` | add resources, episodes, timeline activities and a text item to a feed; what's there already isn't added twice, and what may not show there comes back refused, with why. Needs a write-scope token and editor access |
| `write_spec` | write a technical [spec](notes.md#specs-from-transcripts) from a recording's transcript as a note linked to it: every decision, no and constraint in the speaker's own words with links to when they said them, and the gaps left to decide later; it's written in the background. Needs a write-scope token and the right to process the recording |
| `get_spec` | how a recording's spec is coming (`writing`, `ready` or `failed` with why), its note's id and link, and how many decisions, things turned down, constraints and gaps it holds and quotes the transcript doesn't |
| `nsfw_settings` | how [NSFW classification](nsfw.md) works in a namespace: on or off, the threshold, how flagged resources show, and how many are classified, flagged and not classified yet |
Expand Down Expand Up @@ -152,7 +155,8 @@ Every tool reads, except `suggest_topic`, which adds suggestions people accept o
which records an undoable change (proposed by default), `undo_graph_change`, `edit_entity`, `edit_topic`,
`merge_topics`, `delete_topic` and `undo_topic_change`, which curate entities and topics as an editor does in the web
app (merges and deletes can be undone), `write_note`, which adds a note, `create_podcast`, which
queues an episode, `write_spec`, which queues a spec, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
queues an episode, `create_feed` and `publish_to_feed`, which make and fill a [published feed](feeds.md),
`write_spec`, which queues a spec, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
editors), `set_namespace_integrations`, which turns a namespace's own MCP server or chat completions on or off (owners), `set_picture_descriptions`, which changes what describes pictures (admins), `set_file_storage`, which changes where new files are kept (admins), `subscribe_feed` and `unsubscribe_feed`, which follow and stop following news feeds (admins), and `set_monitoring`, `set_service`,
`set_sandbox_gateway` (the [code sandbox](assistant.md#code-sandbox)'s gateway) and `set_install_settings`, which have the machine change how Lens runs (admins).
Importing stays in the web app and the API.
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Templates a note starts from, and a copy of any note | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_templates_and_copies`; [notes-templates.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-templates.test.tsx); [notes-tree.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-tree.test.tsx) `makes a copy beside the note and opens it` |
| A note shows the notes it's inside and the notes inside it, to people and agents | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_path_and_children`, `test_assistant_keeps_notes`; [test_mcp.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mcp.py) `test_note_tools`; [note-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/note-page.test.tsx) |
| Undo and Recently deleted instead of a confirm | [test_notebook.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notebook.py) `test_recently_deleted`; [notes-delete.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/notes-delete.test.tsx) |
| A namespace publishes a private RSS feed or a podcast feed (Apple Podcasts' tags, audio with byte ranges) of resources, episodes, timeline activities and text, holding only what its readers can already read; through the API and MCP | [test_feeds.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_feeds.py) `test_making_filling_and_reading_a_feed`, `test_a_podcast_feed_lists_what_has_audio`, `test_timeline_activities_in_a_feed`, `test_vaults_and_settings`, `test_agents_use_feeds` |
| An asset or a collection exported as an evidence PDF, whole or the parts you pick | [test_evidence.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_evidence.py) `test_a_collection_as_one_evidence_pdf`, `test_everything_lens_found_goes_in`, `test_picking_the_parts`; [evidence-parts.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/evidence-parts.test.tsx) |

## Automate it
Expand Down
96 changes: 96 additions & 0 deletions fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -2075,6 +2075,102 @@ def get_podcast(ctx: Context, episode_id: int) -> dict[str, Any]:
)


# ---------- published feeds (docs/feeds.md) ----------
def _feed_out(ctx: Context, row: dict[str, Any]) -> dict[str, Any]:
from app.domain import syndication

f = syndication.view(ctx.db, ctx.cfg, row, ctx.web)
keep = ("id", "kind", "title", "namespace", "description", "url", "page", "items", "last_item_at", "fetched_at")
return store.clean({("feed_id" if k == "id" else k): f.get(k) for k in keep})


@tool(
"list_feeds",
"List published feeds",
"The RSS and podcast feeds the person's namespaces publish (docs/feeds.md): each one's kind, title, namespace, how "
"many items it holds, and its private address, which feed readers and podcast apps subscribe to. Anyone with the "
"address can read the feed, so give it only to the person. With feed_id, one feed and its newest items.",
Arg("namespace", "string", "only this namespace's feeds"),
Arg("feed_id", "integer", "one feed, with its newest items"),
)
def list_feeds(ctx: Context, namespace: str | None, feed_id: int | None) -> dict[str, Any]:
from app.domain import syndication

if feed_id is not None:
row = syndication.get(ctx.db, feed_id) # KeyError: not found
ctx.acl.need(row["space"])
shown = [
store.clean({k: i.get(k) for k in ("id", "kind", "title", "summary", "link", "published_at", "recording")})
| ({"url": ctx.link(i["recording"])} if i.get("recording") is not None else {})
for i in syndication.items(ctx.db, feed_id, 20)
]
return {**_feed_out(ctx, row), "newest": shown}
spaces = {ctx.acl.namespace(namespace)} if namespace else ctx.acl.readable()
return {"feeds": [_feed_out(ctx, f) for f in syndication.list_feeds(ctx.db, spaces)]}


@tool(
"create_feed",
"Make a published feed",
"Make an RSS feed (any items) or a podcast feed (items with audio, with Apple Podcasts' tags) in a namespace, for "
"feed readers and podcast apps to follow at its private address; fill it with publish_to_feed, or with a routine "
"whose workflow publishes to it. Needs a write-scope token and editor access to the namespace.",
Arg("namespace", "string", "the namespace it belongs to", required=True),
Arg("title", "string", "its title", required=True, max_length=200),
Arg("kind", "string", "rss or podcast", enum=("rss", "podcast"), default="rss"),
Arg("description", "string", "what it's about", max_length=2000),
writes=True,
)
def create_feed(ctx: Context, namespace: str, title: str, kind: str, description: str | None) -> dict[str, Any]:
from app.domain import syndication

if not ctx.user.can_write:
raise ToolError("this token is read-only: ask for a token with the write scope to make feeds")
sid = ctx.acl.namespace(namespace, "editor")
fid = syndication.create(ctx.db, ctx.cfg, sid, title, kind, by=ctx.user.email, description=description)
auth.audit(ctx.db, ctx.user.as_audit(), "feed.create", f"feed:{fid}", {"kind": kind, "via": "mcp"})
return _feed_out(ctx, syndication.get(ctx.db, fid))


@tool(
"publish_to_feed",
"Add to a published feed",
"Add resources (podcast episodes too: a podcast feed lists them once their audio is made), timeline activities "
"(by id, as timeline gives them) and a text item (title, text, link) to a feed made with create_feed. What's in "
"it already isn't added twice; what may not show there (it's in a namespace not everyone who reads the feed can "
"read) comes back in `refused`, with why. Needs a write-scope token and editor access to the feed's namespace.",
Arg("feed_id", "integer", "the feed", required=True),
Arg("recording_ids", "integer[]", "resources or episodes to list", maximum=200),
Arg("activity_ids", "string[]", "timeline activities to list"),
Arg("title", "string", "a text item's title", max_length=200),
Arg("text", "string", "a text item's text", max_length=4000),
Arg("link", "string", "a text item's link (http or https)", max_length=2000),
writes=True,
)
def publish_to_feed(
ctx: Context,
feed_id: int,
recording_ids: list[int] | None,
activity_ids: list[str] | None,
title: str | None,
text: str | None,
link: str | None,
) -> dict[str, Any]:
from app.api.v1.routes import feeds as feed_routes
from app.schemas.feeds import FeedPublish

if not ctx.user.can_write:
raise ToolError("this token is read-only: ask for a token with the write scope to add to feeds")
texts = [store.clean({"title": title, "summary": text, "link": link})] if title or text else []
if not recording_ids and not activity_ids and not texts:
raise ToolError("give recording_ids, activity_ids, or a title or text to add")
try:
body = FeedPublish(recordings=recording_ids or [], activities=activity_ids or [], texts=texts)
except ValueError as e:
raise ToolError(str(e)) from None
return feed_routes.publish(ctx.acl, ctx.user, ctx.db, ctx.cfg, feed_id, body).model_dump()


# ---------- specs from transcripts (docs/notes.md#specs-from-transcripts) ----------
@tool(
"write_spec",
Expand Down
2 changes: 2 additions & 0 deletions fastapi_backend/app/api/v1/router.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
extensions,
external,
fedora,
feeds,
fields,
files,
filetree,
Expand Down Expand Up @@ -152,6 +153,7 @@
voice,
vaults,
podcasts,
feeds,
specs,
):
api_router.include_router(module.router)
Loading
Loading