Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
Markdown or plain text and exports it, questions and code blocks have their own copy buttons, and the conversation's
header has Export; without a PDF printer on the server, the browser prints it instead
([Copy and export](docs/assistant.md#copy-and-export)).
- **News feeds.** Subscribe to a blog's or a news site's RSS or Atom feed: add it on the Sources page (its address,
or just the site's) and watch it, and each new post comes into a namespace as text, with its link, author and date,
so answers cite it. Feeds are asked for again only when they've changed, and a private feed's address is kept like a
password ([News feeds](docs/processing.md#news-feeds-rss-and-atom)).
- **Automations on feed posts.** Each new post of a news feed is a `feed_post.received` activity with its address,
site, feed, author, date and categories, so an automation can start on posts from one feed, author or category
([Kinds](docs/automations.md#kinds)).

## 0.14.0 <small>October 17, 2026</small> {id="0.14.0"}

Expand Down Expand Up @@ -72,10 +79,6 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
each run shows what its actions did. Try one on an activity without keeping or sending anything, or run it now;
Settings → Automations switches them all off and sets how far they may go
([Running automations](docs/automations.md#running-automations)).
- **News feeds.** Subscribe to a blog's or a news site's RSS or Atom feed: add it on the Sources page (its address,
or just the site's) and watch it, and each new post comes into a namespace as text, with its link, author and date,
so answers cite it. Feeds are asked for again only when they've changed, and a private feed's address is kept like a
password ([News feeds](docs/processing.md#news-feeds-rss-and-atom)).
- **Each answer says what produced it.** The *i* in an answer's header shows the model and the host it ran on, the
tools it was offered and used, the scope, its limits and what it used, tokens, cost (an estimate from your prices),
how long it took, how it ended and which tool results read like instructions to an AI, with the calls made for it.
Expand Down
26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,19 +143,19 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
[notes](docs/notes.md), [evidence PDFs](docs/api.md#evidence-pdfs); [checks](docs/requirements.md#learn-from-it))
- **Automate it.** Pipelines and workflows on a canvas, routines on a schedule, batch runs over a collection, and
notifications to webhooks, Slack, Discord and Matterbridge. Everything that happens in Lens (new assets, emails,
calendar events, files, what people say to Lens, sensor readings, notes, graph changes, runs, budget warnings) is kept
as a stream of activities in W3C Activity Streams form, which agents read through the API, MCP and the assistant, and
people review on the Timeline page (and Latest on Home), marking what they've seen. Automations act on that stream, on
a schedule, at a time or when their own webhook is called: they test conditions, then tell you in your own chat app,
ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a workflow, pipeline or
routine, or change assets and notes. A Classify node in a workflow sorts an asset into labels, content types or
topics, acting only on a sure answer. Anything that reaches outside Lens or changes more than what set it off asks
first, until you let it run on its own: on the Approvals page or, for a change inside Lens, with a yes in the chat it
told you in; agents can decline an approval but never give one. Each automation has limits and budgets, and agents
make, try and run them over MCP and in the assistant, or draft one from a sentence; the studio to draw them in the web
app is planned. ([Processing](docs/processing.md#workflows-and-the-canvas), [notifications](docs/notifications.md),
[automations](docs/automations.md), [approvals](docs/automations.md#approvals), [timeline](docs/timeline.md);
[checks](docs/requirements.md#automate-it))
calendar events, files, feed posts, what people say to Lens, sensor readings, notes, graph changes, runs, budget
warnings) is kept as a stream of activities in W3C Activity Streams form, which agents read through the API, MCP and
the assistant, and people review on the Timeline page (and Latest on Home), marking what they've seen. Automations act
on that stream, on a schedule, at a time or when their own webhook is called: they test conditions, then tell you in
your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a
workflow, pipeline or routine, or change assets and notes. A Classify node in a workflow sorts an asset into labels,
content types or topics, acting only on a sure answer. Anything that reaches outside Lens or changes more than what
set it off asks first, until you let it run on its own: on the Approvals page or, for a change inside Lens, with a yes
in the chat it told you in; agents can decline an approval but never give one. Each automation has limits and budgets,
and agents make, try and run them over MCP and in the assistant, or draft one from a sentence; the studio to draw them
in the web app is planned. ([Processing](docs/processing.md#workflows-and-the-canvas),
[notifications](docs/notifications.md), [automations](docs/automations.md),
[approvals](docs/automations.md#approvals), [timeline](docs/timeline.md); [checks](docs/requirements.md#automate-it))
- **See what it costs.** An activity ledger of every model call, with reports, charts and budgets that cap spending.
Setup asks for one overall budget after a cloud model, with a soft limit that warns and a hard limit that stops paid
calls. A meter in the top bar shows spend against the budget, and the Costs page breaks it down by day, model and
Expand Down
1 change: 1 addition & 0 deletions docs/automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ and the paths holding outside text are in `GET /api/v1/activities/kinds` (`event
| `asset.added` | | `recordings` | `{type: Audio, Video, Image or Document, id: lens:recording/<id>, name: its title, url: /resources/<id>, media: audio, video, image, document or transcript, content_type}` | Lens added “Scan 12” |
| `email.received` | `asset.added` | `recordings` | `from`, `to`, `cc` (addresses), `subject`, `list`, `mailbox`, `date` (sent), `snippet` (the start of its text), `attachments` (how many) | Work mail received “School trip on Friday” from office@stmarys.org |
| `calendar_event.added` | `asset.added` | `recordings` | `startTime`, `endTime`, `location`, `organizer`, `attendees` (addresses), `calendar` | Family calendar added the event “Parents' evening” on 2026-10-20 |
| `feed_post.received` | `asset.added` | `recordings` | `href` (the post's address), `site`, `feed` (its title), `author`, `published`, `categories` | Harbour news received the post “Autumn regatta” from harbour.example.org |
| `file.added` | `asset.added` | `recordings` | `path` (in its source), `mediaType`, `size` (bytes), `duration` (seconds) | Office share added “Minutes” |
| `upload.added` | `asset.added` | `recordings` | `path` (the file's name), `mediaType`, `size`, `duration` | Sam uploaded “Call with the school” |
| `web_page.saved` | `asset.added` | `recordings` | `href` (the page's address), `site` | Sam saved “Museum opening hours” from museum.example.org |
Expand Down
3 changes: 3 additions & 0 deletions docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,9 @@ it and its link.
(never its address). It comes in as text: the title, its link, who wrote it, when, the feed, its categories and
enclosures, then its text. Its content type is Feed post, so a namespace can give feed posts a pipeline of their own
(Settings › Content types).
- **Automations**: each new post is a `feed_post.received` activity (a kind of `asset.added`) with the post's
address, site, feed, author, date and categories, so an automation can start on posts from one feed, author or
category ([Automations](automations.md#kinds)). What the feed wrote is marked as outside text.
- **New posts only, unless you backfill**: as for any watch, a new watch on a feed takes the posts published after it
starts; turn on backfill to bring in those the feed lists already. The newest 500 posts of a feed are listed.
- **Light on the network**: a feed read is reused for a minute, then asked for again with its `ETag` and
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Routines on a schedule | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_routines_are_checked_and_run_when_due`, `test_schedules`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `routine-create`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `routine-create` ≤ 2 clicks |
| Batch runs over a collection | [test_batches.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_batches.py) `test_collections_and_batches` |
| Notifications to webhooks, Slack, Discord and Matterbridge | [test_notifications.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notifications.py) `test_owners_manage_targets_and_a_test_is_signed`, `test_runs_that_end_are_sent_once_to_matterbridge` |
| Everything that happens is kept as W3C Activity Streams activities, read by agents through the API, MCP and the assistant | [test_events.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_events.py) `test_activities_have_one_shape`; [test_activities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_activities.py) `test_the_timeline_lists_new_assets_newest_first`, `test_agents_read_the_timeline_too` |
| Everything that happens is kept as W3C Activity Streams activities, read by agents through the API, MCP and the assistant | [test_events.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_events.py) `test_activities_have_one_shape`; [test_activities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_activities.py) `test_the_timeline_lists_new_assets_newest_first`, `test_agents_read_the_timeline_too`; [test_news_feed_source.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_news_feed_source.py) `test_a_news_feed_as_a_source` |
| The Timeline page and Latest on Home, marking what you've seen | [test_timeline_marks.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_timeline_marks.py) `test_marks_are_set_cleared_and_filter_the_timeline`; [timeline-ui.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/timeline-ui.test.tsx) `Home's Latest card lists what you haven't reviewed` |
| Automations act on the stream, on a schedule, at a time or when their own webhook is called | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_each_new_activity_runs_each_automation_it_matches_once`, `test_a_schedule_runs_its_automation_when_its_timer_fires`, `test_a_call_to_a_webhook_runs_its_automation`; [test_timer_triggers.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_timer_triggers.py) `test_an_at_trigger_fires_once_then_reads_done` |
| They test conditions, then tell you, ask the model or the agent, use a tool, call a web address with sealed credentials, run Lens work, or change assets and notes | [test_automation_runs.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_automation_runs.py) `test_the_if_limits_and_dry_runs`; [test_actions_messages.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_messages.py) `test_notify_tells_me_in_my_chat_app`; [test_actions_ai.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_ai.py) `test_ask_model_answers_in_text_or_as_the_fields_a_schema_names`, `test_ask_agent_works_on_what_happened_and_saves_the_conversation`, `test_call_tool_reads_as_the_person_and_changes_only_by_asking_them`; [test_actions_web.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_web.py) `test_credentials_in_headers_are_sealed_to_the_automation_and_the_host_and_never_shown`; [test_actions_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_run.py) `test_run_workflow_queues_it_on_the_asset_as_the_automation`, `test_run_pipeline_processes_the_asset_again`; [test_actions_lens.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_actions_lens.py) `test_set_field_is_kept_in_the_assets_history_by_the_automation`, `test_write_note_writes_a_page_adds_to_a_things_page_or_notes_an_asset` |
Expand Down
61 changes: 56 additions & 5 deletions fastapi_backend/app/domain/events_content.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
nothing is stored twice.

Each new asset is one activity, `create-recording-<id>`, the same every time it's read, of the most specific kind its
row tells: an email received from a mailbox (`email.received`), a calendar event (`calendar_event.added`), a file
from storage or Drive (`file.added`), an upload (`upload.added`), a page or video saved from the web
row tells: an email received from a mailbox (`email.received`), a calendar event (`calendar_event.added`), a news
feed's post (`feed_post.received`), a file from storage or Drive (`file.added`), an upload (`upload.added`), a page or video saved from the web
(`web_page.saved`), something imported (pasted text, a transcript, an import hook's: `import.received`), else
`asset.added`, their parent. The kind is read from the row's content type, where it came from and who brought it,
never from its file.
Expand All @@ -14,8 +14,8 @@
scan). Labels like `watch:<id>`, a person's email or `routine:<id>` are looked up a page at a time, a few queries for
all of them, and their names kept for NAME_SECONDS (`actors`), so other sources name their actors the same way.

Mail addressing and an event's times are stored on the recording when it's made (`mail`, `event`: sources.py) and
read from there; an email's own details (`email`: subject, who, a snippet, its attachments) once it's been read.
Mail addressing, an event's times and a feed post's address and author are stored on the recording when it's made
(`mail`, `event`, `post`: sources.py) and read from there; an email's own details (`email`: subject, who, a snippet, its attachments) once it's been read.
Older rows without them are read without those fields.
"""

Expand All @@ -36,7 +36,7 @@
R = store.R
FIELDS = (
"id AS rid, record::id(id) AS id, space, title, source, media, remote, content_type, created_at, path, web, engine, "
"created_by, mail, event, email, size, duration_ms, attached_to, sensor, form"
"created_by, mail, event, post, email, size, duration_ms, attached_to, sensor, form"
)
TYPES = {"audio": "Audio", "video": "Video", "image": "Image", "document": "Document", "transcript": "Document"}
AUTOMATION_RX = re.compile(r"^automation:(\d+)$")
Expand All @@ -46,6 +46,7 @@
UPLOAD_RX = re.compile(r"/uploads/[^/]+/[0-9a-f]{24}/[^/]+$") # uploads.py keeps each upload in <ns>/<upload id>/<name>
MAIL_SOURCES = {"imap", "gmail"}
CALENDAR_SOURCES = {"ical", "gcal"}
FEED_SOURCES = {"rss"}
NAME_SECONDS = 30 # how long a looked-up name is used before it's read again
APPS = {
"assistant": {"type": "Application", "id": "lens:assistant", "name": "Assistant"},
Expand Down Expand Up @@ -287,6 +288,23 @@ def _event(r):
)


def _feed_post(r):
"""A feed post's fields: its address, site, feed, author, date and categories, as its feed listed them (`post`)."""
p = r.get("post") if isinstance(r.get("post"), dict) else {}
href = str(p.get("href") or "")[:2000]
site = urllib.parse.urlsplit(href or str(p.get("site") or "")).hostname
return store.clean(
{
"href": href or None,
"site": site or None,
"feed": _text(p.get("feed"), 300),
"author": _text(p.get("author"), 200),
"published": p.get("published"),
"categories": [str(c)[:100] for c in p.get("categories") or []][:10] or None,
}
)


def _file(r, kind):
"""A file's fields: its name in its source (or the name it was uploaded with), media type, size and length."""
remote = r.get("remote") or {}
Expand Down Expand Up @@ -323,6 +341,8 @@ def _kind(r, ctype, source_type, actor):
return "email.received"
if ctype == "calendar_event" or source_type in CALENDAR_SOURCES:
return "calendar_event.added"
if ctype == "feed_post" or source_type in FEED_SOURCES:
return "feed_post.received"
if actor["id"].startswith("lens:import_hook"):
return "import.received"
if (r.get("web") or {}).get("url"):
Expand All @@ -344,6 +364,8 @@ def _summary(kind, actor, obj):
if kind == "calendar_event.added":
start = str(obj.get("startTime") or "")
return f"{who} added the event “{title}”" + (f" on {start[:10]}" if start else "")
if kind == "feed_post.received":
return f"{who} received the post “{title}”" + (f" from {obj['site']}" if obj.get("site") else "")
if kind == "upload.added":
return f"{who} uploaded “{title}”"
if kind == "web_page.saved":
Expand Down Expand Up @@ -415,6 +437,8 @@ def _activities(db, rows):
obj.update(_mail(r))
elif kind == "calendar_event.added":
obj.update(_event(r))
elif kind == "feed_post.received":
obj.update(_feed_post(r))
elif kind in ("file.added", "upload.added"):
obj.update(_file(r, kind))
elif kind == "web_page.saved":
Expand Down Expand Up @@ -614,6 +638,33 @@ def _child(name, label, props, untrusted, sample_actor, sample_obj, summary):
},
"Family calendar added the event “Parents' evening” on 2026-10-20",
)
_child(
"feed_post.received",
"Feed post received",
{
"href": _s("Post address"),
"site": _s("Site"),
"feed": _s("Feed"),
"author": _s("Author"),
"published": _s("Published", format="date-time"),
"categories": {"type": "array", "title": "Categories", "items": {"type": "string", "title": "Category"}},
},
("object.name", "object.href", "object.site", "object.feed", "object.author", "object.categories"),
{"type": "Service", "id": "lens:source/15", "name": "Harbour news"},
{
"type": "Document",
"name": "Autumn regatta",
"media": "transcript",
"content_type": "feed_post",
"href": "https://harbour.example.org/posts/regatta",
"site": "harbour.example.org",
"feed": "Harbour news",
"author": "Mara Keane",
"published": "2026-10-15T08:00:00+00:00",
"categories": ["Harbour"],
},
"Harbour news received the post “Autumn regatta” from harbour.example.org",
)
_child(
"file.added",
"File added",
Expand Down
16 changes: 16 additions & 0 deletions fastapi_backend/app/domain/rss.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,11 +228,27 @@ def parse(raw, base, ctype=None):
"title": p["title"] or None,
"when": p["published"] or p["updated"],
"settled": True, # a post is whole once its feed lists it: no waiting for it to settle
"post": _listed(p, title, site),
}
posts[rel] = (entry, body)
return {"title": title, "site": site, "posts": posts}


def _listed(p, feed_title, site):
"""What a listing says of a post beside its file, kept on its resource as `post` (sources._post_of) for automations
and the timeline: its address, the feed and site it came from, who wrote it, when, and its categories."""
who = next((a["name"] or a["email"] for a in p["authors"] if a["name"] or a["email"]), None)
out = {
"href": p["page"],
"feed": feed_title,
"site": site,
"author": who,
"published": p["published"] or p["updated"],
"categories": p["categories"][:10],
}
return {k: v for k, v in out.items() if v}


def _iso(t):
try:
return dt.datetime.fromtimestamp(calendar.timegm(t), dt.UTC).isoformat(timespec="seconds") if t else None
Expand Down
Loading
Loading