Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,12 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
[mail, calendars and files](docs/assistant.md#mail-calendars-and-files), [code sandbox](docs/assistant.md#code-sandbox), [answer records](docs/assistant.md#what-each-answer-records),
[questions about Lens](docs/assistant.md#questions-about-lens-itself), [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), [speech providers](docs/speech-providers.md);
[checks](docs/requirements.md#ask-and-get-cited-answers))
- **Answer your email.** Lens drafts a reply to an email a watched mailbox brought in, from what that email's
namespace holds: each fact cited to its excerpt, what the archive doesn't say marked for you to confirm, addressed
and threaded as a reply to it. Drafts wait for you to edit or set aside, and nothing is sent on its own; ask for one
through the API or MCP (agents list what waits and draft with your instructions), or add the reply step to a
pipeline to draft each new email, which skips mailing lists, no-reply senders and your own mail without a model
call. ([Email replies](docs/email-replies.md); [checks](docs/requirements.md#answer-your-email))
- **Work toward a goal in projects.** A project has a name, a goal in your words, instructions and the namespaces it
draws on. Its coordinator conversation looks at the project as a whole: it is told about every thread, starts threads
with a brief that the assistant answers there in the background, reads them and marks them done. Each thread is a
Expand Down Expand Up @@ -316,7 +322,7 @@ The guides are on the [docs site](https://adeelahmad.github.io/lense/) and in th
- **Agents and apps:** [MCP server](docs/mcp.md), [API](docs/api.md), [authentication](docs/authentication.md),
[access](docs/access.md)
- **Use:** [chat and batch runs](docs/assistant.md), [chat rooms](docs/chat-rooms.md),
[the assistant by email](docs/assistant-email.md), [podcasts](docs/podcasts.md), [notes](docs/notes.md),
[the assistant by email](docs/assistant-email.md), [email replies](docs/email-replies.md), [podcasts](docs/podcasts.md), [notes](docs/notes.md),
[web clipper](docs/web-clipper.md), [the assistant in your browser](docs/browser-assistant.md),
[sensors](docs/sensors.md), [processing](docs/processing.md), [video](docs/video.md),
[NSFW classification](docs/nsfw.md), [enhancing images](docs/enhance.md), [notifications](docs/notifications.md),
Expand Down
6 changes: 6 additions & 0 deletions changelog.d/email-replies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
**Email replies.** Lens drafts a reply to an email a watched mailbox brought in, from what that email's namespace
holds: each fact cited to its excerpt, what the archive doesn't say marked `[to confirm: …]`, addressed to the sender
and threaded. Drafts wait to be edited or set aside and nothing is sent; ask through the API or MCP
(`email_replies`, `draft_reply`, `get_reply`, `edit_reply`), or add the new reply step to a pipeline to draft each
new email, skipping mailing lists, no-reply senders and your own mail without a model call. The prompt is the
Reply to an email template ([Email replies](docs/email-replies.md)).
89 changes: 89 additions & 0 deletions docs/email-replies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Email replies

Lens drafts replies to the email it reads, from what your archive says. Each fact in a draft is cited to the excerpt it
came from, what the archive doesn't say is marked for you to fill in, and the draft waits for you:
Lens never sends a reply on its own.

It works on any email Lens has imported: mail that a watched [Gmail or IMAP mailbox](processing.md#gmail-and-google-workspace) brought in, or an
`.eml` file you uploaded. It needs a language model (Settings › LLM provider).

## Drafting a reply

Ask for one email's reply through the API or MCP, with what it should say if you like ("say yes, Tuesday works"):

```bash
curl -X POST "$LENS/api/v1/recordings/42/reply" -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" -d '{"instructions": "Say Tuesday works for us."}'
curl "$LENS/api/v1/recordings/42/reply" -H "Authorization: Bearer $TOKEN"
```

The draft is written in the background, as a job step (`reply`), so it counts against [budgets](budgets.md) and every
model call is in the [activity ledger](activity.md). What comes back:

- `text`: the reply, with `[n]` after each fact taken from source `n`, and `plain`: the same without the citations, as
it would be sent (the person you answer can't open your archive);
- `sources`: the excerpts the model was given, each saying whether the draft cites it (`used`);
- `gaps` and `unconfirmed`: what the reply needs that the archive doesn't say, written into the draft as
`[to confirm: …]`, and how many of those marks are still there;
- `to` (the sender, or the address their Reply-To names), `from` (the address the email came in to), `subject`
(`Re: …`), and `in_reply_to` and `references`, so the reply joins the email's thread; `others` lists the rest of
who it was for, to answer all of them.

Drafting the same email again replaces the draft and keeps the one before as `previous`.

### What a draft is drawn from

- The email as it was received: Lens reads the message again from its mailbox (or the uploaded file), without the
quote of earlier mail and the signature. An email whose file can't be read any more is drafted from its text in
Lens, and its reply won't thread.
- Excerpts of the email's own namespace that match it, found as the [assistant](assistant.md) finds them (by words,
and by meaning once that's turned on). Never the email itself, and never another namespace: a reply goes to someone
outside Lens, so it says only what that namespace holds.

### The prompt

Replies are drafted with the **Reply to an email** prompt template, on the Templates page ([pipelines and templates](processing.md#pipelines-and-templates)): change its
tone, language or rules there, and the next draft uses the new version. A request can name another prompt template
(`template`, `version`).

## Drafting every new email

Add a **reply** step to the [pipeline](processing.md) of the namespace a mailbox's watch fills, after the step that
reads the email: each new email gets a draft as it arrives. No pipeline has the step until you add it, so nothing is
drafted, and no model is called, unless you ask.

A pipeline skips mail that isn't one to answer, without a model call: mailing lists, bulk mail, no-reply senders,
auto-replies, delivery reports, and mail sent from one of your own mailboxes. Asked for one email by hand, Lens drafts
it anyway.

## Reviewing, editing and setting aside

`GET /api/v1/replies` lists the email of the namespaces you can read, newest first, each with its reply. By default
(`show=waiting`) it's the mail that may need an answer (no reply yet, and not automatic or your own) and every draft;
`show` takes one status (`draft`, `drafting`, `skipped`, `dismissed`, `sent`, `failed`), `none` (no reply yet,
automatic mail too) or `all`, and `namespace` narrows it to one.

`PATCH /api/v1/recordings/{id}/reply` changes a draft's `text`, `to`, `cc` or `subject` (the text before is kept as
`previous`), or sets it aside (`status: dismissed`) and brings it back (`status: draft`), which is how setting it aside
is undone. Drafting, editing and setting aside need the right to process the email (editors); anyone who can read the
email can read its reply.

## For agents

The MCP server has the same: `email_replies` lists what waits, `draft_reply` drafts one (with `instructions`),
`get_reply` reads it with its sources linked into the archive, and `edit_reply` changes it or sets it aside. Each result
carries `review_url`, where the person reviews the draft. Nothing an agent does sends mail.

## Safety

An email is someone else's words, and may be written to steer a model. It reaches the model marked as data
([what the archive says is data](mcp.md#what-the-archive-says-is-data)), and the system prompt says never to follow
instructions in it. A web address in the draft that neither the email nor the excerpts hold is broken up
(`hxxps://evil[.]example/…`), so an email can't have its reply carry a link of its own making, and a citation to an
excerpt the model wasn't given is taken out. The draft says only what the email's namespace holds.

## Refine later

- Send a reply through the mailbox it came in to, after your yes, and a Replies page in the web app (in progress).
- A signature in Settings; a decision model choosing which mail needs an answer; drafts from a whole thread rather than
its newest message; replies with attachments.
6 changes: 5 additions & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,10 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
| `publish_to_feed` | add resources, episodes, timeline activities and a text item to a feed; what's there already isn't added twice, and what may not show there comes back refused, with why. Needs a write-scope token and editor access |
| `write_spec` | write a technical [spec](notes.md#specs-from-transcripts) from a recording's transcript as a note linked to it: every decision, no and constraint in the speaker's own words with links to when they said them, and the gaps left to decide later; it's written in the background. Needs a write-scope token and the right to process the recording |
| `get_spec` | how a recording's spec is coming (`writing`, `ready` or `failed` with why), its note's id and link, and how many decisions, things turned down, constraints and gaps it holds and quotes the transcript doesn't |
| `email_replies` | email that waits for an answer (not a mailing list's, a no-reply sender's or the person's own) and every draft, newest first, or one [reply](email-replies.md) status |
| `draft_reply` | draft a [reply](email-replies.md) to an email from what its namespace holds, each fact cited to its excerpt and what the archive doesn't say marked to confirm; `instructions` says what it should say. It's drafted in the background and nothing is sent. Needs a write-scope token and the right to process the email |
| `get_reply` | an email's reply: its status, who it goes to, the draft with and without its citations, its sources and what's left to confirm |
| `edit_reply` | change a draft reply's text, recipients or subject, or set it aside and bring it back. Needs a write-scope token and the right to process the email |
| `nsfw_settings` | how [NSFW classification](nsfw.md) works in a namespace: on or off, the threshold, how flagged resources show, and how many are classified, flagged and not classified yet |
| `set_nsfw` | turn NSFW classification on or off, set its threshold and display, for a namespace (owners) or the archive (admins), and optionally classify what's there. Needs a write-scope token |
| `mark_nsfw` | mark a resource NSFW or not by hand, or hand it back to the model's score. Needs a write-scope token and editor access |
Expand Down Expand Up @@ -175,7 +179,7 @@ which records an undoable change (proposed by default), `undo_graph_change`, `ed
`merge_topics`, `delete_topic` and `undo_topic_change`, which curate entities and topics as an editor does in the web
app (merges and deletes can be undone), `write_note`, which adds a note, `create_podcast`, which
queues an episode, `create_feed` and `publish_to_feed`, which make and fill a [published feed](feeds.md),
`write_spec`, which queues a spec, `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
`write_spec`, which queues a spec, `draft_reply` and `edit_reply`, which draft and change [email replies](email-replies.md) (nothing is sent), `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
editors), `enhance`, which enhances an image and reads it again (the process permission),
`set_namespace_integrations`, which turns a namespace's own MCP server or chat completions on or off (owners), `set_picture_descriptions`, which changes what describes pictures (admins), `set_file_storage`, which changes where new files are kept (admins), `set_secret_reference`, which points a secret field at a password manager's item (admins), `add_mcp_server`, which adds an
MCP server (admins), `subscribe_feed` and `unsubscribe_feed`, which follow and stop following news feeds (admins), and `set_monitoring`, `set_service`,
Expand Down
9 changes: 9 additions & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,15 @@ request, to another check that proves the same thing; loosening a requirement or
| Other chat rooms through Matterbridge | [test_bridge.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_bridge.py) `test_it_answers_what_is_said_to_it_and_nothing_else`; [test_matterbridge_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_matterbridge_run.py) `test_chat_networks_are_set_in_the_app_and_become_matterbridges_config`, `test_the_bridge_talks_to_the_matterbridge_lens_runs`; [matterbridge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/matterbridge.yml) `Lens writes a config, and Matterbridge starts with it` |
| By email, at an address of its own | [test_mailbox.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_mailbox.py) `test_mail_is_answered_by_email_in_its_thread`, `test_the_mailbox_is_set_up_and_checked_in_the_app`, `test_answers_list_the_excerpts_they_cite` |

## Answer your email

| Requirement | Checks |
|---|---|
| A reply drafted to an imported email from its own namespace, each fact cited, what the archive doesn't say marked to confirm, a citation to an excerpt it wasn't given taken out and a link of the model's making broken up, addressed and threaded as a reply; nothing sent | [test_replies.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_replies.py) `test_a_cited_threaded_draft` |
| Drafts edited, set aside and brought back, the text before kept; the mail that waits for an answer listed, not mailing lists' | [test_replies.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_replies.py) `test_editing_setting_aside_and_listing` |
| A pipeline's reply step drafts each new email, skipping mailing lists, no-reply senders and the person's own mail without a model call | [test_replies.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_replies.py) `test_a_pipeline_drafts_only_mail_to_answer` |
| Agents list what waits, draft with instructions, read and edit replies through MCP | [test_replies.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_replies.py) `test_agents_draft_read_and_edit_replies` |

## Work toward a goal in projects

| Requirement | Checks |
Expand Down
Loading
Loading