Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
each run shows what its actions did. Try one on an activity without keeping or sending anything, or run it now;
Settings → Automations switches them all off and sets how far they may go
([Running automations](docs/automations.md#running-automations)).
- **Each answer says what produced it.** The *i* in an answer's header shows the model and the host it ran on, the
tools it was offered and used, the scope, its limits and what it used, tokens, cost (an estimate from your prices),
how long it took, how it ended and which tool results read like instructions to an AI, with the calls made for it.
Agents read the same record over the API (`GET /api/v1/chats/<id>/messages/<mid>`), and each tool step shows how long
it took. Answers from before have no record ([What each answer records](docs/assistant.md#what-each-answer-records)).
- **Automations for agents.** Agents list, save, try and run automations through the MCP tools `automations`,
`automation_actions`, `save_automation`, `test_automation` and `run_automation`, with problems in plain words. The
assistant saves them switched off and asks before switching one on or running it
Expand Down
8 changes: 6 additions & 2 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,7 +477,8 @@ read there:
What it cites comes back three ways, so any app can show it: `[n]` in the text with a **Sources** list of links after
it (left out when the request has a `response_format`), `citations` (the link for `[n]` is the n-th), and
`lens.sources` (`n`, `recording_id`, `title`, `time`, `speaker`, `url`, `cited`). `lens.steps` lists the tools the
model used. With `"stream": true` the answer comes as OpenAI's `chat.completion.chunk` events, ending with
model used, and `lens.record` what produced the answer, as a chat answer's record
([What each answer records](assistant.md#what-each-answer-records)) without tokens and cost. With `"stream": true` the answer comes as OpenAI's `chat.completion.chunk` events, ending with
`data: [DONE]`; the last chunk before it carries `finish_reason: "stop"`, `citations` and `lens`. While the tools run,
the stream sends SSE comments, so nothing in between closes it.

Expand Down Expand Up @@ -1448,6 +1449,7 @@ GET /api/v1/chats/{cid}
PATCH /api/v1/chats/{cid}
DELETE /api/v1/chats/{cid}
POST /api/v1/chats/{cid}/messages
GET /api/v1/chats/{cid}/messages/{mid}
POST /api/v1/chats/{cid}/stop
POST /api/v1/chats/{cid}/messages/{mid}/check
GET /api/v1/approvals
Expand All @@ -1469,7 +1471,9 @@ whether it uses tools (`tools`, `max_steps`), whether answers can be checked (`c
(the configured one first; `llm.chat_models`, else the server's list); never the model server's address or key. A
conversation's `model` (`POST /chats`, `PATCH /chats/{cid}`; null for the configured one) answers in it, and a
question can name another (`POST /chats/{cid}/messages {content, model}`, e.g. to retry); a model that isn't offered is
a 400. Each answer records the `model` that wrote it. `POST /chats/{cid}/stop` stops the answer being written in your conversation after the piece or tool
a 400. Each answer records the `model` that wrote it, and its `meta` what produced it; `GET
/chats/{cid}/messages/{mid}` gives one message with its record and its rows in the activity ledger
([What each answer records](assistant.md#what-each-answer-records)). `POST /chats/{cid}/stop` stops the answer being written in your conversation after the piece or tool
step it's on: the stream sends `stopped`, then `done` with the saved message, whose `stopped` is true and whose
`content` is what came before (`(stopped)` when nothing had). `{stopping: false}` when nothing was being written.
`POST /chats/{cid}/messages {content, edit}` edits one of your earlier questions (`edit` is its message id): it and
Expand Down
36 changes: 34 additions & 2 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,12 @@ then.
numbered.
- **Answers:** the model answers only from those excerpts and cites them as [n]. Each citation carries the asset
and timestamp.
- **Streaming:** answers arrive over server-sent events: `passages`, then `token`s, then `done`.
- **Streaming:** answers arrive over server-sent events: `passages`, then `token`s, then `meta` (the answer's
record, [below](#what-each-answer-records)) and `done`.
- **Stopping:** Stop asks the server to end the answer after the piece or tool step it's on (whichever server process
is writing it); what came before is saved, marked stopped. A model call already under way finishes first.
- **Reopening:** a conversation's answers keep the tools the assistant used (and with what), any notice (the model
couldn't use tools), the error when there was no answer, and their latest source check.
couldn't use tools), the error when there was no answer, their latest source check and what produced them.
- **Versions:** a conversation is a tree, and nothing in it is deleted. Regenerating an answer writes a new answer to
the same question beside it, with the same model or another (`POST /api/v1/chats/<id>/messages` with `regenerate`:
the answer's id, and `model`); editing a question makes a new version of it beside the old one, which keeps what
Expand Down Expand Up @@ -63,6 +64,37 @@ then.
- **No model configured:** chat returns the best-matching passages instead. Anyone signed in can see whether a model
is set up, and which (`GET /chats/capabilities`), so the app says so before the first question.

### What each answer records

Every answer keeps a record of what produced it (`meta` on the message), so a person can trust and debug it and an
agent can read it. The **i** in an answer's header (Details) shows it, with the calls made for it and what the
conversation has cost this month; on a phone it opens as a sheet.

| Field | What it says |
| --- | --- |
| `model`, `provider` | the model that wrote it, and the host name of the model server it ran on (never the server's address, path or key) |
| `tools` | `offered`: every tool the model was offered; `picked`: the tools the question asked for (null: all it may use) |
| `scope` | the scope it answered in, after a conversation over everything was narrowed to a namespace |
| `limits`, `used` | the rounds of tool calls it had (and the tool calls, when they're limited), and how many it used |
| `outcome` | `answered` (after looking things up), `direct` (without looking anything up), `limit` (its rounds ran out, so it answered from what it found), `fallback` (from a search: the model can't use tools, or none is set up), `stopped` or `error` |
| `tokens`, `cost_usd`, `cost_estimated` | the tokens of the model calls made for it, and what they cost from the prices in Settings ([Costs](activity.md#costs)); `cost_estimated` when some had a price but no token counts, so it cost at least this |
| `ms` | how long it took, from the question |
| `flagged` | tools whose results read like instructions to an AI ([below](#third-party-content-and-prompt-injection)) |
| `version` | the version of Lens that wrote it |

Each tool step also keeps how long it took (`ms`) and whether it failed (`failed`). A question keeps how it was sent
(`via`: `web`, `api`, `voice`, `room` or `automation`) and the options it was asked with (`options`, such as another
`model`).

An answer is written inside the activity ledger's scope `chat_message:<id>` (with `chat:<id>`), so every model and tool
call made for it names it ([Activity](activity.md)), and its record's tokens and cost are theirs: one ledger total per
answer, and no model call. `GET /api/v1/chats/<id>` returns the record on each message, and `GET
/api/v1/chats/<id>/messages/<mid>` returns one message, of any version, with its record and those ledger rows
(`ledger`). Answers in chat rooms ([Chat rooms](chat-rooms.md)) and the automations' agent record theirs the same way.
An app's chat completions keep no messages, so their record comes back in the reply's `lens.record`, without tokens and
cost ([Chat completions](api.md#chat-completions-for-any-app)). Answers from before Lens kept records have none; Details
shows what they have and says the rest wasn't recorded.

### Picking the namespace for a conversation over everything

When a conversation has no scope (the assistant home starts it over everything), its first question goes to the
Expand Down
5 changes: 4 additions & 1 deletion docs/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,10 @@ optional, and older resources simply don't have them.
## Upgrades

Installs keep their data across updates (running `install.sh` again, a new image, `git pull`). New tables, fields and
indexes arrive through `SCHEMA` on the next start. Anything else, such as rewriting stored values, moving data between
indexes arrive through `SCHEMA` on the next start. A field that is only added needs no step when its readers treat it
as missing on older rows: a chat message's record (`meta`,
[What each answer records](assistant.md#what-each-answer-records)) is one, and messages from before it simply have
none. Anything else, such as rewriting stored values, moving data between
tables, or changing or removing an index that already exists, is a named step in `app/domain/migrations.py`:

* Every process that opens the database (the API, workers, `lens watch`, `lens` commands) runs the steps the database
Expand Down
4 changes: 4 additions & 0 deletions fastapi_backend/app/api/v1/routes/activity.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ def _may_see(db: Db, user: CurrentUser, acl: Acl, resource: str) -> None:
return
if table == "chat" and db.one("SELECT id FROM $r WHERE account = $a", r=activity.R("chat", int(key)), a=user.id):
return
if table == "chat_message": # a message in one of your conversations
m = db.one("SELECT chat FROM $r", r=activity.R("chat_message", int(key)))
if m and db.one("SELECT id FROM $r WHERE account = $a", r=activity.R("chat", m["chat"]), a=user.id):
return
raise HTTPException(403, "only admins see this resource's activity")


Expand Down
Loading
Loading