Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,13 @@ then.
the chat's model does. A long stretch folded at once (`/compact`, or a conversation from before summaries) is
written in as many calls as it takes to fit that model's context, each in the [activity ledger](activity.md) as the
conversation's. Settings › AI assistant › Summarise long conversations (`ai.compact`, on) turns it off; off, or with
no model, only the last six messages are read.
no model, only the last six messages are read. When the model server still says a request is too long for the
model (its context was smaller than Lens knew), the answer is tried again rather than failed: what tools returned
in that answer is shortened first (each result keeps its start and end), then all but the last two messages are
folded into the summary now and half of the excerpts are left out; the context the server named is kept as the
model's, so the next conversation is summarised before it fills. In the web app, a line under the last message a
summary covers says how many messages it stands for (or that `/clear` stopped the model reading them), and Show
summary opens the summary; `GET /api/v1/chats/<id>` gives it as `summary` (`text`, `upto`, `covers`).
- **Titles:**{#titles} a conversation nobody named is titled with its first question's words as soon as it's asked,
then, after the first answer, the model gives it a short title of a few words from the question and the answer
(streamed as `title` at the end of the answer's stream). It's one small call, written by **Model for summaries**
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Answers read aloud by a speech provider, macOS `say` or StyleTTS2 (opt-in) | [test_voice.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_voice.py) `test_answers_are_read_aloud_by_a_speech_model`; [test_speech.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_speech.py) `test_macos_say_reads_aloud_on_a_mac`, `test_styletts2_reads_aloud_from_its_own_service`; [styletts2.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/styletts2.yml) `Read a sentence aloud` |
| Any message read aloud in one click, one at a time, its audio kept so a replay costs nothing | [chat-read-aloud.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-read-aloud.test.tsx) `plays an answer by the speech model in one click, stops on the next, and keeps its audio`, `reads one message at a time: starting another stops the first`, `reads a long answer piece by piece, asking for the next while one plays` |
| Conversations title themselves after their first answer, and a name you give is kept | [test_chat_titles.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chat_titles.py) `test_a_first_answer_titles_the_conversation`, `test_a_title_you_give_is_kept`, `test_agents_rename_and_retitle_over_mcp` |
| Long conversations summarised before they fill the model's context, and `/clear` and `/compact` | [test_compaction.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_compaction.py) `test_older_messages_are_folded_into_a_summary`, `test_the_context_window_is_the_model_servers`, `test_a_long_conversation_is_folded_in_runs_that_fit_the_summary_model`, `test_clear_starts_afresh_and_compact_folds_everything_now`, `test_commands_in_the_app` |
| Long conversations summarised before they fill the model's context, and `/clear` and `/compact` | [test_compaction.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_compaction.py) `test_older_messages_are_folded_into_a_summary`, `test_the_context_window_is_the_model_servers`, `test_a_long_conversation_is_folded_in_runs_that_fit_the_summary_model`, `test_a_conversation_that_doesnt_fit_is_folded_in_and_asked_again`, `test_what_tools_returned_is_cut_to_fit_the_model`, `test_clear_starts_afresh_and_compact_folds_everything_now`, `test_commands_in_the_app`; [chat-summary-mark.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-summary-mark.test.tsx) `says how many messages the summary stands for, and shows it on request` |
| A namespace's own assistant, with its own name, instructions and memory | [test_ns_assistant.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_ns_assistant.py) `test_off_by_default_and_owners_turn_it_on`, `test_remembers_across_conversations_with_citations`; [ns-assistant.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/ns-assistant.test.tsx) `saving a name and forgetting a memory` |
| Telegram and Slack built in, with voice notes and files | [test_telegram.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_telegram.py) `test_it_answers_private_chats_and_what_is_said_to_it_in_groups`, `test_voice_notes_are_heard`, `test_files_are_held_for_the_assistant_to_import`; [test_slack.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_slack.py) `test_it_answers_direct_messages_mentions_and_the_slash_command`, `test_voice_clips_are_heard_and_files_held` |
| Other chat rooms through Matterbridge | [test_bridge.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_bridge.py) `test_it_answers_what_is_said_to_it_and_nothing_else`; [test_matterbridge_run.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_matterbridge_run.py) `test_chat_networks_are_set_in_the_app_and_become_matterbridges_config`, `test_the_bridge_talks_to_the_matterbridge_lens_runs`; [matterbridge.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/matterbridge.yml) `Lens writes a config, and Matterbridge starts with it` |
Expand Down
21 changes: 18 additions & 3 deletions fastapi_backend/app/api/v1/routes/chats.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,8 @@ def _chat_view(db: DB, acl: Access, c: dict[str, Any]) -> Chat:
for m in msgs:
m["versions"] = versions.get(m["id"])
_readable_citations(db, acl, msgs)
return Chat.model_validate(sign_urls({**c, "messages": msgs, "tree": nodes}, full=True))
view = {**c, "messages": msgs, "tree": nodes, "summary": chat.covered(db, c["id"], msgs)}
return Chat.model_validate(sign_urls(view, full=True))


def _readable_citations(db: DB, acl: Access, msgs: list[dict[str, Any]]) -> None:
Expand Down Expand Up @@ -681,6 +682,10 @@ def stopped(text: str, cited: list[dict[str, Any]]) -> Iterator[str]:
yield _ev("stopped", {"message": "Stopped"})
yield from save(text or "(stopped)", cited, "stopped", stopped=True)

def shrink() -> tuple[list[dict[str, Any]], dict[str, Any] | None]:
"""The model said the conversation doesn't fit its context: fold it into the summary now (chat.shrink)."""
return chat.shrink(db, cfg, cid, past)

def answer(on: chat.Answering) -> Iterator[str]:
nonlocal notice
if llm.configured(cfg) and cfg["ai"].get("tools") and tools != []: # given no tools, it answers from a search
Expand Down Expand Up @@ -708,7 +713,17 @@ def answer(on: chat.Answering) -> Iterator[str]:
# setting the server up, or working in the person's browser, takes more steps than looking things up
steps_max = max(cfg["ai"].get("max_steps") or 6, 12 if setup else 0, browser.steps(cfg) if box.browser else 0)
answering = chat.tool_answer(
cfg, box, asked, past, steps_max, model, setup=setup, summary=summary, record=record, max_tool_calls=limit
cfg,
box,
asked,
past,
steps_max,
model,
setup=setup,
summary=summary,
record=record,
max_tool_calls=limit,
shrink=shrink,
)
for kind, data in answering:
if kind == "step":
Expand Down Expand Up @@ -745,7 +760,7 @@ def answer(on: chat.Answering) -> Iterator[str]:
text, error = "", None
try:
if llm.configured(cfg):
pieces = llm.stream_chat(cfg, chat.messages_for(asked, passages, past, summary, cfg), model=model)
pieces = chat.stream_fitted(cfg, asked, passages, past, summary, model=model, shrink=shrink)
for piece in pieces:
text += piece
yield _ev("token", {"text": piece})
Expand Down
9 changes: 7 additions & 2 deletions fastapi_backend/app/domain/bridge.py
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,10 @@ def _answer(db, cfg, base, account, cid, q, files, act_on, outside):
steps, approvals, notice = [], [], None
wrote = cfg["llm"].get("model") if llm.configured(cfg) else None

def shrink():
"""The model said the conversation doesn't fit its context: fold it into the summary now (chat.shrink)."""
return chat.shrink(db, cfg, cid, past)

def said(words, cited, outcome, **kw):
"""The answer, saved with its record."""
meta = record.meta(db, outcome, spent)
Expand All @@ -397,7 +401,8 @@ def said(words, cited, outcome, **kw):
)
try:
text, how = None, "answered"
for kind, data in chat.tool_answer(cfg, box, asked, past, cfg["ai"].get("max_steps") or 6, summary=summary, record=record):
steps_max = cfg["ai"].get("max_steps") or 6
for kind, data in chat.tool_answer(cfg, box, asked, past, steps_max, summary=summary, record=record, shrink=shrink):
if kind == "step":
steps.append(data)
elif kind == "direct" and passages and not box.cited(data): # unless it cites what it remembers
Expand All @@ -415,7 +420,7 @@ def said(words, cited, outcome, **kw):
return "I couldn't answer that: " + str(e), box.approvals, keep | box.links
error = None
try:
text = llm.chat(cfg, chat.messages_for(asked, passages, past, summary, cfg)) if llm.configured(cfg) else chat.fallback(passages)
text = chat.reply_fitted(cfg, asked, passages, past, summary, shrink=shrink) if llm.configured(cfg) else chat.fallback(passages)
except llm.LLMError as e:
text, error = "I couldn't answer that: " + str(e), str(e)
how = "error" if error else "fallback" if notice or not llm.configured(cfg) else "answered"
Expand Down
Loading
Loading