Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ee327cf
fix(assistant): let the assistant's tools be imported first
claude Oct 16, 2026
defb487
feat(chat): each answer records what produced it
claude Oct 17, 2026
983d046
Merge remote-tracking branch 'origin/main' into claude/project-thread…
claude Oct 17, 2026
acf97d7
test(notes): keep the daily notes test from failing on 2026-10-17
claude Oct 17, 2026
db85b14
Merge origin/main into claude/project-thread-r960xc-c1-record
claude Oct 17, 2026
275692b
Merge remote-tracking branch 'origin/main' into port-c1
claude Oct 17, 2026
33a2c19
feat(chat): choose tools, a tool-call limit and a scope per answer
claude Oct 17, 2026
448565d
Merge remote-tracking branch 'origin/claude/project-thread-r960xc-c1-…
claude Oct 17, 2026
957c3f1
docs(chat): choosing tools, namespaces and a limit; regenerate the cl…
claude Oct 17, 2026
8759a60
feat(chat): a choice made as a conversation starts is the latest choice
claude Oct 17, 2026
73d10e7
fix(chat): setting the server up keeps every tool, whatever the lates…
claude Oct 17, 2026
554668b
Merge remote-tracking branch 'origin/main' into claude/project-thread…
claude Oct 17, 2026
a22d140
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
3f0b9de
test(entities): wait for the typed search before counting merge options
claude Oct 17, 2026
33e40f0
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
dde1384
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
e6b5cfa
test(settings): the AI section's server values include ai.max_tool_calls
claude Oct 17, 2026
cdf1f0c
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
a9e21e5
fix(chat): find_resources belongs to the archive's tools
claude Oct 17, 2026
daa8aaa
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
98b7a8b
docs(chat): a new built-in tool joins a tool group
claude Oct 17, 2026
093d4af
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
1b55d2b
fix(ci): the upgrade check counts entities of every type
claude Oct 17, 2026
3256021
feat(chat): deep research, a plan to approve and a cited report
claude Oct 17, 2026
1334ee2
Merge origin/main into claude/project-thread-r960xc-c8-research
claude Oct 17, 2026
7f38bae
Merge origin/main into claude/project-thread-r960xc-c8-research
claude Oct 17, 2026
7181dd6
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
d387fb0
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
5bee623
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
80e32b9
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
78cab4a
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
5cbbbe9
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
c40aaba
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
8c887d7
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 17, 2026
ab19e77
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 18, 2026
0800980
Merge remote-tracking branch 'origin/main' into port-c8
claude Oct 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Deep research.** Give the assistant a goal (`/research <goal>`, `research: true` on a question, `POST
/api/v1/chats/<id>/research`, its `start_research` tool, or MCP) and it plans the research: up to three follow-up
questions, the sub-questions, where it looks (the archive, the graph, notes, the web, the code sandbox), the tool
calls it may make, the report's outline, and a labelled cost estimate. Approve the plan as it is or edited, revise
it, or say "yes" in a chat room; an API key or an app only declines it unless `research.agents_skip_review` is on.
It runs in the background, one run per person at a time, within its tool calls, `research.max_minutes` and your
budgets, survives a restart, and stops when asked; the cited report comes back in the conversation, its claims
checked against their sources, and saves as a note or makes a podcast episode. The web app's plan and progress
cards are in progress ([Deep research](docs/assistant.md#deep-research)).
- **Read any message aloud.** Every question and answer in a conversation has a speaker button: one click reads it
aloud by the speech model and voice picked in Settings → AI assistant → Voice (else the browser's own speech),
another stops it. One message plays at a time, long answers go in pieces under every provider's limit, and each
Expand Down
67 changes: 67 additions & 0 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,7 @@ tools it may use, how many tool calls it may make, and which namespaces it reads
- `changes` (proposals that become approvals);
- `files` (importing files and web pages);
- `code` (the [code sandbox](#code-sandbox): Python and bash, once an admin turns it on);
- `research` ([deep research](#deep-research): `start_research` plans research that waits for your approval);
- `extensions`;
- `server` (admins);
- `browser`;
Expand Down Expand Up @@ -472,6 +473,72 @@ driver); the Podman driver for hosts without Docker; GPUs; a direct sandbox API
assistant); one real run against OpenShell in CI (the tests use a fake gateway with OpenShell's own client); showing
an image the sandbox shared inline in the answer, not only as a download; the 20-file cap as a setting.

## Deep research

Built in the API, MCP, chat rooms and the assistant's tools; the web app's plan and progress cards are in progress.
Give the assistant a goal that needs many look-ups, and it comes back with a plan; once you approve it, the research
runs in the background and a cited report comes back in the conversation.

- **Asking.** Start a question with `/research` (`/research what changed in the capsid work this year?`), send it with
`research: true` (`POST /api/v1/chats/<id>/messages`), call `POST /api/v1/chats/<id>/research {goal}`, or ask the
assistant for research: its `start_research` tool (the `research` [tool group](#choosing-tools-namespaces-and-a-limit))
works from chat, voice and [chat rooms](chat-rooms.md). It needs a language model that uses tools.
- **The plan.** One model call reads the goal and writes up to three follow-up questions, each with answers to pick
from, and a plan: a title, the sub-questions it will answer, where it will look (`archive`, `graph`, `notes`, `web`,
`code`: only the sources this person has, and of those only what the conversation's choice of tools keeps), the
most tool calls it may make, and the report's outline. With it comes an estimate of what it will cost, always
labelled an estimate: from what each tool call of past research runs cost in the activity ledger, else its likely
tokens at the model's price in Settings (no figure when the model has no price). The plan is the answer to the
question, and it waits as an approval of the assistant's (tool `research`).
- **Approve, edit or revise.** Approve it as it is, or with your edits (`POST /api/v1/approvals/<id>` with
`decision: approve` and `params`: any of `title`, `sub_questions`, `sources`, `max_tool_calls`, `report_outline`), in
the conversation or the Approvals page; or send your answers and edits back for one more planning call (`POST
/api/v1/research/<id>/revise {answers, plan}`), which changes the plan and its approval in place. In a chat room,
"yes" approves the plan as it is. Decline it and nothing runs. `review: false` on `POST /chats/<id>/research` starts
it at once, as your own pick.
- **Agents only decline.** An API key, an app or an agent (MCP, the assistant) can decline a plan but not approve it,
as for every approval, unless an admin turns on `research.agents_skip_review` (off by default): then an agent's plan
starts without review. The `research` settings are saved with `PUT /api/v1/settings/research`; their place in
Settings → AI assistant is in progress.
- **The run.** Once approved, one process runs it at a time (a claim it keeps with a heartbeat every 10 seconds;
another process takes it over after two minutes of silence and goes on from the last finished step), in stages:
- *gather*: each sub-question in turn, with the assistant's tool loop, its share of the tool calls that are left,
and the plan's sources' tools alone (read tools only). Searches and page reads on the web, and code runs that
reach the web, don't ask each time: approving the plan approved them. Any other tool that changes something
still asks.
- *write*: one call writes the report in Markdown, following the outline, citing [n] across every finding,
numbered as one list.
- *check*: each cited claim is checked against what it cites ([check the sources](#chat)); one that isn't
supported is rewritten once, or marked "(not supported by its sources)".
- *done*: the report is an answer in the conversation (`kind: research`), after its last message, with its
[record](#what-each-answer-records) (`research` is its id; its tokens and cost are the whole run's).
- **Limits.** A person has one run going at a time; the next waits its turn. `research.max_tool_calls` (40) caps a
plan's tool calls, and `research.max_minutes` (20) a run's time: past it, the report is written from what it found
and says which sub-questions it didn't get to (outcome `limit`). `research.model` picks another model for research
(empty: the conversation's).
- **Budgets.** Before each stage the [budgets](budgets.md) of the conversation's namespaces and `all` are checked. One
that is over holds the run (`held`, saying why) until the budget allows it, or until you let it go on anyway (`POST
/api/v1/research/<id>/resume`); a budget that skips stops it with what it found.
- **Progress and stopping.** `GET /api/v1/research/<id>` gives its status (planned, declined, queued, running, held,
done, stopped, failed), stage, and progress: sub-questions done, sources found, tool calls, minutes, and tokens and
cost so far. `GET /api/v1/research?chat_id=<id>` lists a conversation's. `POST /api/v1/research/<id>/stop` stops it
after the step it's on (at once while it waits), and what it found so far is saved in the conversation as its
answer, marked stopped.
- **After.** `POST /api/v1/research/<id>/note` saves the report as a [note](notes.md) in the namespace it looked in
(`namespace` when there are several), its archive citations as mention links to what they cite, web ones as links,
and a link back to the conversation (`@[Research: title](chat:12)`), which only you see named. `POST
/api/v1/research/<id>/podcast` makes a [podcast episode](podcasts.md) from the assets the report cites, about its
goal. The report reads aloud and exports like any answer.
- **Agents.** MCP has `start_research` (a new conversation unless `chat_id` is given; `namespaces`, `tools`,
`max_tool_calls`) and `research_status` (the plan and progress, and once it's done the report with its sources and
their links).
- **Cost.** Planning is one call; gathering makes at most the plan's tool calls, each round a call; writing and
checking are two or three calls. Every call is in the activity ledger for the conversation and `research:<id>`.

Refine later: sub-questions in parallel, research on a schedule, sharing a report with a namespace's members, research
as an automation action, a report template people can edit, and the report posted back into the chat room it was
asked in (today the room gets the plan, and the report is in the conversation in Lens).

## Third-party content and prompt injection

The assistant reads text nobody vouched for: what was said in recordings, documents, text read off screens, web pages
Expand Down
4 changes: 3 additions & 1 deletion docs/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,9 @@ Installs keep their data across updates (running `install.sh` again, a new image
indexes arrive through `SCHEMA` on the next start. A field that is only added needs no step when its readers treat it
as missing on older rows: a chat message's record (`meta`,
[What each answer records](assistant.md#what-each-answer-records)) is one, and messages from before it simply have
none. Anything else, such as rewriting stored values, moving data between
none. [Deep research](assistant.md#deep-research) is another: its tables (`research`, and `research_run` for the claim
of the process running one) are new, a message's `kind` is missing on older ones, and notes accept `chat` links
without rewriting any. Anything else, such as rewriting stored values, moving data between
tables, or changing or removing an index that already exists, is a named step in `app/domain/migrations.py`:

* Every process that opens the database (the API, workers, `lens watch`, `lens` commands) runs the steps the database
Expand Down
107 changes: 106 additions & 1 deletion fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
from app.api.v1.routes import namespaces as namespace_routes
from app.api.v1.routes import notebook as note_routes
from app.api.v1.routes import recordings as recording_routes
from app.api.v1.routes import research as research_routes
from app.api.v1.routes import routines as routine_routes
from app.api.v1.routes import search as search_routes
from app.api.v1.routes import topics as topic_routes
Expand All @@ -62,6 +63,7 @@
nsfw,
rdf,
render,
research,
rss,
settings,
sources,
Expand All @@ -87,7 +89,7 @@
AutomationTry,
AutomationUpdate,
)
from app.schemas.chats import ChatSummary
from app.schemas.chats import ChatScope, ChatSummary
from app.schemas.entities import EntityHide, EntityList, EntityRename, EntityRetype, EntityUpdate
from app.schemas.notebook import NotePage
from app.schemas.search import SearchResults
Expand Down Expand Up @@ -2768,6 +2770,109 @@ def decline_approval(ctx: Context, id: int) -> dict[str, Any]:
return {"approval": id, "status": out.get("status")}


# ---------- deep research (docs/assistant.md#deep-research) ----------
@tool(
"start_research",
"Start deep research",
"Research a goal for the person: one planning call makes a plan (the sub-questions, where it looks: the archive, the "
"graph, notes, the web or the code sandbox, the tool calls it may make, the report's outline) and up to three "
"follow-up questions. The plan waits for the person's approval in Lens (url), unless the admin let agents start "
"research without review (research.agents_skip_review); approved, it runs in the background and its cited report "
"comes back in the conversation: follow it with research_status. Needs a token with the write scope.",
Arg("goal", "string", "what to research, in the person's words", required=True, max_length=4000),
Arg("chat_id", "integer", "the conversation to research in; a new one when left out", minimum=1),
Arg("namespaces", "string[]", "only these namespaces (a new conversation's scope)"),
Arg("tools", "string[]", "what it may use: tool groups (archive, graph, notes, web, code) and tools; else the conversation's choice"),
Arg("max_tool_calls", "integer", "the most tool calls it may make (research.max_tool_calls at most)", minimum=1, maximum=500),
writes=True,
)
def start_research(
ctx: Context, goal: str, chat_id: int | None, namespaces: list[str] | None, tools: list[str] | None, max_tool_calls: int | None
) -> dict[str, Any]:
_writer(ctx, "start research")
if not research.available(ctx.cfg):
raise ToolError("research needs a language model that uses tools: an admin sets one up in Settings › AI assistant")
asked = [ctx.ns] if ctx.ns else namespaces
scope = chat_routes._scope(ctx.acl, ctx.db, ChatScope(namespaces=asked)) if asked else None
if chat_id is not None:
c = chat_routes._own_chat(ctx.db, chat_id, ctx.user)
scope = scope if scope is not None else c.get("scope")
else:
chat_id = chat.create(ctx.db, ctx.user.id, goal[:80], scope)
c = chat.get(ctx.db, chat_id, ctx.user.id)
box = research_routes.toolbox(ctx.db, ctx.cfg, ctx.user, ctx.acl, chat_id, scope, None, ctx.request)
if tools is not None and box.unknown(tools):
raise ToolError(f"no tool or tool group {box.unknown(tools)[0]!r}")
box = research_routes.toolbox(
ctx.db, ctx.cfg, ctx.user, ctx.acl, chat_id, scope, tools if tools is not None else c.get("tools"), ctx.request
)
skip = research.settings(ctx.cfg)["agents_skip_review"]
out = research.start(
ctx.db,
ctx.cfg,
box,
" ".join(goal.split()),
"api",
model=research.model_for(ctx.cfg),
max_tool_calls=max_tool_calls,
options={"tools": tools, "max_tool_calls": max_tool_calls},
review=not skip,
)
if out.get("error"):
raise ToolError(out["error"])
r = research.view(ctx.db, out["research"])
return {
"research": r["id"],
"chat_id": chat_id,
"status": r["status"],
"plan": r["plan"],
"questions": r["questions"],
"estimate": r["estimate"],
"approval": r["approval"],
"url": f"{ctx.web}/chat/{chat_id}",
"next": "running: follow it with research_status"
if r["status"] in ("queued", "running")
else "waiting for the person to approve the plan in Lens (url); follow it with research_status",
}


@tool(
"research_status",
"Research status",
"Where a deep research run stands: its plan and status (planned: waiting for the person's approval; queued, "
"running, held by a budget, done, stopped, failed), its stage (gather, write, check, done) and progress "
"(sub-questions done, sources found, tool calls and cost so far). Once it's done (or stopped), the report in "
"Markdown with its numbered sources, each with a link.",
Arg("id", "integer", "the research (start_research's)", required=True, minimum=1),
)
def research_status(ctx: Context, id: int) -> dict[str, Any]: # noqa: A002 - the argument's name
try:
r = research.mine(ctx.db, id, ctx.user.id)
except KeyError:
raise ToolError(f"no research {id} of yours") from None
out = research.view(ctx.db, r)
out["url"] = f"{ctx.web}/chat/{r['chat']}"
if r.get("answer"):
m = ctx.db.one("SELECT content, passages FROM $r", r=R("chat_message", r["answer"])) or {}
ok = None if ctx.user.admin else set(ctx.acl.spaces())
spaces = {
x["id"]: x["space"]
for x in ctx.db.rows(
"SELECT record::id(id) AS id, space FROM recording WHERE id IN $ids",
ids=[R("recording", p["recording_id"]) for p in m.get("passages") or [] if p.get("recording_id") is not None],
)
}
sources = []
for p in m.get("passages") or []:
if p.get("source") == "web":
sources.append({"n": p["n"], "title": p.get("title"), "url": p.get("url"), "site": p.get("site")})
elif p.get("recording_id") in spaces and (ok is None or spaces[p["recording_id"]] in ok):
sources.append({"n": p["n"], "title": p.get("title"), "at": p.get("time"), "url": ctx.link(p["recording_id"], p.get("t0"))})
out["report"] = m.get("content")
out["sources"] = sources
return out


@tool(
"event_kinds",
"Kinds of activity",
Expand Down
2 changes: 2 additions & 0 deletions fastapi_backend/app/api/v1/router.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
rdf,
recordings,
requests,
research,
routines,
search,
searches,
Expand Down Expand Up @@ -136,6 +137,7 @@
extensions,
marketplaces,
chats,
research,
browser,
collections,
batches,
Expand Down
Loading
Loading