diff --git a/README.md b/README.md index 65aaaf62..b1cf847d 100644 --- a/README.md +++ b/README.md @@ -99,9 +99,10 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p [questions about Lens](docs/assistant.md#questions-about-lens-itself), [chat rooms](docs/chat-rooms.md), [email](docs/assistant-email.md), [speech providers](docs/speech-providers.md); [checks](docs/requirements.md#ask-and-get-cited-answers)) - **Work toward a goal in projects.** A project has a name, a goal in your words, instructions and the namespaces it - draws on. Its coordinator conversation looks at the project as a whole and is told about every thread; each thread is - a conversation about one part of the work, scoped to the project's namespaces and marked done when finished. Every - answer in a project reads its goal and instructions. Agents start and change projects over MCP and the API. + draws on. Its coordinator conversation looks at the project as a whole: it is told about every thread, starts threads + with a brief that the assistant answers there in the background, reads them and marks them done. Each thread is a + conversation about one part of the work, scoped to the project's namespaces. Every answer in a project reads its goal + and instructions. Agents start and change projects over MCP and the API. ([Projects](docs/projects.md); [checks](docs/requirements.md#work-toward-a-goal-in-projects)) - **Extend the assistant.** Add tools (a prompt, a web request, a graph drawn on the canvas, or sandboxed Python), skills, hooks and MCP servers, by code, on the canvas or just by asking. Remote MCP servers connect by address, and diff --git a/changelog.d/projects.md b/changelog.d/projects.md index 67db56c4..374174dd 100644 --- a/changelog.d/projects.md +++ b/changelog.d/projects.md @@ -2,5 +2,7 @@ and the namespaces it draws on. It comes with a coordinator conversation about the project as a whole, which is told about every thread, and its threads are conversations scoped to its namespaces, marked done when finished; your own conversations can be brought in or taken out, and deleting a project keeps them. Every answer in a project reads its - goal and instructions. `/api/v1/projects` and the MCP tools `list_projects`, `read_project`, `create_project` and + goal and instructions. The coordinator starts threads with a brief (`start_thread`), reads them and marks them done; + a brief, from it, you or an agent, is answered in the background as you, with your tools, approvals and budgets. + `/api/v1/projects` and the MCP tools `list_projects`, `read_project`, `create_project` and `update_project` do it all for agents ([Projects](docs/projects.md)). diff --git a/docs/projects.md b/docs/projects.md index 2d7667ba..548e8863 100644 --- a/docs/projects.md +++ b/docs/projects.md @@ -5,10 +5,11 @@ case, a piece of code. You give it a name, the goal in your own words, instructi it draws on. Every conversation in it reads all of that, so you don't explain the project again each time. Status: **built**: projects with a goal, instructions and one or more namespaces; a coordinator conversation for the -project as a whole, told about every thread; threads, each a conversation in the project's namespaces, marked done when -finished; your own conversations brought in or taken out; through the REST API and MCP. **Planned**: the coordinator -starting threads with a brief and reading them, scheduled threads, a Projects page in the web app, chat rooms and email -routed to a project, members sharing one, and a project's own memory. +project as a whole, told about every thread, which starts threads with a brief, reads them and marks them done; +threads, each a conversation in the project's namespaces, marked done when finished; briefs answered in the background; +your own conversations brought in or taken out; through the REST API and MCP. **Planned**: scheduled threads, a +Projects page in the web app, chat rooms and email routed to a project, members sharing one, and a project's own +memory. ## What a project is @@ -42,6 +43,33 @@ in Chat). Changing the project's namespaces changes every one of its conversatio Deleting a project keeps its conversations, as ordinary ones in Chat. Deleting its coordinator conversation makes a new one the next time the project is opened. +## The coordinator's tools + +In a project's conversations the assistant has the tools of **This project** (one of the groups in the tool picker, +[Choosing tools](assistant.md#choosing-tools-namespaces-and-a-limit)): + +| Tool | What it does | +| --- | --- | +| `project_threads` | the project's threads: each one's title, whether it's done, whether an answer is being written, its last message | +| `read_thread` | a thread's last messages | +| `start_thread` | from the coordinator: a new thread with a title and a brief, which is answered there in the background (below) | +| `finish_thread` | mark a thread done, or not done | + +So you can say to the coordinator "split this into the wood, the tools and a weekend plan" and it starts a thread for +each, then reads what they found. One answer starts at most three threads. A brief written after the answer read text +from outside Lens (a document, a web page, another thread) could carry that text's instructions, so then +`start_thread` asks for your approval first, like the other tools that would act on it. + +## Briefs + +A thread can start with a brief: what to do, with everything it needs. The coordinator writes one with `start_thread`; +you or an agent give one with `brief` on `POST /api/v1/projects/{id}/threads` (or MCP `update_project`'s `new_thread` +and `brief`). It is asked as the thread's question and answered in the background, as you: with the tools you have +there (the conversation's choice of them) and the same approvals, every model call counted for the answer in the +[activity ledger](activity.md) and held to your budgets. The thread shows `answering` until the answer is saved, and +Stop (`POST /api/v1/chats/{id}/stop`) stops it between steps. If Lens stops while a brief is being answered, the thread +says so within a few minutes, so it's never left waiting: ask again. + ## The API | Call | What it does | @@ -52,7 +80,7 @@ new one the next time the project is opened. | `PATCH /api/v1/projects/{id}` | change its name, goal, instructions or namespaces, or archive it | | `DELETE /api/v1/projects/{id}` | delete it; its conversations stay | | `GET /api/v1/projects/{id}/threads` | its threads, the coordinator among them | -| `POST /api/v1/projects/{id}/threads` | a new thread (`title`), or bring in one of your conversations (`chat`) | +| `POST /api/v1/projects/{id}/threads` | a new thread (`title`), or bring in one of your conversations (`chat`); with a `brief`, asked and answered there in the background | | `PATCH /api/v1/projects/{id}/threads/{chat}` | mark a thread done, or not (`done`) | | `DELETE /api/v1/projects/{id}/threads/{chat}` | take a thread out of the project | @@ -62,7 +90,7 @@ same for agents; the last two need a token with the write scope. ## Refine later -- The coordinator starting threads with a brief, reading them and marking them done, as its own tools. +- Threads answering their briefs at the same time as each other, with a limit per person. - Scheduled threads: a prompt asked on a schedule, in a new thread each time or the same one. - A Projects page in the web app. - Chat rooms, Telegram, Slack and email routed to a project. diff --git a/docs/requirements.md b/docs/requirements.md index dd35efb7..5c1389f0 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -106,6 +106,8 @@ request, to another check that proves the same thing; loosening a requirement or | A project with a goal, instructions and namespaces, a coordinator conversation and threads scoped to its namespaces, marked done, brought in or taken out; deleting it keeps its conversations | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_a_project_with_a_goal_namespaces_and_threads`, `test_changing_its_namespaces_moves_its_conversations` | | A project is its owner's alone and draws only on namespaces they can read | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_projects_are_their_owners_and_draw_on_readable_namespaces` | | Every answer in a project reads its goal and instructions; the coordinator is told about the threads, as data | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_every_answer_in_a_project_reads_what_it_is_for` | +| The coordinator starts threads with a brief, reads them and marks them done; a thread can't start threads; it asks first after reading outside text, and starts a few per answer at most | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_the_coordinator_starts_threads_with_a_brief`, `test_starting_threads_after_reading_outside_text_asks_first` | +| A brief from the person, an agent or the coordinator is answered in the background as the person; one cut off by a stopped process is marked, never left waiting | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_a_brief_from_the_api_or_an_agent_is_answered_in_the_background`, `test_a_brief_cut_off_by_a_stop_is_marked_by_the_sweep` | | Agents list, read, start and change projects over MCP | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_agents_use_projects_over_mcp` | ## Extend the assistant diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py index f5280cd3..5edcef2e 100644 --- a/fastapi_backend/app/api/mcp_tools.py +++ b/fastapi_backend/app/api/mcp_tools.py @@ -2087,8 +2087,9 @@ def create_project(ctx: Context, name: str, goal: str | None, instructions: str "update_project", "Change a project", "Change one of your projects: its name, goal, instructions or namespaces (its conversations draw on the new ones), " - "archive it, mark one of its threads done (thread_id, done), or start a thread (new_thread: its title; ask in it " - "with the web app or POST /api/v1/chats/{id}/messages). Only what you give changes. Needs a token with the write scope.", + "archive it, mark one of its threads done (thread_id, done), or start a thread (new_thread: its title) with a " + "brief Lens's assistant answers there in the background (read it with read_chat). Only what you give changes. " + "Needs a token with the write scope.", Arg("project_id", "integer", "the project (from list_projects)", required=True), Arg("name", "string", "its new name", max_length=120), Arg("goal", "string", "its new goal", max_length=4000), @@ -2098,6 +2099,7 @@ def create_project(ctx: Context, name: str, goal: str | None, instructions: str Arg("thread_id", "integer", "a thread to mark done or not done"), Arg("done", "boolean", "with thread_id: done or not"), Arg("new_thread", "string", "start a thread with this title", max_length=120), + Arg("brief", "string", "with new_thread: what to do there, answered in the background by Lens's assistant", max_length=4000), writes=True, personal=True, ) @@ -2112,8 +2114,11 @@ def update_project( thread_id: int | None, done: bool | None, new_thread: str | None, + brief: str | None, ) -> dict[str, Any]: _writer(ctx, "change projects") + if brief and not new_thread: + raise ToolError("give new_thread (its title) with brief") changes = {"name": name, "goal": goal, "instructions": instructions, "namespaces": namespaces, "archived": archived} started = None if any(v is not None for v in changes.values()): @@ -2123,7 +2128,7 @@ def update_project( raise ToolError("give done (true or false) with thread_id") project_routes.update_thread(project_id, thread_id, ThreadUpdate(done=done), ctx.user, ctx.db) if new_thread: - started = project_routes.start_thread(project_id, ctx.user, ctx.db, ThreadCreate(title=new_thread)).id + started = project_routes.start_thread(project_id, ctx.user, ctx.db, ctx.cfg, ThreadCreate(title=new_thread, brief=brief)).id out = _project(ctx, project_routes.get_project(project_id, ctx.user, ctx.db)) if started is not None: out["started"] = {"chat_id": started, "url": f"{ctx.web}/chat/{started}"} diff --git a/fastapi_backend/app/api/v1/routes/projects.py b/fastapi_backend/app/api/v1/routes/projects.py index 87aaffd2..043f7619 100644 --- a/fastapi_backend/app/api/v1/routes/projects.py +++ b/fastapi_backend/app/api/v1/routes/projects.py @@ -7,7 +7,7 @@ from fastapi import APIRouter, HTTPException, Query -from app.api.deps import Acl, CurrentUser, Db, Principal, Writer +from app.api.deps import Acl, Cfg, CurrentUser, Db, Principal, Writer from app.domain import projects from app.domain.store import DB from app.schemas.common import Created, Ok @@ -103,16 +103,20 @@ def list_threads(pid: int, user: CurrentUser, db: Db) -> list[ProjectThread]: @router.post("/{pid}/threads") -def start_thread(pid: int, user: Writer, db: Db, body: ThreadCreate | None = None) -> Created: - """A new thread: a conversation in the project's namespaces (ask in it with POST /chats/{id}/messages). With `chat`, - one of your conversations joins the project instead, drawing on its namespaces from then on.""" +def start_thread(pid: int, user: Writer, db: Db, cfg: Cfg, body: ThreadCreate | None = None) -> Created: + """A new thread: a conversation in the project's namespaces (ask in it with POST /chats/{id}/messages). With a + `brief`, it is asked there at once and answered in the background, as you, with your tools and approvals: GET + /chats/{id} shows the answer once it's written (the thread is `answering` until then). With `chat`, one of your + conversations joins the project instead, drawing on its namespaces from then on (with a brief, asked there next).""" body = body or ThreadCreate() p = own(db, pid, user) try: + cid = body.chat if body.chat is not None else projects.start_thread(db, p, body.title) if body.chat is not None: projects.adopt(db, p, body.chat) - return Created(id=body.chat) - return Created(id=projects.start_thread(db, p, body.title)) + if body.brief and body.brief.strip(): + projects.brief(db, cfg, p, cid, body.brief, via="web" if user.via == "access" else "api") + return Created(id=cid) except KeyError: raise HTTPException(404, "no such conversation of yours") from None except ValueError as e: diff --git a/fastapi_backend/app/domain/ai_tools.py b/fastapi_backend/app/domain/ai_tools.py index ca5830ff..6f7007d6 100644 --- a/fastapi_backend/app/domain/ai_tools.py +++ b/fastapi_backend/app/domain/ai_tools.py @@ -13,6 +13,7 @@ from . import ( auth, batches, + chat as chatmod, classify as classifymod, connection_tools, cypher, @@ -545,6 +546,44 @@ ] +# a project's conversations (projects.py): the coordinator and threads read the project's threads, start one with a +# brief (answered in the background, with the person's own tools and approvals) and mark one done +PROJECT_TOOLS = [ + ( + "project_threads", + "The threads of this conversation's project, the most recent first: each one's id, title, whether it's done, " + "whether an answer is being written in it now, and its last message.", + {"include_done": {"type": "boolean"}}, + [], + False, + ), + ( + "read_thread", + "Read one of the project's threads: its last messages (the version showing), oldest first.", + {"thread_id": _I, "limit": _I}, + ["thread_id"], + False, + ), + ( + "start_thread", + "Start a thread in this project for one part of the work: a short title and a brief saying what to do, with " + "everything it needs (the thread doesn't see this conversation, only the project's goal and instructions). " + "The assistant answers the brief there in the background, with the same tools and approvals; follow it with " + "project_threads and read_thread.", + {"title": _S, "brief": _S}, + ["title", "brief"], + True, + ), + ( + "finish_thread", + "Mark one of the project's threads done when its part of the work is finished (done: false to open it again).", + {"thread_id": _I, "done": {"type": "boolean"}}, + ["thread_id"], + True, + ), +] + + def builtin_names(): """The assistant's own tool names, which extensions can't take.""" every = ( @@ -553,6 +592,7 @@ def builtin_names(): + ops_tools.FILE_TOOLS + AUTHOR_TOOLS + MEMORY_TOOLS + + PROJECT_TOOLS + AUTOMATION_TOOLS + browsermod.TOOLS + sandboxmod.TOOLS @@ -589,6 +629,7 @@ def builtin_names(): + ops_tools.FILE_TOOLS + AUTHOR_TOOLS + MEMORY_TOOLS + + PROJECT_TOOLS + AUTOMATION_TOOLS + sandboxmod.TOOLS + researchmod.TOOLS @@ -638,6 +679,7 @@ def builtin_names(): "connected": {"label": "Mail, calendars and files", "tools": [t[0] for t in connection_tools.TOOLS]}, "notes": {"label": "Notes", "tools": ["find_notes", "read_note", "write_note", "update_note", "write_spec"]}, "memory": {"label": "Namespace memory", "tools": [t[0] for t in MEMORY_TOOLS]}, + "project": {"label": "This project", "tools": [t[0] for t in PROJECT_TOOLS]}, "activity": { "label": "Timeline and automations", "tools": ["timeline", "mark_activities", "approvals", "decline_approval", "automations", "test_automation"] @@ -687,6 +729,8 @@ def _why_missing(group, cfg, admin=False): return "nothing is connected yet: add a connection in Settings › Sources", None if group == "memory": return "in a conversation about one namespace whose own assistant is on", None + if group == "project": + return "in a project's conversations", None return "needs editor access to a namespace", None @@ -726,6 +770,7 @@ def catalog(box): + ops_tools.FILE_TOOLS + AUTHOR_TOOLS + MEMORY_TOOLS + + PROJECT_TOOLS + AUTOMATION_TOOLS + sandboxmod.TOOLS + webmod.TOOLS @@ -831,6 +876,7 @@ def __init__( # the project the conversation is in (projects.py), when it's this person's project = projectsmod.for_chat(db, chat_id) self.project = project if project and project["account"] == self.account else None + self.started = 0 # threads this answer started (start_thread) # the browser the person is talking from, while its side panel is connected (and an admin hasn't turned it off) on = browser and browsermod.enabled(cfg) and browsermod.live(db, browser, user["id"]) self.browser = str(browser) if on else None @@ -922,6 +968,8 @@ def specs(self): tools += AUTOMATION_TOOLS if self.home: tools += [t for t in MEMORY_TOOLS if self.home[0] in self.editable or not t[4]] + if self.project: + tools += PROJECT_TOOLS if self.browser: tools += browsermod.TOOLS if self.sandbox: @@ -1362,6 +1410,73 @@ def t_forget(self, memory_id): m = ns_assistant.get(self.db, sid, int(memory_id)) return self._approval("forget_memory", {"namespace": name, "memory_id": m["id"]}, f"Forget “{m['text'][:120]}”") + # ---- the conversation's project (projects.py) ---- + def _thread(self, thread_id): + try: + return projectsmod.thread(self.db, self.project, int(thread_id)) + except KeyError: + raise ValueError(f"there's no thread {thread_id} in this project") from None + + def t_project_threads(self, include_done=True): + rows = projectsmod.threads(self.db, self.project) + if not include_done: + rows = [t for t in rows if not t["done"]] + out = [ + store.clean( + { + "thread_id": t["id"], + "title": t["title"], + "done": t["done"] or None, + "answering": t["answering"] or None, + "messages": t["messages"], + "last": t.get("last"), + "updated_at": t.get("updated_at"), + } + ) + for t in rows + ] + return {"project": self.project["name"], "threads": out}, f"Listed the project's threads: {len(out)}" + + def t_read_thread(self, thread_id, limit=20): + t = self._thread(thread_id) + msgs = chatmod.path(self.db, t["id"])[-max(1, min(int(limit or 20), 50)) :] + out = [ + store.clean( + { + "role": m["role"], + "text": (m.get("content") or "")[:4000], + "at": m.get("created_at"), + "tools": [s.get("summary") for s in m.get("steps") or [] if s] or None, + "error": m.get("error"), + } + ) + for m in msgs + ] + return {"thread_id": t["id"], "title": t["title"], "done": t["done"], "answering": t["answering"], "messages": out}, ( + f"Read the thread “{t['title']}”: {len(out)} message(s)" + ) + + def t_start_thread(self, title, brief): + if self.chat != self.project.get("coordinator"): + raise ValueError("threads are started from the project's coordinator conversation") + if self.started >= projectsmod.STARTS_PER_ANSWER: + raise ValueError(f"one answer starts at most {projectsmod.STARTS_PER_ANSWER} threads") + title = " ".join(str(title or "").split())[:120] + if not title: + raise ValueError("a thread needs a title") + if len(str(brief or "")) > projectsmod.BRIEF_MAX: + raise ValueError(f"a brief has up to {projectsmod.BRIEF_MAX} characters") + self.started += 1 + args = {"project": self.project["id"], "title": title, "brief": str(brief).strip()} + if self.wary(): # a brief written after reading outside text may carry it: the person looks first + return self._approval("start_thread", args, f"Start the thread “{title}” in {self.project['name']}" + self.why()) + cid = start_brief(self.db, self.cfg, self.project, title, args["brief"]) + return {"thread_id": cid, "title": title, "status": "started: the brief is being answered there"}, f"Started the thread “{title}”" + + def t_finish_thread(self, thread_id, done=True): + t = projectsmod.set_done(self.db, self.project, self._thread(thread_id)["id"], bool(done)) + return {"thread_id": t["id"], "done": t["done"]}, f"Marked “{t['title']}” {'done' if t['done'] else 'not done'}" + def _run(self, name, fn, args): try: out, summary = fn(**{k: v for k, v in (args or {}).items() if v is not None}) @@ -2252,6 +2367,13 @@ def cited(self, text): return [{**r, "used": r["n"] in used} for r in self.refs if r["n"] in used] +def start_brief(db, cfg, p, title, brief): + """A new thread in project `p`, titled `title`, its brief asked by the coordinator and answered in the background.""" + cid = projectsmod.start_thread(db, p, title) + projectsmod.brief(db, cfg, p, cid, brief, by="coordinator", via="project") + return cid + + def approve(db, cfg, aid, user, editable, decision="approve", base=None, admin=False, readable=None, params=None, via="web"): """Carry out an approved action: a batch run (or a sample of it), an entity change, a note an automation's agent proposed, or a change to the server (ops_tools.py: settings, a namespace, importing attached files or web pages, @@ -2341,6 +2463,12 @@ def _carry_out(db, cfg, a, user, editable, decision, base, admin, readable, via= elif a["tool"] in ("write_note", "update_note"): # proposed by an automation's agent (actions_ai.Box) box = Toolbox(db, cfg, user, readable if readable is not None else editable, editable, None, a["chat"], base, admin) result = getattr(box, "t_" + a["tool"])(**args)[0] + elif a["tool"] == "start_thread": # a brief the coordinator wrote after reading outside text, now approved + try: + p = projectsmod.get(db, int(args["project"]), user["id"]) + except KeyError: + raise PermissionError("that project isn't yours, or is gone") from None + result = {"thread": start_brief(db, cfg, p, args["title"], args["brief"])} elif a["tool"] == "forget_memory": sid = {v: k for k, v in store.space_names(db).items()}.get(args["namespace"]) if sid not in editable: diff --git a/fastapi_backend/app/domain/projects.py b/fastapi_backend/app/domain/projects.py index b39fa46f..28bb11af 100644 --- a/fastapi_backend/app/domain/projects.py +++ b/fastapi_backend/app/domain/projects.py @@ -6,7 +6,9 @@ - the coordinator, the project's own conversation, made with it (and made again if it was deleted): talking about the project as a whole, it is told about every thread; -- threads, each about one part of the work; one marked done stays to read. +- threads, each about one part of the work; one marked done stays to read. A thread can start with a brief (from the + person, an agent or the coordinator's start_thread), which the assistant answers in the background (work), with the + person's tools and approvals; a process that stops mid-answer leaves the thread marked by the sweep, not stuck. Every answer in a project's conversation reads what the project is (projects.note): its name, goal and instructions, which conversation this is, and, for the coordinator, the threads. The goal and instructions are the person's own @@ -20,7 +22,13 @@ from __future__ import annotations -from . import chat, store, untrusted +import datetime as dt +import logging +import threading + +from . import activity, auth, chat, llm, store, untrusted + +log = logging.getLogger("lens") R = store.R NAME_MAX = 120 @@ -31,6 +39,12 @@ THREADS_MAX = 500 # per project NOTE_THREADS = 30 # threads the coordinator is told about, the most recent first LAST_CHARS = 300 # of a thread's last message, in the coordinator's note and in listings +BRIEF_MAX = 4000 +STARTS_PER_ANSWER = 3 # threads one answer of the coordinator may start +BRIEF_STEPS = 12 # rounds of tool calls a brief's answer may take, at most +STALE_SECONDS = 600 # a brief's answer with no sign of life for this long was cut off (the process stopped) +SWEEP_SECONDS = 60 +STOPPED = "Lens stopped before this was answered: ask again." FIELDS = ( "record::id(id) AS id, account, name, goal ?? '' AS goal, instructions ?? '' AS instructions, spaces ?? [] AS spaces, " "coordinator, archived ?? false AS archived, created_at, updated_at" @@ -278,6 +292,174 @@ def for_chat(db, cid): return None +# ---------- briefs: a question answered in the background ---------- +def brief(db, cfg, p, cid, text, by="person", via="api"): + """Ask `text` in thread `cid`, where it goes on from, and answer it in the background (launch). `by`: person, or + coordinator when the coordinator's start_thread wrote it (then it isn't taken for the person's own message). The + question's id.""" + text = str(text or "").strip() + if not text: + raise ValueError("A brief says what to do.") + if len(text) > BRIEF_MAX: + raise ValueError(f"A brief can have up to {BRIEF_MAX} characters.") + t = thread(db, p, cid) + if t["answering"]: + raise ValueError("An answer is being written in that thread: wait for it, or stop it.") + c = db.one("SELECT title, title_by, head FROM $r", r=R("chat", int(cid))) or {} + if c.get("head") is None and chat.lens_titles(c): + chat.title_from_question(db, int(cid), text) + origin = {"project": p["id"], "by": "coordinator"} if by == "coordinator" else None + qid = chat.add(db, int(cid), "user", text, origin=origin, meta=chat.question_record(via)) + db.q( + "UPDATE $r SET answering = true, stop_requested = false, working = $w", + r=R("chat", int(cid)), + w={"question": qid, "beat": store.now()}, + ) + touch(db, p["id"]) + launch(db, cfg, int(cid), qid) + return qid + + +def launch(db, cfg, cid, qid): + """Answer the question in the background, on a thread of its own.""" + threading.Thread(target=work, args=(db, cfg, cid, qid), daemon=True, name=f"project thread {cid}").start() + + +def _account(db, aid): + a = auth.get_account(db, aid) + if not a or a.get("disabled") or a.get("removed_at"): + return None + return {"id": a["id"], "email": a["email"], "admin": bool(a.get("admin"))} + + +def _beat(db, cid, qid): + db.q("UPDATE $r SET working.beat = $t WHERE working.question = $q", r=R("chat", cid), t=store.now(), q=qid) + + +def _end(db, cid): + db.q("UPDATE $r SET answering = false, stop_requested = false, working = NONE", r=R("chat", cid)) + + +def work(db, cfg, cid, qid): + """Answer question `qid` of conversation `cid` as its person, with the tools they have there (the conversation's + choice of them) and the same approvals: the tool loop (chat.tool_answer), every call counted for the answer in the + activity ledger, the answer saved with its record. Stop (POST /chats/{cid}/stop) stops it between steps.""" + from . import ai_tools + + try: + c = db.one( + "SELECT account, scope, model, tools, max_tool_calls, title, title_by FROM $r WHERE working.question = $q", + r=R("chat", cid), + q=qid, + ) + if not c: + return + q = chat.said(db, cid, qid, "user") + acc = _account(db, c["account"]) + mid = db.next_id("chat_message") + steps, text, how, error, notice = [], "", "answered", None, None + record = chat.Record(cfg, c.get("model"), c.get("scope"), c.get("tools"), c.get("max_tool_calls")) + if acc is None: + text, how, error = "(no answer)", "error", "the person this thread is for can't sign in anymore" + elif not llm.configured(cfg): + text, how, error = "(no answer)", "error", "no model is set up: pick one in Settings › Model" + else: + roles = auth.roles(db, acc) + readable, editable = set(roles), {s for s in roles if auth.allows(roles, s, "editor")} + past = chat.path(db, cid, q.get("parent")) if q.get("parent") is not None else [] + summary = chat.memory(db, cfg, cid, past) + box = ai_tools.Toolbox( + db, + cfg, + {"id": acc["id"], "email": acc["email"]}, + readable, + editable, + c.get("scope"), + cid, + None, + acc["admin"], + said=q["content"], + # a brief the coordinator wrote may carry what it read: tools that send data out ask first + outside=(["the coordinator's brief"] if (q.get("origin") or {}).get("by") == "coordinator" else []) + + ai_tools.read_before(past, summary), + only=c.get("tools"), + ) + on = chat.Answering(db, cid) # what POST /chats/{cid}/stop asks of it (_end clears it after) + try: + with activity.scope(db, f"chat:{cid}", f"chat_message:{mid}", cfg=cfg): + steps_max = max(int(cfg["ai"].get("max_steps") or 6), 6) + for kind, data in chat.tool_answer( + cfg, + box, + q["content"], + past, + min(steps_max, BRIEF_STEPS), + c.get("model"), + summary=summary, + record=record, + max_tool_calls=c.get("max_tool_calls"), + ): + if kind == "step": + steps.append(data) + _beat(db, cid, qid) + if on.stop_requested(force=True): + text, how, notice = text or "(stopped)", "stopped", None + break + else: + text, how = data, "direct" if kind == "direct" else "answered" + else: + box.after_answer(text) + except llm.ToolsUnsupported: + text, how, error = "(no answer)", "error", "this model can't use tools: pick one that can in Settings › Model" + except (llm.LLMError, activity.OverBudget) as e: + text, how, error = text or "(no answer)", "error", str(e) + cited = box.cited(text) if how != "error" else [] + with activity.scope(db, f"chat:{cid}", f"chat_message:{mid}", cfg=cfg): + meta = record.meta(db, how, f"chat_message:{mid}") + wrote = c.get("model") or cfg["llm"].get("model") + chat.add( + db, + cid, + "assistant", + text or "(no answer)", + cited, + steps=steps, + notice=notice, + error=error, + model=wrote, + parent=qid, + mid=mid, + meta=meta, + stopped=how == "stopped", + ) + if how not in ("error", "stopped") and q.get("parent") is None and chat.lens_titles(c): + chat.auto_title(db, cfg, cid, q["content"], text) + return + chat.add(db, cid, "assistant", text, [], steps=steps, error=error, parent=qid, mid=mid, meta=record.meta(db, how)) + except Exception: # noqa: BLE001 - the thread says so rather than waiting for an answer that won't come + log.exception("project thread %s: the brief's answer failed", cid) + try: + chat.add(db, cid, "assistant", "(no answer)", [], error="Something went wrong while answering. Try again.", parent=qid) + except Exception: # noqa: BLE001 + log.warning("project thread %s: couldn't save the error", cid, exc_info=True) + finally: + _end(db, cid) + + +def sweep(db, cfg, log=None): + """Briefs whose answer stopped with its process (no step for STALE_SECONDS): the thread says so and is free again.""" + cutoff = (dt.datetime.now(dt.timezone.utc) - dt.timedelta(seconds=STALE_SECONDS)).isoformat(timespec="seconds") + for row in db.rows("SELECT record::id(id) AS id, working FROM chat WHERE working != NONE AND working.beat < $c", c=cutoff): + qid = (row.get("working") or {}).get("question") + took = db.rows( + "UPDATE $r SET working = NONE, answering = false WHERE working.question = $q RETURN id", r=R("chat", row["id"]), q=qid + ) + if took and qid is not None: + chat.add(db, row["id"], "assistant", "(no answer)", [], error=STOPPED, parent=qid) + if log: + log(f"project thread {row['id']}: its brief's answer was cut off") + + # ---------- what an answer in a project is told ---------- def note(db, p, cid, cfg=None): """What the model is told in one of the project's conversations: the project, its goal and instructions (the diff --git a/fastapi_backend/app/domain/routines.py b/fastapi_backend/app/domain/routines.py index 9cc0a04f..c91f85bc 100644 --- a/fastapi_backend/app/domain/routines.py +++ b/fastapi_backend/app/domain/routines.py @@ -677,8 +677,9 @@ def start(db, cfg_fn, stop, log=None): """The scheduling threads, until `stop` is set: watched folders scanned when due (every sources.check_seconds), routines run when due (every CHECK_SECONDS), the scheduled backup taken when due, and automations' timers fired when due (timers.py). The API runs them with its background work; `lens worker` runs them too, since Docker and - the packages run the API without it. Research runs no process is on (research.py) are picked up here too.""" - from . import research # it loads the chat, which loads what routines run + the packages run the API without it. Research runs no process is on (research.py) are picked up here too, and projects' briefs + whose answer was cut off are marked (projects.py).""" + from . import projects, research # they load the chat, which loads what routines run def every(seconds, fn, what): def loop(): @@ -700,6 +701,7 @@ def loop(): every(lambda: notebook.QUIET_SECONDS / 2, notebook.organise_due, "notes"), every(lambda: CHECK_SECONDS, backups.run_due, "backups"), every(lambda: research.SWEEP_SECONDS, research.sweep, "research"), + every(lambda: projects.SWEEP_SECONDS, projects.sweep, "projects"), timers.start(db, cfg_fn, stop, log=log), ] diff --git a/fastapi_backend/app/schemas/chats.py b/fastapi_backend/app/schemas/chats.py index 87533f2a..278be719 100644 --- a/fastapi_backend/app/schemas/chats.py +++ b/fastapi_backend/app/schemas/chats.py @@ -193,7 +193,9 @@ class MessageRecord(ResponseModel): """What produced a message, recorded as it was written (docs/assistant.md, "What each answer records"). A question's has `via` and `options`; an answer's the rest. Messages from before it was recorded have none.""" - via: Literal["web", "api", "room", "voice", "automation", "completions"] | None = Field(None, description="a question: how it was sent") + via: Literal["web", "api", "room", "voice", "automation", "completions", "project"] | None = Field( + None, description="a question: how it was sent (project: a brief a project's coordinator wrote)" + ) options: RecordOptions | None = Field(None, description="a question: the options it was asked with") model: str | None = Field(None, description="the model that wrote the answer") provider: str | None = Field(None, description="the host name of the model server it ran on (never its address or key)") diff --git a/fastapi_backend/app/schemas/projects.py b/fastapi_backend/app/schemas/projects.py index a8a92dea..3df10fdd 100644 --- a/fastapi_backend/app/schemas/projects.py +++ b/fastapi_backend/app/schemas/projects.py @@ -62,6 +62,12 @@ class ProjectUpdate(RequestModel): class ThreadCreate(RequestModel): title: str | None = Field(None, max_length=120, description="left out, Lens titles it after its first answer") chat: int | None = Field(None, description="bring in one of your conversations instead of starting a new one") + brief: str | None = Field( + None, + max_length=4000, + description="what to do: asked as the thread's first question and answered in the background, with your tools and " + "approvals (follow it with GET /chats/{id})", + ) class ThreadUpdate(RequestModel): diff --git a/fastapi_backend/tests/api/test_projects.py b/fastapi_backend/tests/api/test_projects.py index 40ef23a4..0fbb5784 100644 --- a/fastapi_backend/tests/api/test_projects.py +++ b/fastapi_backend/tests/api/test_projects.py @@ -194,3 +194,119 @@ def test_agents_use_projects_over_mcp(app, db, cfg, folder, new_client): theirs = _bearer(_grant(cv, hv, scope="read")[1]) assert tool(cv, theirs, "list_projects")["projects"] == [] assert "not found" in tool_error(cv, theirs, "read_project", project_id=pid) + + +def call(i, name, args): + return {"id": f"c{i}", "type": "function", "function": {"name": name, "arguments": json.dumps(args)}} + + +@pytest.fixture +def now(monkeypatch): + """A brief answered where it's asked, not on a thread of its own, so the test sees it end.""" + monkeypatch.setattr(projects, "launch", lambda db, cfg, cid, qid: projects.work(db, cfg, cid, qid)) + + +def _thread(c, h, cid): + return c.get(f"/api/v1/chats/{cid}", headers=h).json() + + +def test_the_coordinator_starts_threads_with_a_brief(app, db, cfg, folder, new_client, llm, now): + s = Assist(app, db, cfg, folder, new_client) + c, h = s.cl["editor"] + p = _new(c, h, instructions="Keep it cheap.") + brief = "Find the cheapest wood for a small treehouse platform." + llm.tool_script = [ + {"content": "", "tool_calls": [call(1, "start_thread", {"title": "Wood", "brief": brief})]}, + {"content": "Pine is cheapest."}, # the thread's answer, written while the coordinator waits + {"content": "I started a thread on wood."}, + ] + ev = sse(c.post(f"/api/v1/chats/{p['coordinator']}/messages", headers=h, json={"content": "Let's plan the build"}).text) + assert [x["tool"] for x in ev["step"]] == ["start_thread"] and ev["step"][0]["summary"] == "Started the thread “Wood”" + assert "approval" not in ev + got = c.get(f"/api/v1/projects/{p['id']}", headers=h).json()["thread_list"] + assert [(t["title"], t["answering"], t["messages"]) for t in got] == [("Wood", False, 2)] + t = _thread(c, h, got[0]["id"]) + q, a = t["messages"] + assert (q["role"], q["content"], a["content"], a["meta"]["outcome"]) == ("user", brief, "Pine is cheapest.", "direct") + # the thread read the project, as a thread; the brief isn't counted as the person's own message + thread_system = next(m["content"] for m in llm.seen[1]["messages"] if m["role"] == "system") + assert "one of its threads, in the project “Treehouse with Sam”" in thread_system and "Keep it cheap." in thread_system + assert db.one("SELECT origin FROM $r", r=projects.R("chat_message", q["id"]))["origin"] == {"project": p["id"], "by": "coordinator"} + + # the coordinator reads threads and marks them done; a thread can't start threads + box = ai_tools.Toolbox( + db, cfg, {"id": t["account"], "email": "ed@x.io"}, {s.pods}, {s.pods}, {"namespaces": ["pods"]}, p["coordinator"] + ) + listed = json.loads(box.call("project_threads", {})[0])["threads"] + assert [(x["thread_id"], x["last"]["text"]) for x in listed] == [(t["id"], "Pine is cheapest.")] + read = json.loads(box.call("read_thread", {"thread_id": t["id"]})[0]) + assert [m["text"] for m in read["messages"]] == [brief, "Pine is cheapest."] + assert json.loads(box.call("finish_thread", {"thread_id": t["id"]})[0])["done"] is True + assert "no thread" in json.loads(box.call("read_thread", {"thread_id": 999999})[0])["error"] + inside = ai_tools.Toolbox(db, cfg, {"id": t["account"], "email": "ed@x.io"}, {s.pods}, {s.pods}, {"namespaces": ["pods"]}, t["id"]) + assert "coordinator" in json.loads(inside.call("start_thread", {"title": "x", "brief": "y"})[0])["error"] + # a conversation outside a project has none of these tools + plain = ai_tools.Toolbox(db, cfg, {"id": t["account"], "email": "ed@x.io"}, {s.pods}, {s.pods}, {}, chat.create(db, t["account"])) + assert not {"project_threads", "start_thread"} & {x["function"]["name"] for x in plain.specs()} + + +def test_starting_threads_after_reading_outside_text_asks_first(app, db, cfg, folder, new_client, llm, now): + s = Assist(app, db, cfg, folder, new_client) + c, h = s.cl["editor"] + p = _new(c, h) + uid = c.get(f"/api/v1/chats/{p['coordinator']}", headers=h).json()["account"] + box = ai_tools.Toolbox( + db, cfg, {"id": uid, "email": "ed@x.io"}, {s.pods}, {s.pods}, {"namespaces": ["pods"]}, p["coordinator"], outside=["page"] + ) + out = json.loads(box.call("start_thread", {"title": "Wood", "brief": "Price pine."})[0]) + assert out["status"] == "waiting for the person's approval" and projects.threads(db, projects.get(db, p["id"], uid)) == [] + llm.tool_script = [{"content": "Pine is $3 a metre."}] + assert c.post(f"/api/v1/approvals/{out['approval']}", headers=h, json={"decision": "approve"}).status_code == 200 + got = projects.threads(db, projects.get(db, p["id"], uid)) + assert [(t["title"], t["last"]["text"]) for t in got] == [("Wood", "Pine is $3 a metre.")] + # one answer starts a few threads at most + box = ai_tools.Toolbox(db, cfg, {"id": uid, "email": "ed@x.io"}, {s.pods}, {s.pods}, {"namespaces": ["pods"]}, p["coordinator"]) + box.started = projects.STARTS_PER_ANSWER + assert "at most" in json.loads(box.call("start_thread", {"title": "More", "brief": "More."})[0])["error"] + + +def test_a_brief_from_the_api_or_an_agent_is_answered_in_the_background(app, db, cfg, folder, new_client, llm, now): + s = Assist(app, db, cfg, folder, new_client) + c, h = s.cl["editor"] + p = _new(c, h) + ca, ha = s.cl["admin"] + assert ca.put("/api/v1/settings/ai", headers=ha, json={"auto_title": True}).status_code == 200 + llm.tool_script = [{"content": "Start with the platform."}] + llm.title = "Where to start" + tid = c.post(f"/api/v1/projects/{p['id']}/threads", headers=h, json={"brief": "Where do we start?"}).json()["id"] + t = _thread(c, h, tid) + assert [m["content"] for m in t["messages"]] == ["Where do we start?", "Start with the platform."] + assert (t["title"], t["title_by"]) == ("Where to start", "assistant") + assert db.one("SELECT origin FROM $r", r=projects.R("chat_message", t["messages"][0]["id"]))["origin"] is None # the person's own + # a brief over the limit, or into a thread still answering, is refused + assert c.post(f"/api/v1/projects/{p['id']}/threads", headers=h, json={"brief": "x" * 4001}).status_code == 422 + db.q("UPDATE $r SET answering = true", r=projects.R("chat", tid)) + assert c.post(f"/api/v1/projects/{p['id']}/threads", headers=h, json={"chat": tid, "brief": "And then?"}).status_code == 400 + + hw = _bearer(_grant(c, h, scope="write")[1]) + llm.tool_script = [{"content": "Buy pine."}] + got = _ok(c, hw, "update_project", {"project_id": p["id"], "new_thread": "Wood", "brief": "What wood?"}) + assert _thread(c, h, got["started"]["chat_id"])["messages"][-1]["content"] == "Buy pine." + assert "new_thread" in _error(c, hw, "update_project", {"project_id": p["id"], "brief": "What wood?"}) + + +def test_a_brief_cut_off_by_a_stop_is_marked_by_the_sweep(app, db, cfg, folder, new_client, llm, monkeypatch): + s = Assist(app, db, cfg, folder, new_client) + c, h = s.cl["editor"] + p = _new(c, h) + monkeypatch.setattr(projects, "launch", lambda db, cfg, cid, qid: None) # the process stopped before it answered + tid = c.post(f"/api/v1/projects/{p['id']}/threads", headers=h, json={"title": "Wood", "brief": "What wood?"}).json()["id"] + assert _thread(c, h, tid)["messages"][-1]["role"] == "user" + projects.sweep(db, cfg) # not yet: it may still be working + assert len(_thread(c, h, tid)["messages"]) == 1 + db.q("UPDATE $r SET working.beat = '2020-01-01T00:00:00+00:00'", r=projects.R("chat", tid)) + projects.sweep(db, cfg) + last = _thread(c, h, tid)["messages"][-1] + assert (last["role"], last["error"]) == ("assistant", projects.STOPPED) + row = db.one("SELECT answering, working FROM $r", r=projects.R("chat", tid)) + assert row["answering"] is False and row.get("working") is None diff --git a/nextjs-frontend/app/openapi-client/sdk.gen.ts b/nextjs-frontend/app/openapi-client/sdk.gen.ts index 4b4e13f4..04e1660d 100644 --- a/nextjs-frontend/app/openapi-client/sdk.gen.ts +++ b/nextjs-frontend/app/openapi-client/sdk.gen.ts @@ -10973,8 +10973,10 @@ export class Projects { /** * Start Thread * - * A new thread: a conversation in the project's namespaces (ask in it with POST /chats/{id}/messages). With `chat`, - * one of your conversations joins the project instead, drawing on its namespaces from then on. + * A new thread: a conversation in the project's namespaces (ask in it with POST /chats/{id}/messages). With a + * `brief`, it is asked there at once and answered in the background, as you, with your tools and approvals: GET + * /chats/{id} shows the answer once it's written (the thread is `answering` until then). With `chat`, one of your + * conversations joins the project instead, drawing on its namespaces from then on (with a brief, asked there next). */ public static startThread( options: Options, diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 1b7b66ec..89feb62c 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -13464,9 +13464,9 @@ export type MessageRecord = { /** * Via * - * a question: how it was sent + * a question: how it was sent (project: a brief a project's coordinator wrote) */ - via?: "web" | "api" | "room" | "voice" | "automation" | "completions" | null; + via?: "web" | "api" | "room" | "voice" | "automation" | "completions" | "project" | null; /** * a question: the options it was asked with */ @@ -25574,6 +25574,12 @@ export type ThreadCreate = { * bring in one of your conversations instead of starting a new one */ chat?: number | null; + /** + * Brief + * + * what to do: asked as the thread's first question and answered in the background, with your tools and approvals (follow it with GET /chats/{id}) + */ + brief?: string | null; }; /** diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 9566df6a..f5ed138c 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -27054,7 +27054,7 @@ "projects" ], "summary": "Start Thread", - "description": "A new thread: a conversation in the project's namespaces (ask in it with POST /chats/{id}/messages). With `chat`,\none of your conversations joins the project instead, drawing on its namespaces from then on.", + "description": "A new thread: a conversation in the project's namespaces (ask in it with POST /chats/{id}/messages). With a\n`brief`, it is asked there at once and answered in the background, as you, with your tools and approvals: GET\n/chats/{id} shows the answer once it's written (the thread is `answering` until then). With `chat`, one of your\nconversations joins the project instead, drawing on its namespaces from then on (with a brief, asked there next).", "operationId": "start_thread", "parameters": [ { @@ -57914,7 +57914,8 @@ "room", "voice", "automation", - "completions" + "completions", + "project" ] }, { @@ -57922,7 +57923,7 @@ } ], "title": "Via", - "description": "a question: how it was sent" + "description": "a question: how it was sent (project: a brief a project's coordinator wrote)" }, "options": { "anyOf": [ @@ -78682,6 +78683,19 @@ ], "title": "Chat", "description": "bring in one of your conversations instead of starting a new one" + }, + "brief": { + "anyOf": [ + { + "type": "string", + "maxLength": 4000 + }, + { + "type": "null" + } + ], + "title": "Brief", + "description": "what to do: asked as the thread's first question and answered in the background, with your tools and approvals (follow it with GET /chats/{id})" } }, "additionalProperties": false, diff --git a/threat-model.json b/threat-model.json index f702bca5..c9113734 100644 --- a/threat-model.json +++ b/threat-model.json @@ -6,7 +6,7 @@ "scope_subpath": null, "description": "Lens is a self-hosted archive for documents, recordings, photos and messages: it reads, transcribes, indexes and publishes them, and people search and chat over them. One operator (the instance admin) runs an instance for their own people: a FastAPI API, a Next.js web app that proxies the API and holds the sign-in session, and background workers, around one SurrealDB database, with content split into namespaces and per-namespace roles. Exposing an instance to the internet behind HTTPS (reverse proxy or Cloudflare tunnel) is supported. Only the instance admin (equivalent to host code execution) and the host operator are trusted; everyone else is trusted only as far as their role.", "confidence": { - "documented": 258, + "documented": 259, "inferred": 99 }, "components": [ diff --git a/threat-model.md b/threat-model.md index 1a2e5785..bd942907 100644 --- a/threat-model.md +++ b/threat-model.md @@ -457,7 +457,7 @@ stored text for the consumer's format. | Report pages | `api/pages.py` | links re-signed on read for every asset of the namespace in the page | that a report link opens only the report | *(inferred, Q10)* | | Assistant answers, summaries, LLM-written notes | Assistant | model output over untrusted content | that it is grounded or free of injected instructions | *(inferred, Q26)* | | Specs from transcripts (a note linked to the asset) | `domain/specs.py` | model output over the transcript, which reaches the model marked as data; web addresses the transcript doesn't hold broken up, images turned into their address, quotes the transcript doesn't hold word for word marked so | that the spec is complete, or that a quote found in the transcript means what the spec says it does | *(documented, docs/notes.md "Specs from transcripts")* | -| Projects (`/projects`, MCP `list_projects`, `read_project`, `create_project`, `update_project`) | `domain/projects.py` | a project's name, goal and instructions, written by its owner, read by the model in every one of its conversations as the owner's words; its threads' titles and last messages, read by the coordinator marked as data; to the owner alone (not on a namespace's own MCP server), its conversations scoped to namespaces the owner can read | that a goal or instructions written by the owner are safe from what the owner pasted into them, or that a thread's text holds no instructions | *(documented, docs/projects.md "marked as data, never as instructions")* | +| Projects (`/projects`, MCP `list_projects`, `read_project`, `create_project`, `update_project`) | `domain/projects.py` | a project's name, goal and instructions, written by its owner, read by the model in every one of its conversations as the owner's words; its threads' titles and messages, read by the coordinator marked as data; briefs answered in the background as the owner with their tools, approvals and budgets, a brief the coordinator writes after reading outside text waiting for the owner's approval; to the owner alone (not on a namespace's own MCP server), its conversations scoped to namespaces the owner can read | that a goal or instructions written by the owner are safe from what the owner pasted into them, or that a thread's text holds no instructions | *(documented, docs/projects.md "marked as data, never as instructions")* | | An answer's record (`meta` on a chat message, `GET /chats/{cid}/messages/{mid}`) | Assistant | the model, the model server's host name (never its address, path, port or key), tool names, scope, counts, tokens, cost and the answer's ledger rows, to the conversation's owner alone | that the model server's host name stays hidden from people who chat | *(documented, docs/assistant.md "never the server's address, path or key")* | | A conversation's export (`GET /chats/{cid}/export.{md,json,pdf}`, MCP `read_chat` and `export_chat`) | Assistant | the person's own conversation, to them alone (the MCP tools aren't on a namespace's own server): questions, answers, each tool call's name, arguments and summary but never its raw result, the answer's record, and the citations they can read now, quoting the asset or web page with a link back; text as the model and the sources wrote it; the PDF prints that Markdown as HTML cleaned of anything that could fetch or run (`convert.clean_html`), by the converters documents use | that a file handed on holds no archive text (its citations quote it), or that its Markdown is safe to render unsanitized | *(documented, docs/assistant.md "never its raw result")* | | MCP tool results | `api/mcp_tools.py` | the person's readable data, as text, with chat-template control tokens broken up and a note when text reads like instructions to an AI | that an asset's text is safe to follow as instructions | *(documented, docs/mcp.md "sees exactly what that person sees")* |