Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
instead of deleting what followed, regenerating an answer (`regenerate`, with the same model or another) writes one
beside it, and a question can go on from any message (`parent`). `GET /api/v1/chats/<id>` shows each message's
versions and every version's messages, and `PUT /api/v1/chats/<id>/head` switches between them; the model reads
only the version showing. The web app's switcher and Regenerate button are in progress. Existing conversations are
upgraded on the next start (`chat-tree`) ([Versions](docs/assistant.md)).
only the version showing. In the web app, Regenerate (and Regenerate with… another model) replaces Try another
model, and `‹ 2 / 3 ›` under a question or an answer moves between its versions. Existing conversations are upgraded
on the next start (`chat-tree`) ([Versions](docs/assistant.md)).
- **Add an MCP server with a form, or paste its config.** Extensions → Add MCP server (admins) takes an address and
its headers, or a command with its arguments and environment, or reads a config pasted from Claude (`.mcp.json`) or
VS Code, one server or many; Test lists its tools before it's saved. Tokens and keys in headers and env are now
Expand Down
15 changes: 9 additions & 6 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,17 @@ then.
others in its place, oldest first), every version's messages as `tree`, and `head`, the last message showing. `PUT
/api/v1/chats/<id>/head` with `{"message": <id>}` shows the version that message is in, to its newest message. A
new version streams `branch` first (`message`, `parent`, `versions`). Chat rooms ([Chat rooms](chat-rooms.md)) and
automations go on from the version showing. In the web app, Edit makes a version; switching between versions and
Regenerate are in progress.
automations go on from the version showing. In the web app, Edit on a question and Regenerate on an answer (its menu:
Regenerate with… another model) make a version, which streams in place; `‹ 2 / 3 ›` under a question or an answer
with versions moves between them (the arrow keys too, while it has focus), showing what followed each, and the next
question goes on from the version showing. Approvals proposed in one version show in that version.
- **Long conversations:** the model reads the latest six messages word for word. Once eight more have built up
behind them, they're folded into a running summary kept on the conversation (one model call, after an answer), and
every answer gets that summary plus the messages it doesn't cover yet. Room conversations (chat-rooms.md) go on for
good, so they rely on it most. A summary covers the version it was written on: on a version that leaves what it
covers, the model reads that version's messages, and the next summary is made from them. A conversation that fills half of the model's context window (**Chat model's context**, `ai.context_tokens`,
16000 when empty; about four characters a token) is folded in early, before the next answer, whatever the count.
covers, the model reads that version's messages, and the next summary is made from them. A conversation that fills
half of the model's context window (**Chat model's context**, `ai.context_tokens`, 16000 when empty; about four
characters a token) is folded in early, before the next answer, whatever the count.
**Model for summaries** (`ai.compact_model`) writes them with a smaller, cheaper model on the same provider; empty,
the chat's model does. Settings › AI assistant › Summarise long conversations (`ai.compact`, on) turns it off; off,
or with no model, only the last six messages are read.
Expand All @@ -45,8 +48,8 @@ then.
reply, is read by the model afterwards.
- **Changed access:** old citations are filtered by the person's current access when a conversation is reopened.
- **Choosing the model:** a conversation can use any model an admin offers (`llm.chat_models`, else whatever the model
server lists); Try another model asks a question again with a different one. Each answer records the model that
wrote it.
server lists); Regenerate with… writes a new answer with a different one, beside the old. Each answer records the
model that wrote it.
- **Chat on any page:** every page except Chat itself has an Ask button that opens a chat panel beside the page (full
screen on phones). Its conversation follows you between pages, and across visits in the same browser, until New
chat; it is an ordinary conversation, also listed in Chat. Each question can share the page's text (the page chip,
Expand Down
141 changes: 141 additions & 0 deletions nextjs-frontend/__tests__/chat-versions.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import "@testing-library/jest-dom";

import type { Chat, ChatMessage, ChatNode } from "@/app/openapi-client/types.gen";
import { Answer } from "@/components/chat/answer";
import { lineage, newestBelow, switched, versionsOf, VersionSwitch } from "@/components/chat/versions";
import { TooltipProvider } from "@/components/ui/tooltip";

jest.mock("@/app/openapi-client", () => ({ Chats: { checkMessage: jest.fn() }, Search: {} }));
jest.mock("@/lib/api/browser", () => ({ useApiClient: () => ({}), data: jest.fn() }));
jest.mock("@/lib/hooks/session", () => ({
useArchive: () => ({ can: () => true, namespaces: [], roleIn: () => "editor" }),
}));

// q1 → a1 → q2 → a2, a1 regenerated as b1 (→ q3 → a3), and q1 edited as e1 (→ f1)
const node = (id: number, parent: number | null, role: "user" | "assistant"): ChatNode => ({ id, parent, role });
const TREE = [
node(1, null, "user"),
node(2, 1, "assistant"),
node(3, 2, "user"),
node(4, 3, "assistant"),
node(5, 1, "assistant"),
node(6, 5, "user"),
node(7, 6, "assistant"),
node(8, null, "user"),
node(9, 8, "assistant"),
];
const said = (id: number, role: "user" | "assistant" = id % 2 ? "user" : "assistant"): ChatMessage => ({
id,
role,
content: `m${id}`,
});

describe("versions of a conversation", () => {
it("groups versions by what they follow and who said them, oldest first", () => {
const v = versionsOf(TREE);
expect(v.get(5)).toEqual({ index: 2, count: 2, ids: [2, 5] });
expect(v.get(8)).toEqual({ index: 2, count: 2, ids: [1, 8] });
expect(v.get(3)).toEqual({ index: 1, count: 1, ids: [3] });
});

it("shows a version to its newest message", () => {
expect(newestBelow(TREE, 2)).toBe(4);
expect(newestBelow(TREE, 1)).toBe(7);
expect(newestBelow(TREE, 9)).toBe(9);
expect(lineage(TREE, 7)).toEqual([1, 5, 6, 7]);
expect(lineage(TREE, 99)).toEqual([]);
});

it("switches at once to a version already seen, and waits for one that isn't", () => {
const chat = {
id: 1,
title: "t",
account: 1,
head: 4,
tree: TREE,
messages: [1, 2, 3, 4].map((i) => said(i)),
} as Chat;
const seen = new Map([1, 2, 3, 4, 5].map((i) => [i, said(i, i === 5 ? "assistant" : undefined)]));
expect(switched(chat, 5, seen)).toBeNull(); // 6 and 7 haven't been loaded
for (const i of [6, 7]) seen.set(i, said(i));
const out = switched(chat, 5, seen)!;
expect(out.head).toBe(7);
expect(out.messages!.map((m) => m.id)).toEqual([1, 5, 6, 7]);
expect(out.messages![1].versions).toEqual({ index: 2, count: 2, ids: [2, 5] });
});
});

describe("the version switch", () => {
const wrap = (ui: React.ReactNode) => render(<TooltipProvider>{ui}</TooltipProvider>);

it("moves between versions with its arrows and the arrow keys, and stops at either end", () => {
const onPick = jest.fn();
wrap(<VersionSwitch versions={{ index: 2, count: 3, ids: [2, 5, 10] }} what="answer" onPick={onPick} />);
expect(screen.getByRole("group", { name: "Versions of this answer" })).toHaveTextContent("2 / 3");
fireEvent.click(screen.getByRole("button", { name: "Previous version of this answer" }));
fireEvent.click(screen.getByRole("button", { name: "Next version of this answer" }));
fireEvent.keyDown(screen.getByRole("button", { name: "Next version of this answer" }), { key: "ArrowLeft" });
expect(onPick.mock.calls).toEqual([[2], [10], [2]]);
});

it("doesn't go past the first or last", () => {
const onPick = jest.fn();
wrap(<VersionSwitch versions={{ index: 1, count: 2, ids: [1, 8] }} what="question" onPick={onPick} />);
const back = screen.getByRole("button", { name: "Previous version of this question" });
expect(back).toHaveAttribute("aria-disabled", "true");
fireEvent.click(back);
fireEvent.keyDown(back, { key: "ArrowLeft" });
expect(onPick).not.toHaveBeenCalled();
fireEvent.keyDown(back, { key: "ArrowRight" });
expect(onPick).toHaveBeenCalledWith(8);
});

it("shows nothing for a message with one version", () => {
const { container } = wrap(<VersionSwitch versions={{ index: 1, count: 1, ids: [3] }} what="answer" />);
expect(container).toBeEmptyDOMElement();
});
});

describe("regenerating an answer", () => {
const answer = (onRegenerate?: (m?: string) => void) =>
render(
<QueryClientProvider client={new QueryClient()}>
<TooltipProvider>
<Answer
chatId={1}
messageId={2}
question="When does it ship?"
text="On Friday."
passages={[]}
status="done"
steps={[{ tool: "search_transcripts", args: {}, summary: "Searched" }]}
scope={{}}
model="fake"
models={["fake", "fake-large"]}
onRegenerate={onRegenerate}
hover={null}
onHover={() => {}}
onPreview={() => {}}
/>
</TooltipProvider>
</QueryClientProvider>,
);

it("writes a new answer with the same model, or another one", async () => {
const onRegenerate = jest.fn();
answer(onRegenerate);
fireEvent.click(screen.getByRole("button", { name: "Regenerate" }));
expect(onRegenerate).toHaveBeenLastCalledWith();
fireEvent.keyDown(screen.getByRole("button", { name: "Regenerate with another model" }), { key: "Enter" });
expect(screen.queryByRole("menuitem", { name: "fake" })).not.toBeInTheDocument(); // it wrote this one
fireEvent.click(await screen.findByRole("menuitem", { name: "fake-large" }));
await waitFor(() => expect(onRegenerate).toHaveBeenLastCalledWith("fake-large")); // a menu acts once it has closed
});

it("isn't offered without a handler, e.g. while another answer is written", () => {
answer();
expect(screen.queryByRole("button", { name: "Regenerate" })).not.toBeInTheDocument();
});
});
54 changes: 32 additions & 22 deletions nextjs-frontend/components/chat/answer.tsx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"use client";

import { useMutation, useQuery } from "@tanstack/react-query";
import { ChevronDown, RotateCcw, ShieldCheck, Sparkles, Square } from "lucide-react";
import { ChevronDown, RefreshCw, RotateCcw, ShieldCheck, Square } from "lucide-react";
import Link from "next/link";
import { useMemo } from "react";

Expand All @@ -22,7 +22,7 @@ import { RichText } from "@/components/chat/rich-text";
import { scopeWords, type Scope } from "@/components/chat/scope";
import { ToolSteps } from "@/components/chat/steps";
import type { ToolStep, TurnStatus } from "@/components/chat/stream";
import { Button } from "@/components/ui/button";
import { Button, IconButton } from "@/components/ui/button";
import { Menu, MenuContent, MenuItem, MenuLabel, MenuTrigger } from "@/components/ui/menu";
import { data, useApiClient } from "@/lib/api/browser";
import { plural } from "@/lib/format";
Expand Down Expand Up @@ -137,9 +137,12 @@ export type AnswerProps = {
/** Stop was pressed: the answer ends after the piece or step it's on. */
stopping?: boolean;
onRetry?: () => void;
/** Ask the same question again with another of these models (the one that wrote this answer is left out). */
/** Write a new answer to the same question, as another version beside this one: with this answer's model, or with
* another of `models` (the one that wrote this answer is left out). */
models?: string[];
onRetryWith?: (model: string) => void;
onRegenerate?: (model?: string) => void;
/** The switch between this answer's versions (VersionSwitch), shown with its actions. */
versions?: React.ReactNode;
onAddScope?: (ns: string) => void;
onShowSources?: () => void;
active?: boolean;
Expand Down Expand Up @@ -178,6 +181,7 @@ export function Answer(p: AnswerProps) {
const result = p.check ?? check.data ?? null;
const unsupported = (result?.verdicts ?? []).filter((v) => v.supported === false).map((v) => String(v.claim ?? ""));
const from = fromWhere(p.passages);
const others = (p.models ?? []).filter((m) => m !== p.model);

return (
<article aria-label="Answer" aria-busy={streaming || undefined} className="flex max-w-[720px] flex-col gap-3">
Expand Down Expand Up @@ -306,24 +310,30 @@ export function Answer(p: AnswerProps) {
{p.stopping ? "Stopping…" : "Stop"}
</Button>
)}
{!streaming && p.onRetryWith && (p.models ?? []).filter((m) => m !== p.model).length > 0 && (
<Menu>
<MenuTrigger asChild>
<Button size="sm" variant="ghost" icon={<Sparkles />}>
Try another model <ChevronDown />
</Button>
</MenuTrigger>
<MenuContent align="start" className="max-h-[320px] min-w-[220px] overflow-y-auto">
<MenuLabel>Ask again with</MenuLabel>
{(p.models ?? [])
.filter((m) => m !== p.model)
.map((m) => (
<MenuItem key={m} onSelect={() => p.onRetryWith?.(m)}>
<span className="font-mono text-[12.5px]">{m}</span>
</MenuItem>
))}
</MenuContent>
</Menu>
{p.versions}
{!streaming && p.onRegenerate && (
<span className="inline-flex items-center">
<IconButton label="Regenerate" size={32} onClick={() => p.onRegenerate?.()} className="[&_svg]:size-4">
<RefreshCw />
</IconButton>
{others.length > 0 && (
<Menu>
<MenuTrigger asChild>
<IconButton label="Regenerate with another model" size={24} className="[&_svg]:size-3.5">
<ChevronDown />
</IconButton>
</MenuTrigger>
<MenuContent align="start" className="max-h-[320px] min-w-[220px] overflow-y-auto">
<MenuLabel>Regenerate with…</MenuLabel>
{others.map((m) => (
<MenuItem key={m} onSelect={() => p.onRegenerate?.(m)}>
<span className="font-mono text-[12.5px]">{m}</span>
</MenuItem>
))}
</MenuContent>
</Menu>
)}
</span>
)}
{p.status === "done" && p.messageId != null && p.chatId != null && cites.length > 0 && !noModel && !result && (
<Button
Expand Down
Loading
Loading