Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
draws on. Its coordinator conversation looks at the project as a whole: it is told about every thread, starts threads
with a brief that the assistant answers there in the background, reads them and marks them done. Each thread is a
conversation about one part of the work, scoped to the project's namespaces. Every answer in a project reads its goal
and instructions. Agents start and change projects over MCP and the API.
([Projects](docs/projects.md); [checks](docs/requirements.md#work-toward-a-goal-in-projects))
and instructions. The Projects page, under Chat, starts threads with a brief and shows which are working. Agents
start and change projects over MCP and the API. ([Projects](docs/projects.md); [checks](docs/requirements.md#work-toward-a-goal-in-projects))
- **Extend the assistant.** Add tools (a prompt, a web request, a graph drawn on the canvas, or sandboxed Python),
skills, hooks and MCP servers, by code, on the canvas or just by asking. Remote MCP servers connect by address, and
ones that run as a program (`npx`, `uvx`) run in a sandboxed container of their own, off until you turn it on. Add
Expand Down
2 changes: 2 additions & 0 deletions changelog.d/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,7 @@
conversations can be brought in or taken out, and deleting a project keeps them. Every answer in a project reads its
goal and instructions. The coordinator starts threads with a brief (`start_thread`), reads them and marks them done;
a brief, from it, you or an agent, is answered in the background as you, with your tools, approvals and budgets.
The Projects page, at the top of Chat's conversation list, starts a project with just a name, starts threads with a
brief on Enter, shows which are working and marks them done in one click.
`/api/v1/projects` and the MCP tools `list_projects`, `read_project`, `create_project` and
`update_project` do it all for agents ([Projects](docs/projects.md)).
18 changes: 15 additions & 3 deletions docs/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ it draws on. Every conversation in it reads all of that, so you don't explain th
Status: **built**: projects with a goal, instructions and one or more namespaces; a coordinator conversation for the
project as a whole, told about every thread, which starts threads with a brief, reads them and marks them done;
threads, each a conversation in the project's namespaces, marked done when finished; briefs answered in the background;
your own conversations brought in or taken out; through the REST API and MCP. **Planned**: scheduled threads, a
Projects page in the web app, chat rooms and email routed to a project, members sharing one, and a project's own
your own conversations brought in or taken out; through the REST API, MCP and the Projects page in the web app.
**Planned**: scheduled threads, chat rooms and email routed to a project, members sharing one, and a project's own
memory.

## What a project is
Expand Down Expand Up @@ -70,6 +70,17 @@ there (the conversation's choice of them) and the same approvals, every model ca
Stop (`POST /api/v1/chats/{id}/stop`) stops it between steps. If Lens stops while a brief is being answered, the thread
says so within a few minutes, so it's never left waiting: ask again.

## In the web app

**Projects** is at the top of the conversation list in Chat (`/projects`). **New project** asks for a name, the goal
and the namespaces, starting from the one picked in the top bar; the instructions are in **Edit** on the project's
page. That page shows the goal, a **Coordinator** button that opens the project's own conversation, a **New thread**
field and the threads, open ones first. Type what a thread should work on and press Enter: it is answered in the
background and marked **Working** until the answer is written. Press Enter with the field empty to open a new thread
and ask in it yourself. The box beside a thread marks it done. **Archive** in the project's menu moves it to the
archived list, with Undo. A conversation in a project has a **Project** link at its top, and one answered in the
background shows "Working on it" until its answer arrives.

## The API

| Call | What it does |
Expand All @@ -92,6 +103,7 @@ same for agents; the last two need a token with the write scope.

- Threads answering their briefs at the same time as each other, with a limit per person.
- Scheduled threads: a prompt asked on a schedule, in a new thread each time or the same one.
- A Projects page in the web app.
- Deleting a project from the web app (the API and MCP can), with Undo.
- A project's threads in the Chat sidebar, grouped under it.
- Chat rooms, Telegram, Slack and email routed to a project.
- Members sharing a project, a project's own memory, and its outputs (notes, files) listed together.
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Every answer in a project reads its goal and instructions; the coordinator is told about the threads, as data | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_every_answer_in_a_project_reads_what_it_is_for` |
| The coordinator starts threads with a brief, reads them and marks them done; a thread can't start threads; it asks first after reading outside text, and starts a few per answer at most | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_the_coordinator_starts_threads_with_a_brief`, `test_starting_threads_after_reading_outside_text_asks_first` |
| A brief from the person, an agent or the coordinator is answered in the background as the person; one cut off by a stopped process is marked, never left waiting | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_a_brief_from_the_api_or_an_agent_is_answered_in_the_background`, `test_a_brief_cut_off_by_a_stop_is_marked_by_the_sweep` |
| The Projects page starts a project with just a name, starts threads with a brief on Enter, shows which are working and marks them done in one click; projects sit under Chat | [projects.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/projects.test.tsx) `starts one in the current namespace with just a name, then opens it`, `starts a thread with a brief on Enter and keeps the page`, `marks a thread done with one click`, `are under Chat in the navigation` |
| Agents list, read, start and change projects over MCP | [test_projects.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_projects.py) `test_agents_use_projects_over_mcp` |

## Extend the assistant
Expand Down
2 changes: 1 addition & 1 deletion fastapi_backend/app/domain/chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ def remembered_tools(db, account):
def get(db, cid, account):
c = db.one(
"SELECT record::id(id) AS id, account, title, title_by, scope, model, kind ?? 'chat' AS kind, head, tools, max_tool_calls, "
"project, done ?? false AS done, created_at, updated_at FROM $r",
"project, done ?? false AS done, answering ?? false AS answering, created_at, updated_at FROM $r",
r=R("chat", cid),
)
if not c or c["account"] != account:
Expand Down
1 change: 1 addition & 0 deletions fastapi_backend/app/schemas/chats.py
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,7 @@ class ChatMemory(ResponseModel):

class Chat(ChatSummary):
account: int
answering: bool = Field(False, description="an answer is being written in it now: streamed, or a brief's in the background")
head: int | None = Field(None, description="the last message of the version showing, which the next question follows")
messages: list[ChatMessage] = Field(default_factory=list, description="the version showing, from the first message to `head`")
tree: list[ChatNode] = Field(default_factory=list, description="every message of every version, oldest first")
Expand Down
146 changes: 146 additions & 0 deletions nextjs-frontend/__tests__/projects.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import "@testing-library/jest-dom";

import { Projects } from "@/app/openapi-client";
import { isActive } from "@/components/app-shell/nav-config";
import { ProjectScreen } from "@/components/projects/project-screen";
import { ProjectsScreen } from "@/components/projects/projects-screen";
import { TooltipProvider } from "@/components/ui/tooltip";

jest.mock("@/app/openapi-client", () => ({
Projects: {
listProjects: jest.fn(),
createProject: jest.fn(),
getProject: jest.fn(),
updateProject: jest.fn(),
startThread: jest.fn(),
updateThread: jest.fn(),
},
}));
jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) }));
const push = jest.fn();
jest.mock("next/navigation", () => ({ useRouter: () => ({ push }) }));
const toast = jest.fn();
jest.mock("@/components/ui/toast", () => ({ useToast: () => toast }));
jest.mock("@/lib/hooks/session", () => ({
useArchive: () => ({
namespace: "home",
namespaces: [
{ id: 1, name: "home" },
{ id: 2, name: "work" },
],
can: () => true,
}),
}));

const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } });
const m = (f: unknown) => f as jest.Mock;
const show = (ui: React.ReactNode) =>
render(
<QueryClientProvider client={new QueryClient({ defaultOptions: { queries: { retry: false } } })}>
<TooltipProvider>{ui}</TooltipProvider>
</QueryClientProvider>,
);

const project = {
id: 4,
name: "Treehouse with Sam",
goal: "Build a treehouse in the oak by summer",
instructions: "",
namespaces: ["home"],
coordinator: 10,
archived: false,
threads: 2,
open: 1,
thread_list: [
{ id: 12, title: "Materials list", done: true, answering: false, last: { role: "assistant", text: "Pine boards" } },
{ id: 11, title: "Find a builder", done: false, answering: true, last: { role: "user", text: "Who builds them?" } },
],
};

beforeEach(() => jest.clearAllMocks());

describe("projects", () => {
it("are under Chat in the navigation", () => {
expect(isActive("/projects", "/chat")).toBe(true);
expect(isActive("/projects/4", "/chat")).toBe(true);
expect(isActive("/projects", "/library")).toBe(false);
});

it("starts one in the current namespace with just a name, then opens it", async () => {
m(Projects.listProjects).mockImplementation(() => ok([]));
m(Projects.createProject).mockImplementation(() => ok({ ...project, thread_list: [] }));
show(<ProjectsScreen />);
fireEvent.click((await screen.findAllByRole("button", { name: "New project" }))[0]);
const name = await screen.findByLabelText("Name");
expect(name).toHaveFocus();
expect(screen.getByRole("checkbox", { name: "home" })).toBeChecked();
fireEvent.change(name, { target: { value: "Treehouse with Sam" } });
fireEvent.keyDown(name, { key: "Enter" });
await waitFor(() => expect(push).toHaveBeenCalledWith("/projects/4"));
expect(m(Projects.createProject).mock.calls[0][0].body).toEqual({
name: "Treehouse with Sam",
goal: "",
namespaces: ["home"],
});
});

it("lists them with how many threads are open", async () => {
m(Projects.listProjects).mockImplementation(() => ok([project]));
show(<ProjectsScreen />);
expect(await screen.findByText("Treehouse with Sam")).toBeInTheDocument();
expect(screen.getByText("1 open of 2")).toBeInTheDocument();
expect(screen.getByRole("link", { name: /Treehouse with Sam/ })).toHaveAttribute("href", "/projects/4");
});
});

describe("a project", () => {
it("shows its goal, coordinator and threads, open ones first, with the ones being answered marked", async () => {
m(Projects.getProject).mockImplementation(() => ok(project));
show(<ProjectScreen id={4} />);
expect(await screen.findByText("Build a treehouse in the oak by summer")).toBeInTheDocument();
expect(screen.getByRole("link", { name: /Coordinator/ })).toHaveAttribute("href", "/chat/10");
const links = screen.getAllByRole("link").filter((a) => a.getAttribute("href")?.match(/^\/chat\/1[12]$/));
expect(links.map((a) => a.getAttribute("href"))).toEqual(["/chat/11", "/chat/12"]);
expect(screen.getByText("Working")).toBeInTheDocument();
});

it("starts a thread with a brief on Enter and keeps the page", async () => {
m(Projects.getProject).mockImplementation(() => ok(project));
m(Projects.startThread).mockImplementation(() => ok({ id: 13 }));
show(<ProjectScreen id={4} />);
const brief = await screen.findByLabelText("New thread");
fireEvent.change(brief, { target: { value: "Price the timber" } });
fireEvent.keyDown(brief, { key: "Enter" });
await waitFor(() => expect(toast).toHaveBeenCalled());
expect(m(Projects.startThread).mock.calls[0][0]).toMatchObject({
path: { pid: 4 },
body: { brief: "Price the timber" },
});
expect(push).not.toHaveBeenCalled();
toast.mock.calls[0][0].action.onClick();
expect(push).toHaveBeenCalledWith("/chat/13");
});

it("opens an empty thread when there's no brief", async () => {
m(Projects.getProject).mockImplementation(() => ok(project));
m(Projects.startThread).mockImplementation(() => ok({ id: 14 }));
show(<ProjectScreen id={4} />);
fireEvent.click(await screen.findByRole("button", { name: "Start thread" }));
await waitFor(() => expect(push).toHaveBeenCalledWith("/chat/14"));
expect(m(Projects.startThread).mock.calls[0][0].body).toEqual({});
});

it("marks a thread done with one click", async () => {
m(Projects.getProject).mockImplementation(() => ok(project));
m(Projects.updateThread).mockImplementation(() => ok({ ok: true }));
show(<ProjectScreen id={4} />);
fireEvent.click(await screen.findByRole("checkbox", { name: "Mark “Find a builder” done" }));
await waitFor(() => expect(Projects.updateThread).toHaveBeenCalled());
expect(m(Projects.updateThread).mock.calls[0][0]).toMatchObject({
path: { pid: 4, cid: 11 },
body: { done: true },
});
});
});
14 changes: 14 additions & 0 deletions nextjs-frontend/app/(app)/projects/[id]/page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import type { Metadata } from "next";
import { notFound } from "next/navigation";

import { ProjectScreen } from "@/components/projects/project-screen";

export const metadata: Metadata = { title: "Project" };

/** A project: its goal, its coordinator, and its threads with what each is doing. */
export default async function Page({ params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
const pid = Number(id);
if (!Number.isInteger(pid) || pid <= 0) notFound();
return <ProjectScreen key={pid} id={pid} />;
}
10 changes: 10 additions & 0 deletions nextjs-frontend/app/(app)/projects/page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import type { Metadata } from "next";

import { ProjectsScreen } from "@/components/projects/projects-screen";

export const metadata: Metadata = { title: "Projects" };

/** Goals worked toward in threads, each with a coordinator (backend: app/domain/projects.py). */
export default function ProjectsPage() {
return <ProjectsScreen />;
}
6 changes: 6 additions & 0 deletions nextjs-frontend/app/openapi-client/types.gen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3449,6 +3449,12 @@ export type Chat = {
* Account
*/
account: number;
/**
* Answering
*
* an answer is being written in it now: streamed, or a brief's in the background
*/
answering?: boolean;
/**
* Head
*
Expand Down
2 changes: 2 additions & 0 deletions nextjs-frontend/components/app-shell/nav-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ export function isActive(pathname: string, href: string): boolean {
if (href === "/sensors" && (pathname === "/sources" || pathname.startsWith("/sources/"))) return true;
// Members opening /sensors see their Sources page.
if (href === "/sources" && pathname === "/sensors") return true;
// Projects are conversations working toward a goal, under Chat.
if (href === "/chat" && (pathname === "/projects" || pathname.startsWith("/projects/"))) return true;
// An entity's page is under /wiki, by name.
if (href === "/entities" && pathname.startsWith("/wiki/")) return true;
// Schedules is a tab of Automations, and the Routines page (its list, proposed graph changes, history) opens from it.
Expand Down
24 changes: 20 additions & 4 deletions nextjs-frontend/components/chat/chat-app.tsx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"use client";

import { useQueryClient } from "@tanstack/react-query";
import { ChevronLeft, Clock, LibraryBig, Plus, X } from "lucide-react";
import { ChevronLeft, Clock, FolderKanban, LibraryBig, Loader2, Plus, X } from "lucide-react";
import { useSession } from "next-auth/react";
import Link from "next/link";
import { useParams, useRouter, useSearchParams } from "next/navigation";
Expand Down Expand Up @@ -99,7 +99,6 @@ export function ChatApp() {
const activeId = params?.id && /^\d+$/.test(params.id) ? Number(params.id) : null;

const chats = useChats();
const chat = useChat(activeId);
const approvals = useApprovals(activeId);
const llm = useLlmStatus();
const index = useRecordingIndex();
Expand All @@ -121,6 +120,8 @@ export function ChatApp() {
),
);
const [live, setLive] = useState<Live | null>(null);
// an answer written elsewhere (a project's brief, in the background) shows here once it's saved
const chat = useChat(activeId, { poll: !live || live.chatId !== activeId });
const [extras, setExtras] = useState<Record<number, Extra>>({});
const [partials, setPartials] = useState<Record<string, TurnState>>({});
const [checks, setChecks] = useState<Record<number, AnswerCheck>>({});
Expand Down Expand Up @@ -715,6 +716,15 @@ export function ChatApp() {
<ChevronLeft />
</IconButton>
<span className="min-w-0 flex-1 truncate text-[15.5px] font-bold text-fg">{title}</span>
{chat.data?.project != null && (
<Link
href={`/projects/${chat.data.project}`}
className="inline-flex shrink-0 items-center gap-1.5 rounded-full px-2.5 py-1 text-[13px] text-fg-secondary hover:bg-surface-neutral print:hidden"
>
<FolderKanban aria-hidden className="size-3.5" />
Project
</Link>
)}
{activeId != null && (
<ExportMenu
label="Export conversation"
Expand Down Expand Up @@ -870,11 +880,17 @@ export function ChatApp() {
/>
) : (
it.q &&
!liveHere && (
!liveHere &&
(chat.data?.answering && it === items[items.length - 1] ? (
<p role="status" className="m-0 flex items-center gap-2 text-[13px] text-fg-secondary">
<Loader2 aria-hidden className="size-3.5 animate-spin" />
Working on it. The answer shows here once it’s written.
</p>
) : (
<p className="m-0 text-[13px] text-fg-muted">
No answer was saved for this question. Ask it again to get one.
</p>
)
))
)}
{memory && (it.a ? memory.upto === it.a.id : memory.upto === it.q?.id) && (
<SummaryMark summary={memory} />
Expand Down
9 changes: 8 additions & 1 deletion nextjs-frontend/components/chat/conversations.tsx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"use client";

import { useMutation, useQueryClient } from "@tanstack/react-query";
import { Ellipsis, Pencil, Plus, Trash2 } from "lucide-react";
import { Ellipsis, FolderKanban, Pencil, Plus, Trash2 } from "lucide-react";
import Link from "next/link";
import { useRouter } from "next/navigation";
import { useState } from "react";
Expand Down Expand Up @@ -81,6 +81,13 @@ export function ConversationList({
<Button variant="secondary" size="sm" icon={<Plus />} className="w-full" onClick={onNew}>
New conversation
</Button>
<Link
href="/projects"
className="mt-1.5 flex items-center gap-2 rounded-sm px-2.5 py-[7px] text-[13.5px] text-fg-secondary hover:bg-surface-neutral hover:text-fg"
>
<FolderKanban aria-hidden className="size-4" />
Projects
</Link>
</div>
{loading && (
<div className="flex flex-col gap-3 px-2.5 py-2" aria-busy="true" aria-label="Loading conversations">
Expand Down
Loading
Loading