Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
`ABOUT`, `NARROWER` and `RELATED`; TERM entities are now labelled `Term` there) and in RDF (`/id/topic/<id>`). See
docs/topics.md.

- **A Topics page.** Topics in the navigation shows a namespace's vocabulary as a tree of broader and narrower topics,
with search, and a drawer to edit, merge or delete a topic and see the recordings about it. A recording's Entities
tab lists its topics for editors to add, remove or accept; a Term entity can be made a topic from its drawer; topics
show on the graph canvas as green tags. Entities of type TERM are now shown as Terms.

- **Query the graph in Cypher, and walk it.** The archive is now a property graph of namespaces, collections,
recordings, speakers and entities. `POST /api/v1/graph/query` runs read-only Cypher (the language of Neo4j and ISO
GQL) over the namespaces you can read; `/graph/related` gives a node's parents, children, ancestors, descendants or
Expand Down
2 changes: 1 addition & 1 deletion DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Todo:
- [x] Vocabulary in the backend: topics with labels, definition, broader/narrower/related; recordings about topics;
create, edit, merge, delete; turn a TERM entity into a topic (and back, by deleting the topic); API with viewer
reads and editor changes; topics in the property graph (Topic, ABOUT, NARROWER, RELATED) and in RDF as SKOS
- [ ] Topics page: the tree of broader and narrower topics, a topic's recordings, edit and merge; a recording's topics
- [x] Topics page: the tree of broader and narrower topics, a topic's recordings, edit and merge; a recording's topics
- [ ] Analysis suggests topics (summary topics and keywords matched to the vocabulary; new ones as suggestions to
accept); a namespace can keep its vocabulary fixed or open, as entities do
- [ ] # links in notes point at topics; assistant and MCP tools for topics
Expand Down
18 changes: 16 additions & 2 deletions docs/topics.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,22 @@ entities. Entities are the named things a transcript mentions (people, organisat
subject someone chose for the vocabulary, like "Gene therapy" or "Funding".

Status: **built**: the vocabulary (labels, definitions, broader, narrower and related topics), recordings about
topics, topics made from topic-like entities, the API, topics in the graph and in RDF. **Planned**: a Topics page,
topic suggestions from analysis, `#` links in notes pointing at topics, and assistant and MCP tools for topics.
topics, topics made from topic-like entities, the API, the Topics page, a recording's topics, topics in the graph
(and on its canvas) and in RDF. **Planned**: topic suggestions from analysis, `#` links in notes pointing at topics,
and assistant and MCP tools for topics.

## In the web app

**Topics** in the navigation shows a namespace's vocabulary as a tree: top topics first, each opening onto its
narrower ones (a topic under two broader topics shows under both). Search finds topics by any of their labels. Pick
a topic to edit it in a drawer: its labels, definition, broader and related topics, its narrower topics, and the
recordings about it, where suggested ones can be accepted or removed. The drawer also merges topics and deletes one.
Editors add topics at the top or under another with **+**.

A recording's **Entities** tab starts with its topics: editors add one from the vocabulary, take one off, or accept
or dismiss a suggestion. An entity of type Term has **Make a topic** in its drawer. On the graph canvas topics are
green tags, with About, Narrower and Related edges; ask the graph (for example `MATCH (t:Topic) RETURN t`) or open a
recording's children to bring them in.

## A topic

Expand Down
2 changes: 1 addition & 1 deletion fastapi_backend/app/domain/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"PLACE": "Place",
"EVENT": "Event",
"WORK": "Work",
"TERM": "Topic",
"TERM": "Term",
"DATE": "Date",
"NUMBER": "Number",
}
Expand Down
2 changes: 1 addition & 1 deletion fastapi_backend/app/domain/entity_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"PLACE": "A country, city, building or other place.",
"EVENT": "A named event: a conference, a launch, a war.",
"WORK": "A book, paper, film, law or other named work.",
"TERM": "A topic or term that isn't one of the other types.",
"TERM": "A term or theme that isn't one of the other types (the vocabulary of topics is kept apart, under Topics).",
"DATE": "A date or time.",
"NUMBER": "An amount, percentage or other number.",
}
Expand Down
61 changes: 61 additions & 0 deletions nextjs-frontend/__tests__/topics-model.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import { broaderChoices, findTopics, splitLabels, topicTree, type TopicItem } from "@/components/topics/model";

const t = (id: number, label: string, broader: number[] = [], alt: string[] = []): TopicItem => ({
id,
label,
alt,
broader,
related: [],
recordings: 0,
narrower: 0,
});

// Biology > Gene therapy > Viral vectors; Medicine > Gene therapy (two broader topics); Funding at the top
const items = [
t(1, "Biology"),
t(2, "Gene therapy", [1, 4], ["GT"]),
t(3, "Viral vectors", [2]),
t(4, "Medicine"),
t(5, "funding"),
];

describe("topic tree", () => {
it("shows top topics by label, closed", () => {
expect(topicTree(items, new Set()).map((r) => r.topic.label)).toEqual(["Biology", "funding", "Medicine"]);
});

it("opens one path at a time, a topic under each of its broader topics", () => {
const rows = topicTree(items, new Set(["1", "1/2"]));
expect(rows.map((r) => [r.topic.label, r.depth, r.path])).toEqual([
["Biology", 0, "1"],
["Gene therapy", 1, "1/2"],
["Viral vectors", 2, "1/2/3"],
["funding", 0, "5"],
["Medicine", 0, "4"],
]);
expect(topicTree(items, new Set(["4"])).map((r) => r.path)).toEqual(["1", "5", "4", "4/2"]);
});

it("cuts cycles and treats unknown broader topics as top", () => {
const loop = [t(1, "A", [2]), t(2, "B", [1]), t(3, "C", [99])];
expect(topicTree(loop, new Set(["1"])).map((r) => r.topic.label)).toEqual(["C"]);
});
});

describe("topic helpers", () => {
it("finds by label and other labels, prefix first", () => {
expect(findTopics(items, "gt").map((x) => x.label)).toEqual(["Gene therapy"]);
expect(findTopics(items, "me").map((x) => x.label)).toEqual(["Medicine"]);
expect(findTopics(items, "v").map((x) => x.label)).toEqual(["Viral vectors"]);
expect(findTopics(items, "i").map((x) => x.label)).toEqual(["Biology", "funding", "Medicine", "Viral vectors"]);
});

it("offers as broader only topics that aren't below it", () => {
expect(broaderChoices(items, 1).map((x) => x.id)).toEqual([5, 4]);
expect(broaderChoices(items, 3).map((x) => x.id)).toEqual([1, 5, 2, 4]);
});

it("splits other labels", () => {
expect(splitLabels(" GT, gene therapies ,gt,, ")).toEqual(["GT", "gene therapies"]);
});
});
8 changes: 8 additions & 0 deletions nextjs-frontend/app/(app)/topics/[id]/page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import { notFound, redirect } from "next/navigation";

/** A topic opens in the Topics page, in its own namespace (the address RDF sends browsers to). */
export default async function TopicPage({ params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
if (!/^\d+$/.test(id)) notFound();
redirect(`/topics?topic=${id}`);
}
15 changes: 15 additions & 0 deletions nextjs-frontend/app/(app)/topics/page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import type { Metadata } from "next";
import { Suspense } from "react";

import { TopicsPage } from "@/components/topics/topics-page";

export const metadata: Metadata = { title: "Topics" };

/** A namespace's vocabulary of topics: what its recordings are about. */
export default function Page() {
return (
<Suspense>
<TopicsPage />
</Suspense>
);
}
4 changes: 3 additions & 1 deletion nextjs-frontend/components/app-shell/nav-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,15 @@ import {
HardDriveDownload,
House,
LibraryBig,
type LucideIcon,
MessagesSquare,
NotebookPen,
RadioTower,
Search,
Settings,
Shapes,
Shield,
Tags,
type LucideIcon,
Waypoints,
Workflow,
} from "lucide-react";
Expand Down Expand Up @@ -44,6 +45,7 @@ export const NAV: NavItem[] = [
{ label: "Chat", href: "/chat", icon: MessagesSquare },
{ label: "Notes", href: "/notes", icon: NotebookPen },
{ label: "Entities", href: "/entities", icon: Shapes },
{ label: "Topics", href: "/topics", icon: Tags },
{ label: "Graph", href: "/graph", icon: Waypoints },
{
label: "Reports",
Expand Down
33 changes: 31 additions & 2 deletions nextjs-frontend/components/entities/entity-drawer.tsx
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
"use client";

import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Eye, EyeOff, NotebookPen, Waypoints } from "lucide-react";
import { Eye, EyeOff, NotebookPen, Tags, Waypoints } from "lucide-react";
import Link from "next/link";
import { useRouter } from "next/navigation";
import { useEffect, useState } from "react";

import { Entities } from "@/app/openapi-client";
import { Entities, Topics } from "@/app/openapi-client";
import type { EntityType } from "@/app/openapi-client/types.gen";
import { recordingHref } from "@/components/search/links";
import { Banner } from "@/components/ui/banner";
Expand Down Expand Up @@ -107,6 +108,21 @@ export function EntityDrawer({
},
onError: fail("Couldn’t delete it"),
});
const router = useRouter();
const toTopic = useMutation({
mutationFn: () => data(Topics.entityToTopic({ client, path: { eid: id } })),
onSuccess: (t) => {
refresh();
qc.invalidateQueries({ queryKey: ["topics"] });
toast({
title: `${t.label} is a topic now`,
body: "The entity is hidden while the topic exists.",
tone: "green",
});
router.push(`/topics/${t.id}`);
},
onError: fail("Couldn’t make it a topic"),
});
const hide = useMutation({
mutationFn: (hidden: boolean) => data(Entities.hideEntity({ client, path: { eid: id }, body: { hidden } })),
onSuccess: (_, hidden) => {
Expand Down Expand Up @@ -249,6 +265,19 @@ export function EntityDrawer({
>
{d.hidden ? "Show again" : "Hide"}
</Button>
{d.type === "TERM" && !builtin && !d.hidden && (
<Button
type="button"
size="sm"
variant="ghost"
icon={<Tags />}
disabled={!canEdit || toTopic.isPending}
disabledReason={!canEdit ? needRole("editor", ns) : undefined}
onClick={() => toTopic.mutate()}
>
Make a topic
</Button>
)}
{d.defined && !d.mentions && (
<Button
type="button"
Expand Down
3 changes: 2 additions & 1 deletion nextjs-frontend/components/graph/filters.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,8 @@ export function GraphFilters({
(k) =>
k.key !== "same thing" &&
(!["contains", "speaks in", "mentioned in"].includes(k.key) ||
["recording", "collection", "namespace"].some((g) => present.has(g))),
["recording", "collection", "namespace"].some((g) => present.has(g))) &&
(!["about", "narrower", "related"].includes(k.key) || present.has("topic")),
).map((k) => (
<Checkbox
key={k.key}
Expand Down
30 changes: 24 additions & 6 deletions nextjs-frontend/components/graph/model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* server's layout, and typed, weighted edges. Filtering, sizing, keyboard movement and the text summary live here.
*/

export type NodeKind = "speaker" | "entity" | "recording" | "collection" | "namespace";
export type NodeKind = "speaker" | "entity" | "recording" | "collection" | "namespace" | "topic";

export type GraphNode = {
id: string;
Expand Down Expand Up @@ -34,19 +34,31 @@ export const NODE_GROUPS: { key: string; label: string; shape: Shape }[] = [
{ key: "ORG", label: "Organisations", shape: "square" },
{ key: "PRODUCT", label: "Products", shape: "diamond" },
{ key: "PLACE", label: "Places", shape: "triangle" },
{ key: "TERM", label: "Topics", shape: "pill" },
{ key: "TERM", label: "Terms", shape: "pill" },
{ key: "EVENT", label: "Events", shape: "hexagon" },
{ key: "WORK", label: "Works", shape: "rounded" },
{ key: "recording", label: "Recordings", shape: "doc" },
{ key: "collection", label: "Collections", shape: "folder" },
{ key: "namespace", label: "Namespaces", shape: "octagon" },
{ key: "topic", label: "Topics", shape: "tag" },
];

export type Shape =
"circle" | "ring" | "square" | "diamond" | "triangle" | "pill" | "hexagon" | "rounded" | "doc" | "folder" | "octagon";
| "circle"
| "ring"
| "square"
| "diamond"
| "triangle"
| "pill"
| "hexagon"
| "rounded"
| "doc"
| "folder"
| "octagon"
| "tag";

/** Node kinds that only appear once you explore (a node's parents, ancestors…), never in the overview. */
export const STRUCTURE_KINDS = new Set<NodeKind>(["recording", "collection", "namespace"]);
export const STRUCTURE_KINDS = new Set<NodeKind>(["recording", "collection", "namespace", "topic"]);

/** Edge kinds: each has its own dash pattern, so they differ by more than colour. */
export const EDGE_KINDS: {
Expand Down Expand Up @@ -77,6 +89,9 @@ export const EDGE_KINDS: {
{ key: "contains", label: "Contains", width: 1.2 },
{ key: "speaks in", label: "Speaks in", dash: "4 2", width: 1.1 },
{ key: "mentioned in", label: "Mentioned in", dash: "2 2", width: 1 },
{ key: "about", label: "About", dash: "5 2", width: 1.2 },
{ key: "narrower", label: "Narrower", width: 1.4 },
{ key: "related", label: "Related", dash: "1 2", width: 1.1 },
];

export function edgeStyle(kind: string) {
Expand All @@ -102,7 +117,7 @@ export function nodeShape(n: Pick<GraphNode, "kind" | "type">): Shape {
export function typeLabel(n: Pick<GraphNode, "kind" | "type">): string {
const g = NODE_GROUPS.find((x) => x.key === nodeGroup(n));
if (!g) return "Entity";
return g.key === "TERM" ? "Topic" : g.label.replace(/s$/, "");
return g.label.replace(/s$/, "");
}

/** Identity links are shown whatever their weight. */
Expand Down Expand Up @@ -264,7 +279,9 @@ export type ApiEdge = {
export function fromApiNode(n: ApiNode): GraphNode {
const label = n.labels[0]?.toLowerCase();
const kind: NodeKind =
label === "speaker" || label === "recording" || label === "collection" || label === "namespace" ? label : "entity";
label === "speaker" || label === "recording" || label === "collection" || label === "namespace" || label === "topic"
? label
: "entity";
const num = Number(n.id.replace(/^[a-z]/, ""));
return {
id: n.id,
Expand Down Expand Up @@ -308,6 +325,7 @@ export function nodeHref(n: Pick<GraphNode, "kind" | "refs" | "id">): string | n
if (n.kind === "recording") return `/recordings/${id}`;
if (n.kind === "collection") return `/collections/${id}`;
if (n.kind === "entity" && n.refs.length === 1) return `/entities/${id}`;
if (n.kind === "topic" && n.refs.length === 1) return `/topics/${id}`;
return null;
}

Expand Down
9 changes: 9 additions & 0 deletions nextjs-frontend/components/graph/shape.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export function nodeFill(n: Pick<GraphNode, "kind" | "type" | "refs">): string {
if (n.kind === "recording") return "var(--aladdin-blue)";
if (n.kind === "collection") return "var(--aladdin-gold)";
if (n.kind === "namespace") return "var(--text-strong)";
if (n.kind === "topic") return "var(--aladdin-green)";
switch (n.type) {
case "ORG":
case "PERSON":
Expand Down Expand Up @@ -82,6 +83,14 @@ export function ShapePath({
{...common}
/>
);
case "tag":
return (
<path
d={`M${-r} ${-r * 0.7} H${r * 0.35} L${r} 0 L${r * 0.35} ${r * 0.7} H${-r} Z`}
strokeLinejoin="round"
{...common}
/>
);
case "folder":
return (
<path
Expand Down
Loading
Loading