Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
Lens also keeps a few counted events (install, daily active, feature used, upgrade, inactive, return). They can
optionally go to your own Google Analytics 4 property too; that is off by default and is set in Settings → Usage
analytics or the setup wizard ([Usage analytics](docs/analytics.md)).
- **Merge entities in fewer clicks, and undo instead of confirm.** On the Entities page, tick rows and the bar at the
bottom merges them into one (the one on the fixed list, else the most mentioned) or gives them one type. An entity's
side panel has **Merge**, a find box where Enter merges the entity picked into it, and **Maybe the same**, its likely
doubles with **Merge** and **Not the same**. Renaming an entity to another's name offers to merge them. Merges, type
changes and hiding say what happened with **Undo** ([Entities](docs/processing.md#entities-and-the-graph-explorer)).

## 0.9.0 <small>October 15, 2026</small> {id="0.9.0"}

Expand Down
22 changes: 22 additions & 0 deletions ci/clicks/journeys.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -639,6 +639,28 @@ export const JOURNEYS = [
await page.getByRole("heading", { name: "Alice", level: 1 }).waitFor();
},
},
{
id: "entity-merge",
// the namespace the imports above went to: with all namespaces picked, Entities opens the first by name, which
// namespace-create has made the empty customer-calls
ns: "interviews",
module: "Entities",
feature: "Merge two entities into one, then undo it",
note: "The merge happens at once and the notice has Undo, so nothing asks to confirm; Undo puts both back.",
from: "/",
async run(ctx) {
const { page, meter } = ctx;
await go(ctx, "Entities");
const main = page.getByRole("main");
await meter.click(main.getByRole("checkbox", { name: "Pick Lens", exact: true }), "tick Lens");
await meter.click(main.getByRole("checkbox", { name: "Pick Raspberry Pi", exact: true }), "tick Raspberry Pi");
const bar = page.getByRole("toolbar", { name: "2 entities selected" });
await meter.click(bar.getByRole("button", { name: /^Merge into / }), "Merge into the one it keeps");
await page.getByText(/^Merged (Lens|Raspberry Pi) into /).first().waitFor();
await meter.click(page.getByRole("button", { name: "Undo" }), "Undo on the notice");
await page.getByText("Merge undone").first().waitFor();
},
},
{
// Last: it points Lens at a cloud model, which the stand-in model behind the journeys above isn't
id: "cloud-model",
Expand Down
9 changes: 8 additions & 1 deletion docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,14 @@ People, organisations, products, places, events, works and topics are extracted
numbers are extracted too, but hidden unless asked for.

The **Entities** page in the web app lists a namespace's entities (search, type and collection filters, a Hidden tab)
and opens each in a side panel where editors rename, retype, describe and hide it.
and opens each in a side panel where editors rename, retype, describe, merge and hide it.

- **Merge from the side panel:** **Merge** opens a find box; type a name and press Enter, and the entity picked is
merged into the open one at once. Its **Maybe the same** list offers likely doubles (same letters, an acronym, a near
spelling) with **Merge** and **Not the same**. Renaming an entity to a name another entity has offers to merge them.
- **Merge or retype many:** editors tick rows (or the whole page) and the bar at the bottom merges them into one (the
one on the fixed list, else the most mentioned; the arrow picks another) or gives them all one type.
- Every merge and bulk type change happens at once and has **Undo** on its notice, so nothing asks to confirm.

- **Index** (`GET /api/v1/entities`):
- Search forgives misspellings and covers aliases.
Expand Down
3 changes: 3 additions & 0 deletions fastapi_backend/app/domain/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -606,6 +606,9 @@ def put(a, b, reason, conf):


def suggestions(db, spaces, eid=None, limit=50):
if eid: # one entity's: only its namespace can hold its doubles
own = db.one("SELECT space FROM $r", r=R("entity", int(eid)))
spaces = {own["space"]} & set(spaces) if own else set()
ents = (
[
e
Expand Down
3 changes: 3 additions & 0 deletions fastapi_backend/tests/domain/test_entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ def test_suggestions_and_merge_undo(db, cfg, spaces):
assert ("same letters", {"Northwind Labs", "North Wind Labs"}) in pairs
assert ("acronym", {"AWS", "Amazon Web Services"}) in pairs
keep, other = eid(db, spaces, "Northwind Labs"), eid(db, spaces, "North Wind Labs")
mine = entities.suggestions(db, spaces, keep) # one entity's: only pairs with it, from its own namespace
assert [{s["a"]["id"], s["b"]["id"]} for s in mine] == [{keep, other}]
assert entities.suggestions(db, {store.ns_id(db, "calls")}, keep) == [] # not where it can't be read
entities.not_same(db, eid(db, spaces, "AWS"), eid(db, spaces, "Amazon Web Services"))
assert ("acronym", {"AWS", "Amazon Web Services"}) not in [
(s["reason"], {s["a"]["name"], s["b"]["name"]}) for s in entities.suggestions(db, spaces)
Expand Down
156 changes: 156 additions & 0 deletions nextjs-frontend/__tests__/entities-curate.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import "@testing-library/jest-dom";

import { Entities } from "@/app/openapi-client";
import { EntityBulkBar } from "@/components/entities/bulk-bar";
import { keepOf, MaybeSame, MergeInto, mergeTitle } from "@/components/entities/curate";
import type { EntityRow } from "@/components/entities/model";
import { TooltipProvider } from "@/components/ui/tooltip";

jest.mock("@/app/openapi-client", () => ({
Entities: {
listEntities: jest.fn(),
mergeEntities: jest.fn(),
undoEntityMerge: jest.fn(),
listEntitySuggestions: jest.fn(),
markEntitiesNotSame: jest.fn(),
},
}));
jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) }));
const toast = jest.fn();
jest.mock("@/components/ui/toast", () => ({ useToast: () => toast }));

const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } });
const m = (f: unknown) => f as jest.Mock;
const show = (ui: React.ReactNode) => {
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
return render(
<QueryClientProvider client={qc}>
<TooltipProvider>{ui}</TooltipProvider>
</QueryClientProvider>,
);
};

const row = (id: number, name: string, mentions: number, extra: Partial<EntityRow> = {}): EntityRow => ({
id,
name,
type: "ORG",
type_label: "Organisation",
key: name.toLowerCase(),
aliases: [],
mentions,
recordings: 1,
hidden: false,
...extra,
});

beforeEach(() => jest.clearAllMocks());

describe("which entity a merge keeps", () => {
it("keeps the one on the fixed list, else the most mentioned, else the oldest", () => {
expect(keepOf([row(3, "NWL", 2), row(1, "Northwind Labs", 9)])?.id).toBe(1);
expect(keepOf([row(3, "NWL", 2, { defined: true }), row(1, "Northwind Labs", 9)])?.id).toBe(3);
expect(keepOf([row(5, "B", 4), row(2, "A", 4)])?.id).toBe(2);
expect(keepOf([])).toBeUndefined();
});

it("says what was merged into what", () => {
expect(mergeTitle("Acme", ["Acme Inc"])).toBe("Merged Acme Inc into Acme");
expect(mergeTitle("Acme", ["Acme Inc", "ACME", "Acme Co"])).toBe("Merged 3 entities into Acme");
});
});

describe("merging into an entity", () => {
it("finds one by typing and merges it at once, with Undo on the toast", async () => {
m(Entities.listEntities).mockReturnValue(
ok({
total: 3,
items: [
{ id: 1, name: "Northwind Labs", type_label: "Organisation", mentions: 9 },
{ id: 2, name: "North Wind Labs", type_label: "Organisation", mentions: 2 },
{ id: 7, name: "Unknown", builtin: "unknown", mentions: 0 },
],
}),
);
m(Entities.mergeEntities).mockReturnValue(ok({ ok: true, merge: 44 }));
m(Entities.undoEntityMerge).mockReturnValue(ok({ ok: true }));
show(<MergeInto keep={{ id: 1, name: "Northwind Labs" }} ns="pods" open onOpenChange={() => undefined} />);
const box = screen.getByRole("combobox", { name: "Entity to merge into Northwind Labs" });
expect(box).toHaveFocus();
fireEvent.change(box, { target: { value: "north" } });
// itself and the entities that are always there can't be merged
await waitFor(() => expect(screen.getAllByRole("option")).toHaveLength(1));
expect(screen.getByRole("option")).toHaveTextContent("North Wind Labs");
fireEvent.keyDown(box, { key: "Enter" });
await waitFor(() => expect(Entities.mergeEntities).toHaveBeenCalled());
expect(m(Entities.mergeEntities).mock.calls[0][0].body).toEqual({ keep: 1, others: [2] });
await waitFor(() => expect(toast).toHaveBeenCalled());
const t = toast.mock.calls[0][0];
expect(t.title).toBe("Merged North Wind Labs into Northwind Labs");
t.action.onClick();
await waitFor(() => expect(Entities.undoEntityMerge).toHaveBeenCalled());
expect(m(Entities.undoEntityMerge).mock.calls[0][0].path).toEqual({ mid: 44 });
});

it("offers the ones that may be the same, to merge or set apart", async () => {
m(Entities.listEntitySuggestions).mockReturnValue(
ok([
{
reason: "acronym",
confidence: 0.85,
a: { id: 4, name: "AWS", mentions: 3 },
b: { id: 5, name: "Amazon Web Services", mentions: 1 },
},
]),
);
m(Entities.markEntitiesNotSame).mockReturnValue(ok({ ok: true }));
m(Entities.mergeEntities).mockReturnValue(ok({ ok: true, merge: 3 }));
show(<MaybeSame entity={{ id: 5, name: "Amazon Web Services" }} canEdit />);
expect(await screen.findByText("AWS")).toBeInTheDocument();
fireEvent.click(screen.getByRole("button", { name: "Not the same" }));
await waitFor(() => expect(Entities.markEntitiesNotSame).toHaveBeenCalled());
expect(m(Entities.markEntitiesNotSame).mock.calls[0][0].body).toEqual({ a: 5, b: 4 });
fireEvent.click(screen.getByRole("button", { name: "Merge" }));
await waitFor(() => expect(Entities.mergeEntities).toHaveBeenCalled());
expect(m(Entities.mergeEntities).mock.calls[0][0].body).toEqual({ keep: 5, others: [4] });
});

it("shows nothing to viewers", () => {
show(<MaybeSame entity={{ id: 5, name: "Amazon Web Services" }} canEdit={false} />);
expect(Entities.listEntitySuggestions).not.toHaveBeenCalled();
});
});

describe("the bulk bar", () => {
it("merges the picked into the one it guesses, in one click", () => {
const onMerge = jest.fn();
show(
<EntityBulkBar
picked={[row(3, "NWL", 2), row(1, "Northwind Labs", 9)]}
types={[{ type: "ORG", label: "Organisation" } as never]}
busy={false}
onMerge={onMerge}
onRetype={() => undefined}
onClear={() => undefined}
/>,
);
expect(screen.getByRole("toolbar", { name: "2 entities selected" })).toBeInTheDocument();
fireEvent.click(screen.getByRole("button", { name: /Merge into Northwind Labs/ }));
expect(onMerge.mock.calls[0][0].id).toBe(1);
});

it("needs two to merge", () => {
show(
<EntityBulkBar
picked={[row(3, "NWL", 2)]}
types={[]}
busy={false}
onMerge={() => undefined}
onRetype={() => undefined}
onClear={() => undefined}
/>,
);
expect(screen.getByRole("button", { name: "Merge" })).toBeDisabled();
});
});
94 changes: 94 additions & 0 deletions nextjs-frontend/components/entities/bulk-bar.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
"use client";

import { ChevronDown, GitMerge, Shapes, X } from "lucide-react";

import type { EntityType } from "@/app/openapi-client/types.gen";
import { keepOf } from "@/components/entities/curate";
import type { EntityRow } from "@/components/entities/model";
import { Menu, MenuContent, MenuItem, MenuLabel, MenuTrigger } from "@/components/ui/menu";
import { count, plural } from "@/lib/format";
import { cn } from "@/lib/utils";

const actionCls =
"flex h-[34px] items-center gap-1.5 whitespace-nowrap rounded-pill px-3 text-[13px] font-semibold outline-offset-0 [&_svg]:size-[15px] hover:bg-[color-mix(in_srgb,var(--background)_16%,transparent)] disabled:cursor-not-allowed disabled:opacity-50";

/** The Entities page's floating bar for the rows picked: merge them into one (the one to keep is guessed and can be
* changed) or give them all one type. Both happen at once and can be undone from the toast. */
export function EntityBulkBar({
picked,
types,
busy,
onMerge,
onRetype,
onClear,
}: {
picked: EntityRow[];
types: EntityType[];
busy: boolean;
onMerge: (keep: EntityRow) => void;
onRetype: (type: string) => void;
onClear: () => void;
}) {
if (!picked.length) return null;
const keep = keepOf(picked)!;
const many = picked.length > 1;
return (
<div className="pointer-events-none sticky bottom-5 z-30 mt-2 flex justify-center px-4">
<div
role="toolbar"
aria-label={`${plural(picked.length, "entity", "entities")} selected`}
className="pointer-events-auto flex max-w-full items-center gap-1 overflow-x-auto rounded-pill bg-fg py-1.5 pl-4 pr-1.5 text-background shadow-3 scrollbar-none"
>
<span className="whitespace-nowrap pr-2.5 text-[13px] font-semibold" aria-live="polite">
{count(picked.length)} selected
</span>
<button
type="button"
className={cn(actionCls, "pr-2")}
disabled={!many || busy}
title={many ? undefined : "Pick two or more to merge them"}
onClick={() => onMerge(keep)}
>
<GitMerge aria-hidden />
{many ? `Merge into ${keep.name.length > 24 ? `${keep.name.slice(0, 23)}…` : keep.name}` : "Merge"}
</button>
<Menu>
<MenuTrigger className={cn(actionCls, "-ml-1 px-1.5")} disabled={!many || busy} aria-label="Keep another one">
<ChevronDown aria-hidden />
</MenuTrigger>
<MenuContent align="center" className="max-h-[50dvh] overflow-y-auto">
<MenuLabel>Merge them all into</MenuLabel>
{picked.map((e) => (
<MenuItem key={e.id} onSelect={() => onMerge(e)}>
<span className="min-w-0 flex-1 truncate">{e.name}</span>
<span className="text-[12px] text-fg-muted">{plural(e.mentions, "mention")}</span>
</MenuItem>
))}
</MenuContent>
</Menu>
<Menu>
<MenuTrigger className={actionCls} disabled={busy}>
<Shapes aria-hidden />
Type
</MenuTrigger>
<MenuContent align="center" className="max-h-[50dvh] overflow-y-auto">
<MenuLabel>Make them all</MenuLabel>
{types.map((t) => (
<MenuItem key={t.type} onSelect={() => onRetype(t.type)}>
{t.label}
</MenuItem>
))}
</MenuContent>
</Menu>
<button
type="button"
onClick={onClear}
aria-label="Clear selection"
className="grid size-[34px] shrink-0 place-items-center rounded-full hover:bg-[color-mix(in_srgb,var(--background)_16%,transparent)]"
>
<X className="size-4" />
</button>
</div>
</div>
);
}
Loading
Loading