From b74fb9e4253a5c2a2464246c2dadb5cb77830a98 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sun, 18 Oct 2026 06:13:22 +0000 Subject: [PATCH 1/2] feat(routines): graph workflows find wrong types and noise to tidy The Find what to tidy node (candidates) looks for two more kinds: entities whose type the naming rules read differently now (retype), and noise to hide (hide). Ask the model judges them a kind at a time, and Apply changes makes the sure ones and proposes the rest, each undoable like a merge. A change someone dismissed or undid is no longer found again, so a run never redoes it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018Nbf5G7PvMgThj8BbUPGT8 --- README.md | 4 +- docs/processing.md | 40 +- docs/requirements.md | 1 + fastapi_backend/app/api/mcp_tools.py | 3 +- fastapi_backend/app/api/v1/routes/routines.py | 2 +- fastapi_backend/app/domain/graph_history.py | 2 +- fastapi_backend/app/domain/name_types.py | 31 +- fastapi_backend/app/domain/organize.py | 393 +++++++++++++++--- fastapi_backend/app/schemas/routines.py | 8 +- fastapi_backend/tests/api/test_routines.py | 94 ++++- fastapi_backend/tests/fake_llm.py | 18 + .../__tests__/graph-changes.test.tsx | 69 +++ .../__tests__/routines-model.test.ts | 6 +- .../app/openapi-client/types.gen.ts | 22 +- .../components/routines/graph-changes.tsx | 127 ++++-- .../components/workflows/node-settings.tsx | 8 +- .../components/workflows/workflow-editor.tsx | 6 +- .../components/workflows/workflow-model.ts | 36 +- nextjs-frontend/openapi.json | 47 ++- 19 files changed, 755 insertions(+), 162 deletions(-) create mode 100644 nextjs-frontend/__tests__/graph-changes.test.tsx diff --git a/README.md b/README.md index a897e69e9..9fee5e9b4 100644 --- a/README.md +++ b/README.md @@ -158,7 +158,9 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p namespaces), then tell you in your own chat app, ask the model or the agent, use a tool, call a web address (its credentials kept sealed), run a workflow, pipeline or routine, or change assets and notes. A Classify node in a workflow sorts an asset into labels, - content types or topics, acting only on a sure answer. Anything that reaches outside Lens or changes more than what + content types or topics, acting only on a sure answer. Graph workflows tidy the entities on a routine: look-alikes + to merge or link, types the rules now read differently, and noise to hide, making the changes they're sure of and + proposing the rest, each one undoable. Anything that reaches outside Lens or changes more than what set it off asks first, until you let it run on its own: on the Approvals page or, for a change inside Lens, with a yes in the chat it told you in; agents can decline an approval but never give one. Each automation has limits and budgets, and agents make, try and run them over MCP and in the assistant, or draft one from a sentence; the studio to draw them diff --git a/docs/processing.md b/docs/processing.md index b9f469e35..2f4c4329b 100644 --- a/docs/processing.md +++ b/docs/processing.md @@ -754,20 +754,36 @@ log. A workflow's scope is assets (above), the graph, or an event: an automation's workflow, run by its automation ([The event workflow scope](automations.md#the-event-workflow-scope)). Graph workflows have their own nodes: -- **Candidates** finds pairs of entities that may be one thing, by the same rules as the merge suggestions (same - letters, acronym, spelling, sounds alike, one name inside the other): inside each namespace (merge) or across - namespaces whose graph is shared (link). Pairs someone marked as different, and pairs already proposed, are skipped. -- **Ask the model** (`llm_judge`) shows the model each pair with lines where the names appear and asks whether - they are the same thing, how sure it is, and which name to keep. -- **Filter** keeps the pairs that pass a test, e.g. `verdict.same` equals true. -- **Apply changes** merges or links the pairs whose confidence is at least *apply above* (at most *max apply* a run) - and proposes the rest. Without *apply above* it only proposes. +- **Find what to tidy** (`candidates`) finds, by rules and without a model, what *Look for* says: + - **Look-alikes** (merge, link): pairs of entities that may be one thing, by the same rules as the merge + suggestions (same letters, acronym, spelling, sounds alike, one name inside the other), inside each namespace + (merge) or across namespaces whose graph is shared (link). Pairs someone marked as different are skipped. + - **Wrong types** (retype): entities whose type the naming rules read differently now, from the name itself (a + title, an ending like *Labs* or *Inc*, a known place or organisation: 90% for a term, 80% for another type) or from + most of the lines that say it, at least two (up to 90%). An entity is typed by its first asset; this catches a + term that later lines show is a person. Types a person or an agent set, entities on a fixed list and hidden ones + are left alone, and so are types the namespace doesn't keep. + - **Noise** (hide): entities with no letters or one letter, made of filler words (*Okay so*; not acronyms like + *US*), or a phrase rather than a name (95%; filler words are 70% when the entity has a type other than Term), and + terms said only once (50%, below the default 70%). Entities someone showed again are left alone. + Anything already proposed, dismissed or undone is skipped too, so a run never redoes what someone took back. +- **Ask the model** (`llm_judge`) shows the model each candidate with lines where the names appear and asks, a kind + at a time: whether a pair is the same thing and which name to keep, what type a name is (from the namespace's + types), or whether it is noise; and how sure it is. Each gets a `verdict`, with `agree` true when the change + should be made (and, for pairs, `same`). +- **Filter** keeps the candidates that pass a test, e.g. `verdict.agree` (or `verdict.same`) equals true. +- **Apply changes** merges, links, retypes or hides the candidates whose confidence (the model's, else the rules') is + at least *apply above* (at most *max apply* a run) and proposes the rest. Without *apply above* it only proposes. A + candidate the model said no to is skipped. A fresh archive has the workflow *Organise the entity graph* (both kinds of candidates, the model, then merges at 95% or more, 25 a run) and the routine *Organise the graph every night* that runs it at 03:00 UTC, switched off. Turn it on, or Run now with *propose only* first to see what it would do. -Every change is recorded (`GET /api/v1/graph-changes`): editors of the namespaces involved accept or dismiss -proposals (dismissing says the two are different, so the pair isn't suggested again) and undo applied changes; an -admin can undo everything a run applied at once (`POST /api/v1/routine-runs/{id}/undo`). Merges are undone exactly as -from an entity's page. +Every change is recorded (`GET /api/v1/graph-changes`), and shows in Routines › Proposed changes: editors of the +namespaces involved accept or dismiss proposals (dismissing a pair says the two are different; dismissing a type or a +hide leaves the entity as it is; neither is suggested again), choose another type when accepting a retype (`to`), and +undo applied changes; an admin can undo everything a run applied at once (`POST /api/v1/routine-runs/{id}/undo`). +Merges are undone exactly as from an entity's page; a type or a hide someone has changed since stays as they left it. +Agents do the same through the API, and `undo_graph_change` over MCP. To tidy types and noise, add a workflow that +looks for wrong types and noise (no built-in workflow does, so nothing changes until you add one). diff --git a/docs/requirements.md b/docs/requirements.md index b5bff08fd..5837704b2 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -180,6 +180,7 @@ request, to another check that proves the same thing; loosening a requirement or | Pipelines and workflows on a canvas | [test_workflows.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_workflows.py) `test_workflows_run_in_pipelines_drawn_as_graphs`; [workflow-canvas.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-canvas.test.ts); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `pipeline-create`, `workflow-create` | | A Classify node in workflows chooses from labels, content types or topics, and acting on a sure answer keeps something | [test_classify.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_classify.py) `test_a_workflow_classifies`; [workflow-flow.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-flow.test.ts) `chooses from labels, content types or topics, and acting on a sure answer keeps something` | | Routines on a schedule | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_routines_are_checked_and_run_when_due`, `test_schedules`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `routine-create`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `routine-create` ≤ 2 clicks | +| Graph workflows find look-alikes to merge or link, wrong types and noise, make the sure changes and propose the rest, each undoable and not made again once taken back | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_graph_workflows_organise_entities`, `test_rules_read_types_and_noise`, `test_graph_workflows_retype_and_hide`; [graph-changes.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/graph-changes.test.tsx) `say a type change in words and make it` | | Batch runs over a collection | [test_batches.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_batches.py) `test_collections_and_batches` | | Notifications to webhooks, Slack, Discord and Matterbridge | [test_notifications.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notifications.py) `test_owners_manage_targets_and_a_test_is_signed`, `test_runs_that_end_are_sent_once_to_matterbridge` | | Everything that happens is kept as W3C Activity Streams activities, read by agents through the API, MCP and the assistant | [test_events.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_events.py) `test_activities_have_one_shape`; [test_activities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_activities.py) `test_the_timeline_lists_new_assets_newest_first`, `test_agents_read_the_timeline_too`; [test_news_feed_source.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_news_feed_source.py) `test_a_news_feed_as_a_source` | diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py index 2ed601155..37b4aad29 100644 --- a/fastapi_backend/app/api/mcp_tools.py +++ b/fastapi_backend/app/api/mcp_tools.py @@ -1583,7 +1583,8 @@ def propose_graph_change(ctx: Context, kind: str, a: str, b: str, reason: str | @tool( "undo_graph_change", "Undo a graph change", - "Take back a merge or link made by propose_graph_change (with apply, or once accepted): its id is the change's. " + "Take back a graph change that was made: a merge or link from propose_graph_change (with apply, or once accepted), " + "or a merge, link, type change or hide a graph workflow made (GET /api/v1/graph-changes). Its id is the change's. " "Needs a token with the write scope and editor access.", Arg("change_id", "integer", "the change (propose_graph_change's id)", required=True), writes=True, diff --git a/fastapi_backend/app/api/v1/routes/routines.py b/fastapi_backend/app/api/v1/routes/routines.py index cdc2823ab..f6ae2f43f 100644 --- a/fastapi_backend/app/api/v1/routes/routines.py +++ b/fastapi_backend/app/api/v1/routes/routines.py @@ -159,7 +159,7 @@ def list_graph_changes( def accept_graph_change(cid: int, body: GraphChangeAccept, user: Writer, acl: Acl, db: Db) -> Ok: _change(db, acl, cid, "curate") with domain_errors(): - organize.accept(db, cid, user.email, body.keep) + organize.accept(db, cid, user.email, body.keep, body.to) auth.audit(db, user.as_audit(), "graph_change.accept", f"graph_change:{cid}") return Ok() diff --git a/fastapi_backend/app/domain/graph_history.py b/fastapi_backend/app/domain/graph_history.py index 9f0b3a6d9..b87c71214 100644 --- a/fastapi_backend/app/domain/graph_history.py +++ b/fastapi_backend/app/domain/graph_history.py @@ -526,7 +526,7 @@ def resolve(db, ref): # ---------- rolling back ---------- MERGES = ("entity.merge", "merge.apply", "merge.accept") -APPLIED = ("merge.apply", "link.apply", "merge.accept", "link.accept") +APPLIED = tuple(f"{k}.{how}" for k in ("merge", "link", "retype", "hide") for how in ("apply", "accept")) KEPT = ("analysis",) # what analysis found stays: it comes from the transcripts, and mentions point at it diff --git a/fastapi_backend/app/domain/name_types.py b/fastapi_backend/app/domain/name_types.py index ce41e5f56..051d42c9d 100644 --- a/fastapi_backend/app/domain/name_types.py +++ b/fastapi_backend/app/domain/name_types.py @@ -285,25 +285,34 @@ def kind(words, before="", after="", people=frozenset()): return None -def guess(name, lines=(), people=frozenset()): - """The type rules give `name` from what it is and the lines it's said in: the type most of them agree on, else - None.""" +def votes(name, lines=(), people=frozenset()): + """({type: how many of the lines `name` is said in give it that type}, how many lines say it).""" words = WORD.findall(name) or [name] - votes = {} - found = False + out, said = {}, 0 for line in lines: text = line or "" m = re.search(rf"(?= 2 and counts[top] * 2 >= said: + share = counts[top] / said + return top, f"{counts[top]} of {said} lines say so", round(min(0.9, 0.5 + 0.4 * share) if term else min(0.8, 0.4 + 0.4 * share), 2) + return None + + +def wrong_types(db, spaces, floor=0.7, limit=100, types=()): + """Entities whose type the rules read differently now: [{kind: retype, pair, a, to, reason, confidence}].""" + held, people_set, names, found = _held(db), entities.typed_by_people(db), store.space_names(db), [] + for sid in sorted(spaces): + ents, counts = _live(db, sid, set(types)) + ents = [e for e in ents if e["id"] not in people_set] + if not ents: + continue + codes, setup, people = entity_setup.type_codes(db, sid), entity_setup.effective(db, sid), analyze.people_of(db, sid) + for k in range(0, len(ents), entities.TYPED_CHUNK): + part = ents[k : k + entities.TYPED_CHUNK] + lines = entities._lines_of(db, [e["id"] for e in part]) + for e in part: + got = better_type(e["name"], e["type"], lines.get(e["id"]) or [], people) + if not got or got[0] not in codes or not entity_setup.keeps(setup, got[0]): + continue + to, why, conf = got + pk = f"retype:{e['id']}:{to}" + if conf >= floor and pk not in held: + found.append((conf, counts[e["id"]], e, to, f"{_type_label(db, sid, to)}: {why}", pk)) + found.sort(key=lambda x: (-x[0], -x[1])) + return [ + {"kind": "retype", "pair": pk, "a": _side(db, e, n, names), "to": to, "reason": why, "confidence": conf} + for conf, n, e, to, why, pk in found[:limit] + ] + + +FILLER = {w.casefold() for w in analyze.STOP | analyze.STOPCAP} +PHRASE_WORDS, PHRASE_CHARS = 6, 60 # longer than this reads like a phrase, not a name + + +def noise_reason(name, typ, mentions): + """(why, confidence) a name looks like noise, or None.""" + words = analyze.WORD.findall(name) + term = typ == "TERM" + if not re.search(r"[^\W\d_]", name): + return "no letters in it", 0.95 + if len(name.strip()) == 1: + return "a single letter", 0.95 + if all(w.casefold() in FILLER for w in words) and not (name.isupper() and len(name) > 1): # US, OK are names + return "filler words, not a name", 0.95 if term else 0.7 + if len(words) > PHRASE_WORDS or len(name) > PHRASE_CHARS: + return "reads like a phrase, not a name", 0.8 + if term and mentions == 1: + return "a term said only once", 0.5 + return None + + +def _shown_again(db): + """Entities someone showed after they were hidden.""" + out = set() + for ev in db.rows("SELECT ops FROM graph_event WHERE op = 'entity.show'"): + out.update(int(o["k"]) for o in ev.get("ops") or [] if o.get("t") == "entity" and str(o.get("k")).isdigit()) + return out + + +def noise(db, spaces, floor=0.7, limit=100, types=()): + """Entities that look like noise, to hide: [{kind: hide, pair, a, reason, confidence}].""" + held, shown, names, found = _held(db), _shown_again(db), store.space_names(db), [] + for sid in sorted(spaces): + ents, counts = _live(db, sid, set(types)) + for e in ents: + got = noise_reason(e["name"], e["type"], counts[e["id"]]) + pk = f"hide:{e['id']}" + if got and got[1] >= floor and pk not in held and e["id"] not in shown: + found.append((got[1], -counts[e["id"]], e, counts[e["id"]], got[0], pk)) + found.sort(key=lambda x: (-x[0], -x[1])) + return [ + {"kind": "hide", "pair": pk, "a": _side(db, e, n, names), "reason": why, "confidence": conf} + for conf, _o, e, n, why, pk in found[:limit] + ] + + # ---------- the model's verdict ---------- def _describe(k, item): def side(s): @@ -182,36 +336,87 @@ def side(s): return f"{k}. a: {side(item['a'])}\n b: {side(item['b'])}\n rules thought: {item['reason']} ({item['confidence']})" -def judge(cfg, items, c, say=print): - """The items, each with the model's `verdict` (pairs it gave no answer for get none).""" - size, out = c.get("batch") or 25, [] - for start in range(0, len(items), size): - batch = items[start : start + size] - ask = [] - if c.get("instructions"): - ask.append(c["instructions"].strip()) - ask.append("Pairs:\n" + "\n".join(_describe(k, it) for k, it in enumerate(batch, 1))) - try: - reply = llm.json_out(cfg, JUDGE_SYSTEM, "\n\n".join(ask), JUDGE_SCHEMA, c.get("model")) - except Exception as e: # noqa: BLE001 - one bad batch leaves its pairs unjudged, not the run failed - say(f"pairs {start + 1}-{start + len(batch)}: the model failed ({type(e).__name__}: {str(e)[:200]})") - reply = None - got = {} - for d in (reply or {}).get("judgements") or []: - if isinstance(d, dict) and isinstance(d.get("pair"), int) and isinstance(d.get("same"), bool): - conf = d.get("confidence") if _num(d.get("confidence")) else 0 - got[d["pair"]] = store.clean( - { - "same": d["same"], - "confidence": round(min(1.0, max(0.0, float(conf))), 2), - "keep": d.get("keep") if d.get("keep") in ("a", "b") else None, - "why": str(d.get("why") or "")[:300] or None, - } - ) - for k, it in enumerate(batch, 1): - out.append({**it, **({"verdict": got[k]} if k in got else {})}) - say(f"judged {len(got)} of {len(batch)} pairs") - return out +def _pairs_judged(cfg, batch, c): + """{number in the batch: verdict} for pairs: one thing or two, and which name to keep.""" + ask = [c["instructions"].strip()] if c.get("instructions") else [] + ask.append("Pairs:\n" + "\n".join(_describe(k, it) for k, it in enumerate(batch, 1))) + reply = llm.json_out(cfg, JUDGE_SYSTEM, "\n\n".join(ask), JUDGE_SCHEMA, c.get("model")) + got = {} + for d in (reply or {}).get("judgements") or []: + if isinstance(d, dict) and isinstance(d.get("pair"), int) and isinstance(d.get("same"), bool): + got[d["pair"]] = store.clean( + { + "same": d["same"], + "agree": d["same"], + "confidence": _sure(d.get("confidence")), + "keep": d.get("keep") if d.get("keep") in ("a", "b") else None, + "why": str(d.get("why") or "")[:300] or None, + } + ) + return got + + +def _sure(v): + return round(min(1.0, max(0.0, float(v))), 2) if _num(v) else 0 + + +def _describe_one(k, item): + s = item["a"] + said = "; ".join(f'"{x[:160]}"' for x in s["samples"]) or "no lines" + line = f'{k}. "{s["name"]}" ({s["type"]}, {s["mentions"]} mentions in {s["namespace"]}; found in: {said})' + if item["kind"] == "retype": + return f"{line}\n rules think: {item['to']} ({item['reason']}; {item['confidence']})" + return f"{line}\n rules think: noise ({item['reason']}; {item['confidence']})" + + +def _ones_judged(db, cfg, kind, batch, c): + """{number in the batch: verdict} for entities: their type, or whether they are noise.""" + ask = [c["instructions"].strip()] if c.get("instructions") else [] + codes = set() + if kind == "retype": + for sid in {it["a"]["space"] for it in batch}: + codes |= entity_setup.type_codes(db, sid) - set(entities.QUIET) + ask.append("Types: " + ", ".join(sorted(codes))) + ask.append("Names:\n" + "\n".join(_describe_one(k, it) for k, it in enumerate(batch, 1))) + reply = llm.json_out(cfg, ONE_SYSTEM[kind], "\n\n".join(ask), _one_schema(kind), c.get("model")) + got = {} + for d in (reply or {}).get("tidying") or []: + if not (isinstance(d, dict) and isinstance(d.get("item"), int) and 1 <= d["item"] <= len(batch)): + continue + it, why = batch[d["item"] - 1], str(d.get("why") or "")[:300] or None + if kind == "retype": + typ = str(d.get("type") or "").strip().upper() + if typ not in codes: + continue + got[d["item"]] = store.clean( + {"agree": typ != it["a"]["type"], "type": typ, "confidence": _sure(d.get("confidence")), "why": why} + ) + elif isinstance(d.get("noise"), bool): + got[d["item"]] = store.clean({"agree": d["noise"], "confidence": _sure(d.get("confidence")), "why": why}) + return got + + +def judge(cfg, items, c, say=print, db=None): + """The items, each with the model's `verdict` (those it gave no answer for get none). Each kind is asked on its own, + a batch at a time.""" + size, verdicts = c.get("batch") or 25, {} + by_kind = defaultdict(list) + for n, it in enumerate(items): + if isinstance(it, dict) and it.get("kind") in KINDS and isinstance(it.get("a"), dict): + by_kind["pair" if it["kind"] in PAIRS else it["kind"]].append(n) + for kind, ns in by_kind.items(): + what = "pairs" if kind == "pair" else "names" + for start in range(0, len(ns), size): + part = ns[start : start + size] + batch = [items[n] for n in part] + try: + got = _pairs_judged(cfg, batch, c) if kind == "pair" else _ones_judged(db, cfg, kind, batch, c) + except Exception as e: # noqa: BLE001 - one bad batch leaves its candidates unjudged, not the run failed + say(f"{what} {start + 1}-{start + len(batch)}: the model failed ({type(e).__name__}: {str(e)[:200]})") + got = {} + verdicts.update({part[k - 1]: v for k, v in got.items() if 1 <= k <= len(part)}) + say(f"judged {len(got)} of {len(batch)} {what}") + return [{**it, "verdict": verdicts[n]} if n in verdicts else it for n, it in enumerate(items)] # ---------- changes ---------- @@ -228,6 +433,8 @@ def _keep(item): def _make(db, ch, user): """Make one change; the fields to record on it (merge id), or None when it can't be made any more.""" + if ch["kind"] not in PAIRS: + return _make_one(db, ch) a, b = ch["a"]["id"], ch["b"]["id"] if not (_alive(db, a) and _alive(db, b)): return None @@ -238,6 +445,34 @@ def _make(db, ch, user): return {} +def _make_one(db, ch): + """Retype or hide one entity, unless it's gone or has changed since the change was found.""" + e = db.one(f"SELECT {entities.FIELDS} FROM $r", r=R("entity", int(ch["a"]["id"]))) + if not e or e.get("hidden"): + return None + if ch["kind"] == "retype": + to = ch.get("to") + if e["type"] != ch["a"]["type"] or not to or to == e["type"]: + return None + try: + entities.retype(db, [e["id"]], to) + except ValueError: # the type is gone from the namespace, or the entity is on a fixed list now + return None + return {"to": to} + entities.hide(db, e["id"], True, f"noise: {ch.get('reason') or 'tidied'}") + return {} + + +def _target(it): + """A retype's type: the model's, when it gave one, else the rules'.""" + v = it.get("verdict") or {} + return v.get("type") or it.get("to") + + +def _touches(it): + return [it["a"]["id"], *([it["b"]["id"]] if it["kind"] in PAIRS else [])] + + def _record(db, item, status, origin, extra=None): cid = db.next_id("graph_change") # a new id also tells the graph cache to rebuild row = store.clean( @@ -245,8 +480,9 @@ def _record(db, item, status, origin, extra=None): "kind": item["kind"], "pair": item["pair"], "a": item["a"], - "b": item["b"], - "spaces": sorted({item["a"]["space"], item["b"]["space"]}), + "b": item.get("b"), + "to": _target(item) if item["kind"] == "retype" else None, + "spaces": sorted({item["a"]["space"], *([item["b"]["space"]] if item.get("b") else [])}), "reason": item.get("reason"), "confidence": item.get("confidence"), "verdict": item.get("verdict"), @@ -267,18 +503,24 @@ def apply_changes(db, items, c, origin=None, propose_only=False, say=print): stats = {"applied": 0, "proposed": 0, "skipped": 0} user = f"routine:{origin['routine']}" if origin and origin.get("routine") else "workflow" for it in items if isinstance(items, list) else []: - if not (isinstance(it, dict) and it.get("kind") in KINDS and isinstance(it.get("a"), dict) and isinstance(it.get("b"), dict)): + if not ( + isinstance(it, dict) + and it.get("kind") in KINDS + and isinstance(it.get("a"), dict) + and (it["kind"] not in PAIRS or isinstance(it.get("b"), dict)) + and (it["kind"] != "retype" or _target(it)) + ): stats["skipped"] += 1 continue v = it.get("verdict") or {} - if v.get("same") is False: + if v.get("same") is False or v.get("agree") is False: stats["skipped"] += 1 continue sure = v.get("confidence", it.get("confidence") or 0) if not propose_only and above is not None and sure >= above and stats["applied"] < cap: why = v.get("why") or it.get("reason") - with graph_history.change(db, f"{it['kind']}.apply", entities=[it["a"]["id"], it["b"]["id"]], why=why) as ch: - done = _make(db, {**it, "keep": _keep(it) if it["kind"] == "merge" else None}, user) + with graph_history.change(db, f"{it['kind']}.apply", entities=_touches(it), why=why) as ch: + done = _make(db, {**it, "keep": _keep(it) if it["kind"] == "merge" else None, "to": _target(it)}, user) if done is not None: ch.add(graph_change=_record(db, it, "applied", origin, {**done, "decided_at": store.now(), "decided_by": user})) if done is None: @@ -349,18 +591,21 @@ def get_change(db, cid): return ch -def accept(db, cid, user=None, keep=None): - """Make a proposed change.""" +def accept(db, cid, user=None, keep=None, to=None): + """Make a proposed change (`keep`: a merge's entity to keep; `to`: a retype's type, in place of the one proposed).""" ch = get_change(db, cid) if ch["status"] != "proposed": raise ValueError(f"this change is {ch['status']}, not proposed") if keep is not None: - if int(keep) not in (ch["a"]["id"], ch["b"]["id"]): + if ch["kind"] != "merge" or int(keep) not in (ch["a"]["id"], ch["b"]["id"]): raise ValueError("keep one of the two entities") ch["keep"] = int(keep) - with graph_history.change( - db, f"{ch['kind']}.accept", entities=[ch["a"]["id"], ch["b"]["id"]], why=ch.get("reason"), graph_change=ch["id"] - ): + if to is not None: + if ch["kind"] != "retype": + raise ValueError("only a change of type takes a type") + entities._check_type(db, ch["a"]["space"], to) + ch["to"] = to + with graph_history.change(db, f"{ch['kind']}.accept", entities=_touches(ch), why=ch.get("reason"), graph_change=ch["id"]): done = _make(db, ch, user) if done is None: db.q( @@ -369,7 +614,9 @@ def accept(db, cid, user=None, keep=None): t=store.now(), u=user, ) - raise ValueError("one of these entities is gone (merged or deleted since)") + if ch["kind"] in PAIRS: + raise ValueError("one of these entities is gone (merged or deleted since)") + raise ValueError("the entity is gone, hidden or changed since this was proposed") db.q( "UPDATE $r MERGE $d", r=R("graph_change", ch["id"]), @@ -382,24 +629,34 @@ def dismiss(db, cid, user=None): ch = get_change(db, cid) if ch["status"] != "proposed": raise ValueError(f"this change is {ch['status']}, not proposed") - if _alive(db, ch["a"]["id"]) and _alive(db, ch["b"]["id"]): + if ch["kind"] in PAIRS and _alive(db, ch["a"]["id"]) and _alive(db, ch["b"]["id"]): with graph_history.change(db, f"{ch['kind']}.dismiss", graph_change=ch["id"]): entities.not_same(db, ch["a"]["id"], ch["b"]["id"]) db.q("UPDATE $r SET status = 'dismissed', decided_at = $t, decided_by = $u", r=R("graph_change", ch["id"]), t=store.now(), u=user) def undo(db, cid, user=None): - """Take back an applied change (a merge is undone as the entity page would).""" + """Take back an applied change (a merge is undone as the entity page would). A type or a hide that someone has + changed since stays as they left it.""" ch = get_change(db, cid) if ch["status"] != "applied": raise ValueError(f"this change is {ch['status']}, not applied") - with graph_history.change(db, f"{ch['kind']}.undo", graph_change=ch["id"]): + one = [] if ch["kind"] in PAIRS else [ch["a"]["id"]] + with graph_history.change(db, f"{ch['kind']}.undo", entities=one, graph_change=ch["id"]): if ch["kind"] == "merge": m = db.one("SELECT undone FROM $r", r=R("entity_merge", int(ch["merge"]))) if m and not m.get("undone"): # someone may have undone it from the entity page already entities.undo_merge(db, ch["merge"]) - else: + elif ch["kind"] == "link": entities.unlink(db, ch["a"]["id"], ch["b"]["id"]) + else: + e = db.one(f"SELECT {entities.FIELDS} FROM $r", r=R("entity", int(ch["a"]["id"]))) + if not e: + raise ValueError("the entity is gone (merged or deleted since)") + if ch["kind"] == "retype" and e["type"] == ch.get("to") and not e.get("builtin"): + db.q("UPDATE $r SET type = $t", r=R("entity", e["id"]), t=ch["a"]["type"]) + elif ch["kind"] == "hide" and e.get("hidden") and str(e.get("hidden_reason") or "").startswith("noise:"): + entities.hide(db, e["id"], False) db.q("UPDATE $r SET status = 'undone', undone_at = $t, undone_by = $u", r=R("graph_change", ch["id"]), t=store.now(), u=user) @@ -465,7 +722,7 @@ def run_node(r, f, node, value, vals): r.say(f"{name}: {len(out)} pairs") return {"out": out} if t == "llm_judge": - return {"out": judge(r.cfg, value if isinstance(value, list) else [], c, r.say)} + return {"out": judge(r.cfg, value if isinstance(value, list) else [], c, r.say, r.db)} if t == "apply_changes": if r.dry: r.say(f"{name}: would make or propose {len(value) if isinstance(value, list) else 0} changes") diff --git a/fastapi_backend/app/schemas/routines.py b/fastapi_backend/app/schemas/routines.py index 96972ad6d..2469839be 100644 --- a/fastapi_backend/app/schemas/routines.py +++ b/fastapi_backend/app/schemas/routines.py @@ -102,9 +102,10 @@ class SchedulePreview(ResponseModel): class GraphChange(ResponseModel): id: int - kind: Literal["merge", "link"] - a: dict[str, Any] - b: dict[str, Any] + kind: Literal["merge", "link", "retype", "hide"] + a: dict[str, Any] = Field(description="the entity changed, or the first of a pair") + b: dict[str, Any] | None = Field(None, description="a merge's or link's other entity") + to: str | None = Field(None, description="a retype's new type") spaces: list[int] reason: str | None = None confidence: float | None = None @@ -121,6 +122,7 @@ class GraphChange(ResponseModel): class GraphChangeAccept(RequestModel): keep: int | None = Field(None, description="for a merge: the entity to keep (default: the one proposed)") + to: str | None = Field(None, description="for a retype: the type to give (default: the one proposed)", max_length=40) class Undone(ResponseModel): diff --git a/fastapi_backend/tests/api/test_routines.py b/fastapi_backend/tests/api/test_routines.py index 45aea0f87..1b08e99a3 100644 --- a/fastapi_backend/tests/api/test_routines.py +++ b/fastapi_backend/tests/api/test_routines.py @@ -1,5 +1,5 @@ """Routines: schedules, running what is due once, pipeline and workflow actions on recordings, and the graph-organising -workflow proposing, making and undoing merges and links.""" +workflows proposing, making and undoing merges, links, new types and hides.""" from __future__ import annotations @@ -7,7 +7,7 @@ import pytest -from app.domain import analyze, entities, ingest, organize, routines, schedule, store, workflows +from app.domain import analyze, entities, graph_history, ingest, organize, routines, schedule, store, workflows from tests import fake_llm from tests.helpers import login, make_user, quiet, seed @@ -283,6 +283,96 @@ def test_propose_only(db, cfg, env): assert all(entities._entity(db, x) for x in north) +TIDY_GRAPH = { + "nodes": [ + {"id": "in", "type": "input", "config": {}}, + {"id": "types", "type": "candidates", "config": {"kind": "retype"}}, + {"id": "noise", "type": "candidates", "config": {"kind": "hide"}}, + {"id": "all", "type": "merge", "config": {}}, + {"id": "judge", "type": "llm_judge", "config": {"batch": 10}}, + {"id": "yes", "type": "filter", "config": {"path": "verdict.agree", "op": "equals", "value": True}}, + {"id": "apply", "type": "apply_changes", "config": {"apply_above": 0.95, "max_apply": 25}}, + ], + "edges": [ + {"source": "in", "target": "types"}, + {"source": "in", "target": "noise"}, + {"source": "types", "target": "all"}, + {"source": "noise", "target": "all"}, + {"source": "all", "target": "judge"}, + {"source": "judge", "target": "yes"}, + {"source": "yes", "target": "apply"}, + ], +} + + +def test_rules_read_types_and_noise(): + assert organize.better_type("Northwind Labs", "TERM", []) == ("ORG", "its name says so", 0.9) + assert organize.better_type("Northwind Labs", "ORG", []) is None + lines = ["Then Ferris said no.", "We asked Dr Ferris.", "Dr Ferris agreed."] + assert organize.better_type("Ferris", "TERM", lines)[:2] == ("PERSON", "2 of 3 lines say so") + assert organize.better_type("Ferris", "TERM", lines[:1]) is None # one line isn't enough + assert organize.noise_reason("Okay so", "TERM", 4) == ("filler words, not a name", 0.95) + assert organize.noise_reason("US", "PLACE", 4) is None # an acronym, not the word "us" + assert organize.noise_reason("42", "TERM", 4)[0] == "no letters in it" + assert organize.noise_reason("the thing we talked about at the start of it", "TERM", 2)[0].startswith("reads like a phrase") + assert organize.noise_reason("Capsid", "TERM", 1) == ("a term said only once", 0.5) + assert organize.noise_reason("Capsid", "TERM", 3) is None + + +def test_graph_workflows_retype_and_hide(client, db, cfg, env, llm): + h = env["admin"] + pods = store.ns_id(db, "pods") + labs, aws_long, dyno = eid(db, "Northwind Labs"), eid(db, "Amazon Web Services"), eid(db, "Dyno Therapeutics") + db.q("UPDATE $r SET type = 'TERM'", r=R("entity", labs)) # as an older analysis left it + db.q("UPDATE $r SET type = 'TERM', name = 'Um okay'", r=R("entity", aws_long)) # what an extractor took for a name + wid = workflows.create(db, "Tidy", TIDY_GRAPH, "", "test", "graph") + _, totals = organize.run(db, cfg, wid, {pods}) + assert totals == {"applied": 2, "proposed": 0, "skipped": 0}, totals + assert entities._entity(db, labs)["type"] == "ORG" + hidden = entities._entity(db, aws_long) + assert hidden["hidden"] and hidden["hidden_reason"] == "noise: filler words, not a name" + + made = {c["kind"]: c for c in client.get("/api/v1/graph-changes", params={"status": "applied"}, headers=h).json()} + assert set(made) == {"retype", "hide"} + assert made["retype"]["to"] == "ORG" and made["retype"]["b"] is None and made["retype"]["verdict"]["agree"] + for c in made.values(): + assert client.post(f"/api/v1/graph-changes/{c['id']}/undo", headers=h).status_code == 200 + assert entities._entity(db, labs)["type"] == "TERM" and not entities._entity(db, aws_long)["hidden"] + _, again = organize.run(db, cfg, wid, {pods}) + assert again == {"applied": 0, "proposed": 0, "skipped": 0}, again # what someone took back isn't made again + + # proposed only: accepted with another type, or dismissed and not proposed again + db.q("UPDATE $r SET type = 'TERM'", r=R("entity", dyno)) + _, got = organize.run(db, cfg, wid, {pods}, propose_only=True) + assert got["proposed"] == 1, got + ch = client.get("/api/v1/graph-changes", params={"status": "proposed"}, headers=h).json()[0] + assert (ch["kind"], ch["a"]["id"], ch["to"]) == ("retype", dyno, "ORG") + assert client.post(f"/api/v1/graph-changes/{ch['id']}/accept", headers=h, json={"keep": dyno}).status_code == 400 + assert client.post(f"/api/v1/graph-changes/{ch['id']}/accept", headers=h, json={"to": "NOPE"}).status_code == 400 + assert client.post(f"/api/v1/graph-changes/{ch['id']}/accept", headers=h, json={"to": "PRODUCT"}).status_code == 200 + assert entities._entity(db, dyno)["type"] == "PRODUCT" + assert client.post(f"/api/v1/graph-changes/{ch['id']}/undo", headers=h).status_code == 200 + assert entities._entity(db, dyno)["type"] == "TERM" + + other = eid(db, "Dyno Therapeutics", "calls") + db.q("UPDATE $r SET type = 'TERM'", r=R("entity", other)) + _, got = organize.run(db, cfg, wid, {store.ns_id(db, "calls")}, propose_only=True) + ch = client.get("/api/v1/graph-changes", params={"status": "proposed"}, headers=h).json()[0] + assert client.post(f"/api/v1/graph-changes/{ch['id']}/dismiss", headers=h).status_code == 200 + assert entities._entity(db, other)["type"] == "TERM" + _, got = organize.run(db, cfg, wid, {store.ns_id(db, "calls")}) + assert got["applied"] == got["proposed"] == 0, got + + # a rollback past an applied retype marks it undone + db.q("UPDATE $r SET type = 'TERM', name = 'Okay well'", r=R("entity", labs)) + before = graph_history.head(db) + _, got = organize.run(db, cfg, wid, {pods}) + assert got["applied"] == 1, got # hidden as noise (its name no longer says what it is) + graph_history.rollback(db, before, dry_run=False, by="root@x.io") + assert not entities._entity(db, labs)["hidden"] + assert not client.get("/api/v1/graph-changes", params={"status": "applied"}, headers=h).json() + + def test_new_recordings_advance_only_as_far_as_queued(db, cfg, env): pods = store.ns_id(db, "pods") rid = routines.create(db, "Small batches", [{"type": "pipeline", "steps": ["analyze"], "limit": 1}], namespaces=[pods]) diff --git a/fastapi_backend/tests/fake_llm.py b/fastapi_backend/tests/fake_llm.py index 62a0160b2..520ea1294 100644 --- a/fastapi_backend/tests/fake_llm.py +++ b/fastapi_backend/tests/fake_llm.py @@ -221,6 +221,24 @@ def do_POST(self): ] } ) + elif "tidying" in schema.get("properties", {}): + # graph tidying, one entity at a time: the rules are right (the type they think, or noise), and sure + text = body["messages"][-1]["content"] + guesses = re.findall(r"^(\d+)\. .*\n rules think: (\S+)", text, re.M) + asked = schema["properties"]["tidying"]["items"]["properties"] + content = json.dumps( + { + "tidying": [ + { + "item": int(k), + **({"type": g} if "type" in asked else {"noise": True}), + "confidence": 0.96, + "why": "the lines say so", + } + for k, g in guesses + ] + } + ) elif "mappings" in schema.get("properties", {}): # entity matching: a name goes to the entity whose description says it, else the first other answer offered text = body["messages"][-1]["content"] diff --git a/nextjs-frontend/__tests__/graph-changes.test.tsx b/nextjs-frontend/__tests__/graph-changes.test.tsx new file mode 100644 index 000000000..1e964a3e4 --- /dev/null +++ b/nextjs-frontend/__tests__/graph-changes.test.tsx @@ -0,0 +1,69 @@ +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import "@testing-library/jest-dom"; + +import { Routines } from "@/app/openapi-client"; +import type { GraphChange } from "@/app/openapi-client/types.gen"; +import { ChangeCard, typeName } from "@/components/routines/graph-changes"; +import { ToastProvider } from "@/components/ui/toast"; +import { TooltipProvider } from "@/components/ui/tooltip"; + +jest.mock("@/app/openapi-client", () => ({ + Routines: { acceptGraphChange: jest.fn(), dismissGraphChange: jest.fn(), undoGraphChange: jest.fn() }, +})); +jest.mock("next-auth/react", () => ({ useSession: () => ({ data: { accessToken: "t" } }) })); +jest.mock("@/lib/hooks/session", () => ({ useArchive: () => ({ can: () => true }), needRole: () => "" })); + +const ok = (data: unknown) => Promise.resolve({ data, response: { ok: true, status: 200 } }); +const side = { + id: 5, + name: "Northwind Labs", + type: "TERM", + namespace: "pods", + mentions: 3, + samples: ["Northwind Labs again."], +}; + +function card(ch: Partial) { + const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + const full = { id: 9, spaces: [1], status: "proposed", a: side, ...ch } as GraphChange; + return render( + + + + + + + , + ); +} + +describe("graph changes about one entity", () => { + beforeEach(() => jest.mocked(Routines.acceptGraphChange).mockImplementation(() => ok({ ok: true }) as never)); + + it("say a type change in words and make it", async () => { + card({ kind: "retype", to: "ORG", reason: "Organisation: its name says so", confidence: 0.9 }); + expect(screen.getByText("Type: Term → Organisation?")).toBeInTheDocument(); + expect(screen.getByText("Leave it")).toBeInTheDocument(); + fireEvent.click(screen.getByRole("button", { name: "Make it Organisation" })); + await waitFor(() => expect(Routines.acceptGraphChange).toHaveBeenCalledWith(expect.objectContaining({ body: {} }))); + expect(await screen.findByText("Northwind Labs: Organisation now")).toBeInTheDocument(); + }); + + it("offer to hide noise, and undo it once made", () => { + card({ + kind: "hide", + status: "applied", + reason: "filler words, not a name", + verdict: { agree: true, confidence: 0.96 }, + }); + expect(screen.getByText("Hide as noise?")).toBeInTheDocument(); + expect(screen.getByText("Noise")).toBeInTheDocument(); + expect(screen.getByRole("button", { name: /Undo/ })).toBeInTheDocument(); + }); + + it("name a namespace's own types", () => { + expect(typeName("CLIENT_TEAM")).toBe("Client team"); + expect(typeName("PERSON")).toBe("Person"); + }); +}); diff --git a/nextjs-frontend/__tests__/routines-model.test.ts b/nextjs-frontend/__tests__/routines-model.test.ts index 41e691cbd..194321d9a 100644 --- a/nextjs-frontend/__tests__/routines-model.test.ts +++ b/nextjs-frontend/__tests__/routines-model.test.ts @@ -163,7 +163,11 @@ describe("graph workflows", () => { "Make 95%+ (up to 25), propose the rest", ); expect(nodeSummary({ id: "c", type: "candidates", config: { kind: "link", min_confidence: 0.85 } }, names)).toBe( - "across namespaces · 85%+ · up to 100", + "look-alikes across namespaces · 85%+ · up to 100", ); + expect(nodeSummary({ id: "c", type: "candidates", config: { kind: "retype" } }, names)).toBe( + "wrong types · 70%+ · up to 100", + ); + expect(nodeSummary({ id: "j", type: "llm_judge", config: {} }, names)).toBe("25 at a time"); }); }); diff --git a/nextjs-frontend/app/openapi-client/types.gen.ts b/nextjs-frontend/app/openapi-client/types.gen.ts index 66d75b2a3..5765b00d3 100644 --- a/nextjs-frontend/app/openapi-client/types.gen.ts +++ b/nextjs-frontend/app/openapi-client/types.gen.ts @@ -8807,19 +8807,29 @@ export type GraphChange = { /** * Kind */ - kind: "merge" | "link"; + kind: "merge" | "link" | "retype" | "hide"; /** * A + * + * the entity changed, or the first of a pair */ a: { [key: string]: unknown; }; /** * B + * + * a merge's or link's other entity */ - b: { + b?: { [key: string]: unknown; - }; + } | null; + /** + * To + * + * a retype's new type + */ + to?: string | null; /** * Spaces */ @@ -8883,6 +8893,12 @@ export type GraphChangeAccept = { * for a merge: the entity to keep (default: the one proposed) */ keep?: number | null; + /** + * To + * + * for a retype: the type to give (default: the one proposed) + */ + to?: string | null; }; /** diff --git a/nextjs-frontend/components/routines/graph-changes.tsx b/nextjs-frontend/components/routines/graph-changes.tsx index 4eecca6f5..6fcd4cbc6 100644 --- a/nextjs-frontend/components/routines/graph-changes.tsx +++ b/nextjs-frontend/components/routines/graph-changes.tsx @@ -1,7 +1,7 @@ "use client"; import { useMutation, useQueryClient } from "@tanstack/react-query"; -import { GitMerge, Link2, Undo2, Waypoints } from "lucide-react"; +import { EyeOff, GitMerge, Link2, Tags, Undo2, Waypoints } from "lucide-react"; import Link from "next/link"; import { useState } from "react"; @@ -26,10 +26,41 @@ type Side = { mentions?: number; samples?: string[]; }; -type Verdict = { same?: boolean; confidence?: number; keep?: "a" | "b" | null; why?: string }; +type Verdict = { + same?: boolean; + agree?: boolean; + confidence?: number; + keep?: "a" | "b" | null; + type?: string; + why?: string; +}; const pct = (n: number | null | undefined) => (n == null ? "—" : `${Math.round(n * 100)}%`); +const TYPE_NAMES: Record = { + PERSON: "Person", + ORG: "Organisation", + PRODUCT: "Product", + PLACE: "Place", + EVENT: "Event", + WORK: "Work", + TERM: "Term", + DATE: "Date", + NUMBER: "Number", +}; +/** "Organisation" for ORG; a namespace's own CLIENT_TEAM reads "Client team". */ +export const typeName = (code: string | null | undefined) => + code ? (TYPE_NAMES[code] ?? (code.charAt(0) + code.slice(1).toLowerCase()).replace(/_/g, " ")) : "—"; + +/** What a change is about, in a few words, and its icon. */ +function heading(ch: GraphChange): { icon: typeof GitMerge; title: string } { + const a = ch.a as Side; + if (ch.kind === "merge") return { icon: GitMerge, title: "Merge: the same thing in one namespace?" }; + if (ch.kind === "link") return { icon: Link2, title: "Link: the same thing across namespaces?" }; + if (ch.kind === "retype") return { icon: Tags, title: `Type: ${typeName(a.type)} → ${typeName(ch.to)}?` }; + return { icon: EyeOff, title: "Hide as noise?" }; +} + function EntitySide({ side, kept, choose }: { side: Side; kept?: boolean; choose?: () => void }) { return (
(ch.keep ?? (v?.keep === "b" ? b.id : a.id)); - const editor = can("editor", a.namespace) && can("editor", b.namespace); - const why = editor ? undefined : needRole("editor", a.namespace ?? b.namespace); + const pair = b != null; + const [keep, setKeep] = useState(ch.keep ?? (v?.keep === "b" && b ? b.id : a.id)); + const editor = can("editor", a.namespace) && (!b || can("editor", b.namespace)); + const why = editor ? undefined : needRole("editor", a.namespace ?? b?.namespace); + const { icon: Icon, title } = heading(ch); + const kept = keep === a.id || !b ? a.name : b.name; const act = useMutation({ mutationFn: (what: "accept" | "dismiss" | "undo") => { @@ -94,17 +129,30 @@ function ChangeCard({ ch }: { ch: GraphChange }) { onSuccess: (_, what) => { void qc.invalidateQueries({ queryKey: ["graph-changes"] }); void qc.invalidateQueries({ queryKey: ["graph"] }); + const made = { + merge: `Merged into ${kept}`, + link: "Linked", + retype: `${a.name}: ${typeName(ch.to)} now`, + hide: `${a.name} is hidden`, + }; toast({ tone: "green", title: what === "accept" - ? merge - ? `Merged into ${keep === a.id ? a.name : b.name}` - : "Linked" + ? made[ch.kind] : what === "dismiss" - ? "Marked as different" + ? pair + ? "Marked as different" + : "Left as it is" : "Change undone", - body: what === "dismiss" ? "They won’t be suggested again." : `${a.name} · ${b.name}`, + body: + what === "dismiss" + ? pair + ? "They won’t be suggested again." + : "It won’t be suggested again." + : b + ? `${a.name} · ${b.name}` + : a.name, }); }, onError: (e: Error) => toast({ tone: "red", title: "Couldn’t do that", body: e.message }), @@ -112,18 +160,12 @@ function ChangeCard({ ch }: { ch: GraphChange }) { return (
- {merge ? ( - - ) : ( - - )} - - {merge ? "Merge: the same thing in one namespace?" : "Link: the same thing across namespaces?"} - + + {title} {ch.status === "proposed" ? "Proposed" : ch.status === "applied" ? "Made" : ch.status}{" "} @@ -137,11 +179,13 @@ function ChangeCard({ ch }: { ch: GraphChange }) { kept={merge && (ch.status === "proposed" ? keep === a.id : ch.keep === a.id)} choose={merge && ch.status === "proposed" && editor ? () => setKeep(a.id) : undefined} /> - setKeep(b.id) : undefined} - /> + {b && ( + setKeep(b.id) : undefined} + /> + )}
Rule
@@ -152,8 +196,20 @@ function ChangeCard({ ch }: { ch: GraphChange }) {
{v ? ( <> - - {v.same === false ? "Different" : "Same"} + + {pair + ? v.same === false + ? "Different" + : "Same" + : ch.kind === "retype" + ? v.agree === false + ? `Keep ${typeName(a.type)}` + : typeName(v.type ?? ch.to) + : v.agree === false + ? "Not noise" + : "Noise"} {" "} · {pct(v.confidence)} {v.why ? ` · ${v.why}` : ""} @@ -172,7 +228,14 @@ function ChangeCard({ ch }: { ch: GraphChange }) { disabledReason={why} onClick={() => act.mutate("accept")} > - {merge ? `Merge, keep “${keep === a.id ? a.name : b.name}”` : "Link them"} + { + { + merge: `Merge, keep “${kept}”`, + link: "Link them", + retype: `Make it ${typeName(ch.to)}`, + hide: "Hide it", + }[ch.kind] + } )} @@ -250,8 +313,8 @@ export function GraphChangesPage({ run }: { run?: number }) { ) : ( } title={status === "proposed" ? "Nothing to review" : "No changes made"}> {status === "proposed" - ? "When a workflow that organises the graph isn’t sure two entities are one thing, it proposes the change here for you to accept or dismiss." - : "Merges and links that workflows made show here, so you can undo them."} + ? "When a workflow that organises the graph isn’t sure of a change (two entities as one, a new type, hiding noise), it proposes it here for you to accept or dismiss." + : "Merges, links, new types and hides that workflows made show here, so you can undo them."} )} diff --git a/nextjs-frontend/components/workflows/node-settings.tsx b/nextjs-frontend/components/workflows/node-settings.tsx index 7ee98c2a4..3f8831608 100644 --- a/nextjs-frontend/components/workflows/node-settings.tsx +++ b/nextjs-frontend/components/workflows/node-settings.tsx @@ -544,11 +544,13 @@ export function NodeSettings({ options={[ { value: "merge", label: "Look-alikes in a namespace (to merge)" }, { value: "link", label: "Look-alikes across shared namespaces (to link)" }, + { value: "retype", label: "Entities whose type looks wrong (to retype)" }, + { value: "hide", label: "Noise: filler words, fragments, one-offs (to hide)" }, ]} /> )} - + {({ id, describedBy }) => ( )} - + {({ id, describedBy }) => ( - + {({ id, describedBy }) => (

How graph workflows run

- A graph workflow starts from the namespaces a routine runs it over. Look-alike entities finds pairs that - may be one thing, Ask the model judges each pair, and Apply changes merges or links the sure ones and - proposes the rest. + A graph workflow starts from the namespaces a routine runs it over. Find what to tidy finds look-alike + entities, entities whose type looks wrong, or noise; Ask the model judges each one, and Apply changes + makes the sure ones and proposes the rest.

Run it from a routine. Proposed changes wait in Routines → Proposed changes, and every change can be diff --git a/nextjs-frontend/components/workflows/workflow-model.ts b/nextjs-frontend/components/workflows/workflow-model.ts index ebcffdd38..6924b9860 100644 --- a/nextjs-frontend/components/workflows/workflow-model.ts +++ b/nextjs-frontend/components/workflows/workflow-model.ts @@ -311,26 +311,26 @@ export const NODES: Record = { }, // Graph workflows (scope graph) candidates: { - label: "Look-alike entities", - describe: - "Pairs of entities that may be one thing, found by rules (same letters, acronym, spelling, sound, one name in the other)", + label: "Find what to tidy", + describe: "Found by rules: look-alike entities to merge or link, entities whose type looks wrong, or noise to hide", icon: SearchCheck, tone: "green", inputs: 1, outputs: ["out"], takes: "namespaces", - gives: "pairs[]", + gives: "candidates[]", group: "Entities", }, llm_judge: { label: "Ask the model", - describe: "Asks the model whether each pair is the same thing, with lines where each name appears", + describe: + "Asks the model about each candidate, with lines where each name appears: one thing or two, its type, or noise", icon: Gavel, tone: "gold", inputs: 1, outputs: ["out"], - takes: "pairs[]", - gives: "pairs[] with a verdict", + takes: "candidates[]", + gives: "candidates[] with a verdict", group: "AI", }, filter: { @@ -346,13 +346,14 @@ export const NODES: Record = { }, apply_changes: { label: "Apply changes", - describe: "Merges or links the pairs it’s sure of and proposes the rest for someone to accept; all can be undone", + describe: + "Makes the changes it’s sure of (merges, links, types, hides) and proposes the rest for someone to accept; all can be undone", icon: CheckCheck, tone: "gold", inputs: 1, outputs: [], - takes: "pairs[]", - gives: "merges · links · proposals", + takes: "candidates[]", + gives: "changes · proposals", group: "Keep", }, }; @@ -406,6 +407,13 @@ export const GROUPS = ["Entities", "AI", "Logic", "Data", "Loops", "Keep"] as co export const KEEPS = ["output", "field", "save_entities", "apply_changes"]; /** A node that keeps something only as it's set up: a classify node that acts on a sure answer. */ export const savesAsSet = (n: WfNode) => n.type === "classify" && Boolean(n.config.apply); +/** What a candidates node looks for. */ +export const CANDIDATE_KINDS: Record = { + merge: "look-alikes in a namespace", + link: "look-alikes across namespaces", + retype: "wrong types", + hide: "noise", +}; /** What a classify node chooses from. */ export const VOCABULARIES: Record = { content_type: "Content types", topic: "The namespace’s topics" }; /** Nodes with a body of nodes inside, opened by double-clicking. */ @@ -843,13 +851,13 @@ export function nodeSummary( return c.field != null ? (names.field(Number(c.field)) ?? `field #${c.field}`) : "Choose a field"; case "candidates": return [ - c.kind === "link" ? "across namespaces" : "in each namespace", + CANDIDATE_KINDS[String(c.kind ?? "merge")] ?? String(c.kind), `${Math.round(Number(c.min_confidence ?? 0.7) * 100)}%+`, `up to ${Number(c.limit ?? 100)}`, ...((c.types as string[] | undefined)?.length ? [(c.types as string[]).join(", ")] : []), ].join(" · "); case "llm_judge": - return `${Number(c.batch ?? 25)} pairs a call${c.model ? ` · ${String(c.model)}` : ""}`; + return `${Number(c.batch ?? 25)} at a time${c.model ? ` · ${String(c.model)}` : ""}`; case "apply_changes": return c.apply_above == null ? "Propose every change" @@ -975,8 +983,8 @@ export function problems( fail("Only the asset’s classification can be acted on."); else if (n.type === "candidates" && !between(c.min_confidence, 0, 1, false)) fail("How sure is a number from 0 to 1."); - else if (n.type === "candidates" && !between(c.limit, 1, 1000, true)) fail("At most is 1 to 1000 pairs."); - else if (n.type === "llm_judge" && !between(c.batch, 1, 100, true)) fail("Pairs a call is 1 to 100."); + else if (n.type === "candidates" && !between(c.limit, 1, 1000, true)) fail("At most is 1 to 1000 a run."); + else if (n.type === "llm_judge" && !between(c.batch, 1, 100, true)) fail("At a time is 1 to 100."); else if (n.type === "apply_changes" && !between(c.apply_above, 0, 1, false)) fail("Sure enough at is a number from 0 to 1."); else if (n.type === "apply_changes" && !between(c.max_apply, 0, 1000, true)) fail("At most is 0 to 1000."); diff --git a/nextjs-frontend/openapi.json b/nextjs-frontend/openapi.json index 25685123c..76873952d 100644 --- a/nextjs-frontend/openapi.json +++ b/nextjs-frontend/openapi.json @@ -47932,19 +47932,42 @@ "type": "string", "enum": [ "merge", - "link" + "link", + "retype", + "hide" ], "title": "Kind" }, "a": { "additionalProperties": true, "type": "object", - "title": "A" + "title": "A", + "description": "the entity changed, or the first of a pair" }, "b": { - "additionalProperties": true, - "type": "object", - "title": "B" + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "B", + "description": "a merge's or link's other entity" + }, + "to": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "To", + "description": "a retype's new type" }, "spaces": { "items": { @@ -48081,7 +48104,6 @@ "id", "kind", "a", - "b", "spaces", "status" ], @@ -48100,6 +48122,19 @@ ], "title": "Keep", "description": "for a merge: the entity to keep (default: the one proposed)" + }, + "to": { + "anyOf": [ + { + "type": "string", + "maxLength": 40 + }, + { + "type": "null" + } + ], + "title": "To", + "description": "for a retype: the type to give (default: the one proposed)" } }, "additionalProperties": false, From b945990c25ec0df62e1db5bdeaae8a3b4d7e7bc8 Mon Sep 17 00:00:00 2001 From: Adeel Ahmad Date: Sun, 18 Oct 2026 07:35:16 +0000 Subject: [PATCH 2/2] Merge origin/main; list the assistant's Lens help (#400) as built Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018Nbf5G7PvMgThj8BbUPGT8 --- README.md | 5 ++--- docs/requirements.md | 1 + 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 4e3d0f7c2..90f92dfc3 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p cost), opened from its header. It reads answers aloud with a speech provider, macOS `say` or StyleTTS2 on this machine (opt-in), any message in one click. Long conversations are summarised before they fill the model's context, and `/clear` and `/compact` start afresh or summarise now. Conversations title themselves after their first answer, unless - you name them. Each namespace can have its own assistant with + you name them. It answers questions about Lens itself (its MCP address, features and settings) from its docs and + this server, cited. Each namespace can have its own assistant with its own name, instructions and memory, answering in Telegram and Slack (built in, with voice notes and files), in Discord, Matrix, WhatsApp and other chat rooms through Matterbridge, and by email at an address of its own that you email or forward things to. ([Chat](docs/assistant.md), [web search](docs/assistant.md#web-search-and-reading-pages), @@ -241,8 +242,6 @@ still loading. ([Components](docs/components.md), These are open pull requests, not yet part of Lens: -- **The assistant** answering questions about Lens itself, from its docs and this server - ([#400](https://github.com/adeelahmad/lense/pull/400)). - **Chat:** a conversation exported as Markdown, PDF or JSON ([#474](https://github.com/adeelahmad/lense/pull/474)). - **Pictures:** low-quality images enhanced before they're read ([#479](https://github.com/adeelahmad/lense/pull/479)). - **Web clipper:** signing in, or using your API token, not only an import webhook's diff --git a/docs/requirements.md b/docs/requirements.md index 976038ba3..f745c3319 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -77,6 +77,7 @@ request, to another check that proves the same thing; loosening a requirement or | A chat assistant on every page | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_asking_from_a_page`; [page-chat.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/page-chat.test.tsx); [ask-place.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/ask-place.test.tsx) | | By voice | [assistant-mode.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/assistant-mode.test.tsx) `starts a voice chat from the mic`; [test_voice.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_voice.py) `test_what_is_said_comes_back_as_text` | | It searches by words, and by meaning once search by meaning is set up | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_tools_approvals_and_checking`; [test_semantic.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_semantic.py) `test_chat_retrieval_by_meaning`, `test_off_or_unconfigured` | +| It answers questions about Lens itself from its docs and this server, cited, in chat and over MCP | [test_lens_help.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_lens_help.py) `test_the_docs_are_found_by_words`, `test_the_assistant_answers_about_lens_itself`, `test_agents_ask_lens_over_mcp` | | Answers cite their sources, and only what the person may read | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_chat_cites_only_what_you_can_read`; [chat.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat.test.tsx) `splits [n], [n, m] and [n][m]` | | It acts with tools, and changes wait for approval | [test_chats.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_chats.py) `test_tools_approvals_and_checking`; [test_approvals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_approvals.py) `test_the_assistant_lists_and_declines_but_never_approves` | | Web search and reading pages, cited like the archive, off until an admin turns it on | [test_web_search.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_web_search.py) `test_an_answer_cites_the_web_and_the_conversation_keeps_it`, `test_reading_pages_and_pdfs`; [chat-web-citations.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/chat-web-citations.test.tsx) |