Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -603,7 +603,7 @@ Todo:
- [x] API tests
- [x] Web app: Routines page, run history, proposed changes review, graph nodes on the canvas
- [ ] Notify (Matterbridge/webhooks thread) when a run fails or leaves proposals
- [ ] More graph nodes: retype entities, hide noise, cluster topics
- [x] More graph nodes: retype entities, hide noise, cluster topics (the candidates node's retype, hide and cluster kinds)
- [ ] Routing rules (shared IMAP inbox → namespaces) slot in between sync and ingest

## 2026-10-02 · Workflow canvas
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
automation schedule and the backup in one list with when each runs next and last, to switch, re-time or run now, and
agents do the same over the API and MCP. A Classify node in a workflow sorts an asset into labels, content types or
topics, acting only on a sure answer, and an Entity filter node keeps the entities you want. Graph workflows tidy the
entities on a routine: look-alikes to merge or link, types the rules now read differently, and noise to hide, making
the changes they're sure of and proposing the rest, each one undoable. Anything that reaches outside Lens or changes
entities on a routine: look-alikes to merge or link, types the rules now read differently, noise to hide and topics
to group under a broader one, making the changes they're sure of and proposing the rest, each one undoable. Anything that reaches outside Lens or changes
more than what set it off asks first, until you let it run on its own: on the Approvals page or, for a change inside
Lens, with a yes in the chat it told you in; agents can decline an approval but never give one. Each automation has
limits and budgets, and agents make, try and run them over MCP and in the assistant, or draft one from a sentence. An
Expand Down
19 changes: 13 additions & 6 deletions docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -816,13 +816,18 @@ A workflow's scope is assets (above), the graph, or an event: an automation's wo
- **Noise** (hide): entities with no letters or one letter, made of filler words (*Okay so*; not acronyms like
*US*), or a phrase rather than a name (95%; filler words are 70% when the entity has a type other than Term), and
terms said only once (50%, below the default 70%). Entities someone showed again are left alone.
- **Topics to group** (cluster): a namespace's topics that go under a broader topic: those whose labels end in
another topic's label (*Machine learning* and *Deep learning* under *Learning*, 85%), and three or more with no
broader topic that end in the same word, under a new topic for it (*Sales calls*, *Support calls* and *Team
calls* under *Calls*, 70% for three, up to 85%). Topics have no types, so *Only these types* doesn't apply.
Anything already proposed, dismissed or undone is skipped too, so a run never redoes what someone took back.
- **Ask the model** (`llm_judge`) shows the model each candidate with lines where the names appear and asks, a kind
at a time: whether a pair is the same thing and which name to keep, what type a name is (from the namespace's
types), or whether it is noise; and how sure it is. Each gets a `verdict`, with `agree` true when the change
types), whether it is noise, or whether a group's topics go under its broader topic (with a better label for a new
one, and which topics to leave out); and how sure it is. Each gets a `verdict`, with `agree` true when the change
should be made (and, for pairs, `same`).
- **Filter** keeps the candidates that pass a test, e.g. `verdict.agree` (or `verdict.same`) equals true.
- **Apply changes** merges, links, retypes or hides the candidates whose confidence (the model's, else the rules') is
- **Apply changes** merges, links, retypes, hides or groups the candidates whose confidence (the model's, else the rules') is
at least *apply above* (at most *max apply* a run) and proposes the rest. Without *apply above* it only proposes. A
candidate the model said no to is skipped.

Expand All @@ -832,11 +837,13 @@ on, or Run now with *propose only* first to see what it would do.

Every change is recorded (`GET /api/v1/graph-changes`), and shows in Routines › Proposed changes: editors of the
namespaces involved accept or dismiss proposals (dismissing a pair says the two are different; dismissing a type or a
hide leaves the entity as it is; neither is suggested again), choose another type when accepting a retype (`to`), and
undo applied changes; an admin can undo everything a run applied at once (`POST /api/v1/routine-runs/{id}/undo`).
hide or a group leaves things as they are; neither is suggested again), choose another type when accepting a retype
(`to`) or another label for a group's new broader topic (`label`), and undo applied changes; an admin can undo everything a run applied at once (`POST /api/v1/routine-runs/{id}/undo`).
Merges are undone exactly as from an entity's page; a type or a hide someone has changed since stays as they left it.
Agents do the same through the API, and `undo_graph_change` over MCP. To tidy types and noise, add a workflow that
looks for wrong types and noise (no built-in workflow does, so nothing changes until you add one).
Undoing a group takes its topics out from under the broader topic, and deletes that topic when the group made it and
nothing uses it since (a recording, a narrower or related topic).
Agents do the same through the API, and `undo_graph_change` over MCP. To tidy types, noise and topics, add a workflow
that looks for them (no built-in workflow does, so nothing changes until you add one).

### Publishing from the graph

Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ request, to another check that proves the same thing; loosening a requirement or
| A Classify node in workflows chooses from labels, content types or topics, and acting on a sure answer keeps something | [test_classify.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_classify.py) `test_a_workflow_classifies`; [workflow-flow.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-flow.test.ts) `chooses from labels, content types or topics, and acting on a sure answer keeps something` |
| An Entity filter node keeps a list's entities by type, stop list, length, pattern or how often they're said | [test_flow.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_flow.py) `test_entity_filter_keeps_the_entities_asked_for`; [workflow-flow.test.ts](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/workflow-flow.test.ts) `an entity filter says what it keeps and refuses lowercase types` |
| Routines on a schedule | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_routines_are_checked_and_run_when_due`, `test_schedules`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `routine-create`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `routine-create` ≤ 2 clicks |
| Graph workflows find look-alikes to merge or link, wrong types and noise, make the sure changes and propose the rest, each undoable and not made again once taken back | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_graph_workflows_organise_entities`, `test_rules_read_types_and_noise`, `test_graph_workflows_retype_and_hide`; [graph-changes.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/graph-changes.test.tsx) `say a type change in words and make it` |
| Graph workflows find look-alikes to merge or link, wrong types, noise and topics to group, make the sure changes and propose the rest, each undoable and not made again once taken back | [test_routines.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_routines.py) `test_graph_workflows_organise_entities`, `test_rules_read_types_and_noise`, `test_graph_workflows_retype_and_hide`, `test_graph_workflows_group_topics`; [graph-changes.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/graph-changes.test.tsx) `say a type change in words and make it`, `show a group of topics under a new broader one` |
| Everything that runs on a clock (routines, automations' schedule and at triggers, the backup) in one Schedules list, switched, re-timed and run from it, by people and by agents over the API and MCP | [test_schedules.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_schedules.py) `test_one_list_holds_every_routine_automation_schedule_and_the_backup`, `test_schedules_are_changed_switched_and_run_from_the_list`, `test_agents_list_change_and_run_schedules_over_mcp`; [schedules-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/schedules-page.test.tsx) `lists every routine, automation schedule and the backup, each linking to its own page`, `switches one off, runs one now and changes when one runs, from the list` |
| Batch runs over a collection | [test_batches.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_batches.py) `test_collections_and_batches` |
| Notifications to webhooks, Slack, Discord and Matterbridge | [test_notifications.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_notifications.py) `test_owners_manage_targets_and_a_test_is_signed`, `test_runs_that_end_are_sent_once_to_matterbridge` |
Expand Down
3 changes: 2 additions & 1 deletion docs/topics.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ a topic to edit it in a drawer: its labels, definition, broader and related topi
assets about it, where suggested ones can be accepted or removed. **Merge** in the drawer opens a find box: type
part of a label and pick (or press Enter) to merge that topic into this one at once. **Delete** deletes it at once.
Both say what they did with **Undo** on the message, so there is no confirm step. Editors add topics at the top or
under another with **+**.
under another with **+**. A graph workflow can also group a flat vocabulary under broader topics on a routine,
proposing or making the groups it finds ([Organising the graph](processing.md#organising-the-graph)).

An asset's **Entities** tab starts with its topics: editors add one from the vocabulary, take one off, or accept
or dismiss a suggestion. An entity of type Term has **Make a topic** in its drawer. On the graph canvas topics are
Expand Down
2 changes: 1 addition & 1 deletion fastapi_backend/app/api/v1/routes/routines.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ def list_graph_changes(
def accept_graph_change(cid: int, body: GraphChangeAccept, user: Writer, acl: Acl, db: Db) -> Ok:
_change(db, acl, cid, "curate")
with domain_errors():
organize.accept(db, cid, user.email, body.keep, body.to)
organize.accept(db, cid, user.email, body.keep, body.to, body.label)
auth.audit(db, user.as_audit(), "graph_change.accept", f"graph_change:{cid}")
return Ok()

Expand Down
Loading
Loading