Skip to content

feat(search): find content similar to a text, a resource, a topic or an entity - #296

Merged
adeelahmad merged 2 commits into
mainfrom
claude/project-thread-5mx3gh
Oct 10, 2026
Merged

adeelahmad merged 2 commits into
mainfrom
claude/project-thread-5mx3gh

Conversation

@adeelahmad

@adeelahmad adeelahmad commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: search found moments that contain a query's words (or, with embeddings, its meaning). There was no way to ask "what else is like this passage, this resource, this topic or this entity".

After: GET /api/v1/similar takes exactly one of text, recording, topic or entity and lists the resources most like it, best first, each with why: up to three cited moments (open at the line or page), the key words it says, and the topics and entities in common. The chat assistant and MCP clients get the same as find_similar.

How: app/domain/similar.py fuses four ranked lists by reciprocal rank: passages close in meaning (only when search by meaning is set up; a resource uses the centre of its own passage vectors, so no model call), the text's rare key words (from the term table) or a resource's keywords OR'ed through the word index, shared topics, and shared entities (rarer ones weigh more). Without an embeddings server, the default install still answers by words, topics and entities, and calls no model.

This is the API for the Similar tab and the selection popover; the web app part follows in the next PR.

Motivation and Context

Adeel asked for similar content as a dedicated tab on resources and in a popover when selecting text anywhere, including concepts, topics and the graph canvas. This PR is the API underneath it.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (ruff, ruff format, mypy clean).
  • I added or updated tests (tests/api/test_similar.py, test_mcp.py).
  • My change requires documentation updates, and I have updated docs/ accordingly (api.md, processing.md#similar-content, mcp.md, assistant.md; DEVLOG refine-later list).
  • CHANGELOG: written with the web app PR, where people see it.

Done questions (AGENTS.md)

  • Goal: the API part; the tab and popover follow in the next PR. Merged on green CI.
  • Agents: GET /similar, assistant find_similar, MCP find_similar.
  • Zero setup: no new prerequisite or setting.
  • Fewest clicks and keystrokes: n/a, API only.
  • Configurable: n/a, no new setting; reuses embeddings.* and search.*.
  • Light: works without embeddings; no new dependency.
  • Private and secure: read-only, through the same namespace and collection checks as search; no new surface for the threat model.
  • Cost: no LLM call; a text costs at most one embedding request (ledgered, cached); a resource none.
  • User decides: read-only, defaults unchanged.
  • Upgrades: n/a, no stored-data change.
  • Honest: every result cites the moments, words, topics and entities behind it; docs describe only what's merged.
  • Tests, docs and generated files updated (make openapi).
  • Open standards; no new dependencies.
  • Result reported in the project thread; refine-later items in DEVLOG.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NSG4xbWi1TYHFmWSUh1CSt

…an entity

GET /similar, and find_similar for the assistant and MCP clients: the
resources most like what was given, fused by rank from passages close in
meaning (when search by meaning is set up), shared key words, shared
topics and shared entities, each citing the moments that show it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NSG4xbWi1TYHFmWSUh1CSt
@adeelahmad adeelahmad self-assigned this Oct 10, 2026
@adeelahmad
adeelahmad merged commit 79b9bb1 into main Oct 10, 2026
8 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-5mx3gh branch October 10, 2026 20:55
adeelahmad added a commit that referenced this pull request Oct 10, 2026
…308)

#296, #291, #285 and #294 merged after the README refresh, so they move from
"In progress" to the feature list, and the web clipper guide's status names
Edge as built.


Claude-Session: https://claude.ai/code/session_01HDx7PPUauyjMn1LZs8JAsS

Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants