Repository navigation
feat(analysis): find entities with spaCy, fetched the first time it's picked - #492
Merged
Merged
Conversation
automations took automation_nodes.provided at import time, but automation_nodes imports automations (through actions), so importing app.domain.workflows first failed with "cannot import name 'provided' from partially initialized module". provided is now looked up when it's called. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014z3r72SB2quWGKbgGt46mZ
… picked Settings > Analysis > Entity extraction > spaCy needed spaCy and a model installed by hand. spaCy is now a component: the `spacy` extra at uv.lock's versions and the model named in analysis.spacy_model as a pinned wheel checked against its SHA-256 (en_core_web_sm, de_core_news_sm or xx_ent_wiki_sm, all MIT), fetched the first time an asset needs it. In spaCy mode the defined entities come first and keep their names and types where they overlap spaCy's; a lone word like "Okay" isn't taken for a name; PER and MISC (the German and multi-language models) map to people and terms. Only spaCy's tokenizer, a sentencizer and its ner run. Until spaCy is there the rules find the entities and the job log says so. The workflows' rules node uses the same extractor. Docs, changelog, dev log and threat model updated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014z3r72SB2quWGKbgGt46mZ
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: Settings → Analysis → Entity extraction → spaCy only worked if someone had installed spaCy and a model by hand; otherwise analysis quietly used the rules. Importing
app.domain.workflowson its own also failed with a circular import.After: picking spaCy fetches it and its small model the first time an asset needs it (about 60 MB): English (
en_core_web_sm), German (de_core_news_sm) or one for many languages (xx_ent_wiki_sm), all MIT. The defined entities still come first and keep their names and types where spaCy overlaps them; a lone word like "Okay" isn't taken for a name; German and multi-language labels (PER, MISC) map to people and terms. Until spaCy is there the rules find the entities and the job's log says so. Workflows' Extract entities (rules) node uses the same extractor.How: a
Spacycomponent installs thespacyextra (versions from uv.lock), then the model wheel pinned incomponents.FILESand checked against its SHA-256, with--no-deps.analyze.entity_finderloads only spaCy's tokenizer, a sentencizer and its NER. The first commit makesautomations.provideda call-time lookup, which breaks the import cycle.Motivation and Context
Adeel asked for "nlp libraries for extracting NER" in the cleanup series (#476 cleanup, #485 punctuation). Dates and grammar follow.
Types of changes
!in the title)Checklist:
make hookschecks it on commit).docs/accordingly.## UnreleasedinCHANGELOG.md.Done questions (AGENTS.md)
PUT /api/v1/settings/analysis(entities,spacy_model), the setup assistant'schange_settings, the components API; entities reach agents through the existing graph, MCP and assistant tools.tests/domain/test_spacy_entities.py(with a fake pipeline, and the realen_core_web_smwhen installed: passed locally with it);docs/processing.md,docs/components.md, CHANGELOG, DEVLOG updated; OpenAPI unchanged.pip_installandfetch_filechecks.Local CI (
ci/local-ci.sh): backend (embedded and server SurrealDB), Next.js, pre-commit and title passed.Generated by Claude Code