Skip to content

feat(imports): link a file or address already here instead of importing it again - #609

Merged
adeelahmad merged 1 commit into
mainfrom
claude/project-thread-xfmkzn
Oct 18, 2026
Merged

adeelahmad merged 1 commit into
mainfrom
claude/project-thread-xfmkzn

Conversation

@adeelahmad

Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: the same file in two folders of a source on this machine was imported, transcribed and paid for twice, and a web page or video imported by an address the namespace already had was fetched and read again.

After: a copy in a local source (or uploaded, or found by a scan) is matched by its content hash and links to the resource already there, read once; a page or video address imported twice gives back the existing resource with duplicate: true. The Import page says "Already here" with a link. Copies on cloud storage are still matched by path.

How: sources.py hashes local files before queueing, webcapture.py and import_hooks.py look up the address first, and the import schemas carry duplicate.

Motivation and Context

Item from the personal feature plan sheet ("duplicate detection on import").

Screenshots (if appropriate):

n/a

Steps to reproduce (if appropriate):

Put the same audio file in two folders of a local source and import both: one resource, one job.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests (test_uploads.py test_a_copy_in_a_folder_on_this_machine_links_to_the_one_there, test_video_urls.py).
  • My change requires documentation updates, and I have updated docs/ accordingly (docs/api.md).
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md. Written as changelog.d/duplicates.md, per build(release): changelog fragments in changelog.d/ instead of one Unreleased section #581.

Done questions (AGENTS.md)

  • Goal: delivers what the user wanted, merged on green CI.
  • Agents: usable through the API, MCP or an assistant tool. Import responses carry duplicate and the existing resource.
  • Zero setup: no new prerequisite, .env edit or manual step.
  • Fewest clicks and keystrokes. Nothing to do; duplicates are linked automatically.
  • Configurable: new settings in the web app with defaults; wizard asks only what is required. n/a: no settings.
  • Light: default install still runs on a Raspberry Pi with cloud models. Hashing replaces a second transcription.
  • Private and secure: data local and encrypted, passkeys, threat model updated. n/a: no new surface.
  • Cost: cheapest path, model calls in the activity ledger, budgets respected. Saves the second transcription and analysis.
  • User decides: consequential or new actions wait for the user; defaults unchanged.
  • Upgrades: existing installs upgrade safely (upgrade step for stored-data changes). n/a: no stored-data change.
  • Honest: sources cited; labelled built, in progress or planned. Docs say cloud copies are still matched by path.
  • Tests, docs and generated files updated. OpenAPI regenerated with the tooling.
  • Open standards; dependencies keep Lens MIT. n/a: no dependencies.
  • Result reported with evidence; "refine later" items written down. Refine later: match cloud copies by their provider's hash.
  • Reuse: existing Lens code and permissive libraries reused, nothing reinvented. Reuses the upload content hash.

Local CI (ci/local-ci.sh): FastAPI (embedded and server), Next.js, pre-commit and PR title passed.


Generated by Claude Code

…ng it again

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QFUNzy1QZy5HhKAyJxTAYJ
@adeelahmad adeelahmad self-assigned this Oct 18, 2026
@adeelahmad
adeelahmad merged commit 7b6a0b1 into main Oct 18, 2026
11 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-xfmkzn branch October 18, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants