Skip to content

feat(evidence): put everything Lens found about a resource in its evidence PDF - #370

Merged
adeelahmad merged 4 commits into
mainfrom
claude/project-thread-b6iq9o
Oct 15, 2026
Merged

adeelahmad merged 4 commits into
mainfrom
claude/project-thread-b6iq9o

Conversation

@adeelahmad

@adeelahmad adeelahmad commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: an evidence PDF held each resource's own file (and, for audio and video, its transcript) and a provenance page. Emotion, speakers, entities, tags, topics, metadata, the summary and every other enrichment were left out.

After: after each resource's file comes <title> - what Lens found.txt, laid out by anytopdf as pages, with every part Lens holds:

  • Details: type, namespace, collection, dates, length or pages, size, language, words, content type, where it came from, status, NSFW classification (where it is on); an email's headers, a web page's address, what a document says about itself (Tika), how its PDF was made
  • Metadata: the effective description (title, description, creators, contributors, subjects, identifiers, rights, Dublin Core terms, statements, access) and custom fields
  • Tags, Topics (accepted, then suggested, with how each came)
  • Summary (labelled as model-written): key points and follow-ups with where each comes from, people, overall tone, importance
  • Analysis (labelled as model-written): every pipeline output, by key, with its model
  • Sections, Speakers (talk time, share, turns, words, pace, how recognised)
  • Emotion and sounds: share of speech time overall and by speaker, sounds heard; each transcript line is also marked like subtitles, Alice: (Happy, Laughter) The invoice is paid.
  • Entities by type with counts, Keywords
  • What's seen: shots, faces, objects, page/shot descriptions, text on screen
  • Places and dates anytopdf found (members of the namespace only, as on the resource page)
  • Notes, highlights and comments: the shared ones, off by default, never anyone's private notes

include (repeatable) picks the parts on GET /resources/{rid}/evidence.pdf and GET /collections/{cid}/evidence.pdf; without it every part but notes goes in. include=file gives the old PDF. In the web app, Export keeps one-click Evidence PDF and adds Choose what goes in it…; a collection's page gets the same picker beside its button. The picker remembers the last picks in the browser and Enter downloads.

Motivation and Context

Adeel asked for the PDF export to include everything (emotion and the rest); it left all enrichments out.

How: domain/evidence_report.py reads what Lens already stored (player data, talk stats, metadata, topics, outputs, enrichment, NSFW classification, notes) into plain text, which anytopdf lays out the same way here and on a conversion node (it renders Markdown and HTML as text too). evidence.make adds that file after each resource's own file and marks transcript lines when emotion is picked. No model calls.

Screenshots (if appropriate):

n/a; the report's text is asserted line by line in tests/api/test_evidence.py.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests.
  • My change requires documentation updates, and I have updated docs/ accordingly.
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Done questions (AGENTS.md)

  • Goal: the evidence PDF holds every enrichment by default, with parts pickable; merged on green CI.
  • Agents: include on both endpoints is a typed enum in OpenAPI, described part by part.
  • Zero setup: same anytopdf path as before, nothing new to install or set.
  • Fewest clicks and keystrokes: Evidence PDF stays one click with everything; the picker is optional, starts from the last picks, Download is focused so Enter downloads.
  • Configurable: n/a, no new setting; parts are picked per download (API and web app).
  • Light: reads stored rows into a text file; no new dependency or process.
  • Private and secure: places only for namespace members; shared notes only when picked, private notes never; NSFW classification only where it is on; threat model §1.8 has an Evidence PDFs row.
  • Cost: no model calls; only reads what is stored.
  • User decides: notes stay out unless picked; the existing one-click export still works, now with what was asked for.
  • Upgrades: n/a, no stored data changes.
  • Honest: model-written parts say so; lists over 1,000 lines say how many more there are; docs describe only what this PR adds.
  • Tests, docs and generated files updated: backend and jest tests, docs/api.md#evidence-pdfs, CHANGELOG, make openapi.
  • Open standards; dependencies keep Lens MIT: no new dependencies.
  • Result reported with evidence. Refine later: an MCP tool for evidence PDFs; the requester's own private notes as an opt-in part; a rendered (HTML) layout where Chromium is present.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Cbw4LbcmWzn27B4ai6tX7C

…dence PDF

After each resource's own file, an evidence PDF now has pages of what Lens
holds about it: details, metadata and custom fields, tags, topics, the
summary and analysis models wrote, sections, speakers, emotion and sounds
(overall, by speaker, and marked on each transcript line), entities,
keywords, what is seen in it, and the places and dates found in it.
Shared notes, highlights and comments go in when picked; private notes
never do. `include` picks the parts in the API, and the web app's Export
menu and collection page open a picker.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cbw4LbcmWzn27B4ai6tX7C
@adeelahmad adeelahmad self-assigned this Oct 15, 2026
Keep both changelog entries and both header imports, recompute the threat
model's prose hash, and add a resource's NSFW classification (where it is on
in its namespace) to the evidence report's details.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cbw4LbcmWzn27B4ai6tX7C
Keep both changelog entries and recompute the threat model's prose hash.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cbw4LbcmWzn27B4ai6tX7C
Keep both changelog entries and recompute the threat model's prose hash.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cbw4LbcmWzn27B4ai6tX7C
@adeelahmad
adeelahmad merged commit 559aeb2 into main Oct 15, 2026
12 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-b6iq9o branch October 15, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants