Repository navigation
docs(audit): shared reader evaluation evidence (#797) - #807
Merged
Merged
Conversation
Spec for evaluating a non-generic extraction of OnDiskReader's read logic against the v3 base and the #772 boxed-backend prototype. Ship only if native saves >= 2,000 bytes with no artifact growth, point families stay within a 1.02 one-sided 95% upper bound (two campaigns, baseline control), and every other workload within 1.05, including validated cold reads, reader creation, concurrency, WASI and BrowserDb. Reviewed over three rounds with Codex; approved 2026-10-30. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2a7QjqyDrSDo1LKTK619q
Size, gate and runtime scripts for comparing the shared-helper reader with the base and the boxed prototype, and the campaign specs and block counts fixed from the pilots before any acceptance run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2a7Qo5yfvBEjzSuUoB2n9
Neither candidate meets the agreed margins, so no production change is proposed. The shared helper saves 3,288 native bytes and passes every correctness gate, but costs about 1-2% on opt-level-3 point queries: the second campaign's upper bounds for attribute/value and :as-of point queries are 1.0245 and 1.0260 against 1.02. The boxed prototype passes the point gate but slows direct reader construction by about 17%. The helper and its entry-point tests are kept as patches, with every campaign's raw records, the sizes, gate status and per-workload tables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2a7Qo5yfvBEjzSuUoB2n9
) The opt-level-z library consumer grows by 48 bytes (H) and 64 bytes (P), so neither candidate meets the "no artifact grows" size bar. The README now says which workloads carry a real result digest; concurrency, which was checked by row counts only, is marked not fully accepted. The cold rows are labelled outstanding, as the residency rule requires. The tables add a validation column, absolute deltas in ns, and n/a for fields the browser and WASI harnesses cannot measure. The browser build's LTO setting is disclosed. The plan's pre-registration commit and timing are recorded in plan-provenance.txt. Author and trailer lines are removed from the patches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2a7Qo5yfvBEjzSuUoB2n9
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2a7QjQLd9pG9qVFdqbo5d
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Evidence package for #797 (child of the #772 size audit). Docs only: no production source changes.
Outcome
Neither candidate passes the acceptance gates fixed in
docs/superpowers/specs/2026-10-30-797-shared-reader-design.md:ReaderCore): saves 3,288 native bytes but fails the primary runtime gate. Envelope point-query upper bounds are 1.0245 (attribute/value) and 1.0260 (:as-of) against a limit of ≤ 1.02. The B:B control stays within 0.9937–1.0022.con_direct). The upper bound is 1.174 native and 1.130 WASI, against ≤ 1.05. It makes two allocations per construction, versus one on base.The patches, harness, raw data, tables and limitations are in
docs/audits/797/. The acceptance plan was committed before the first acceptance block (2313d245b, kept onaudit/797-h-measured; seeevidence/plan-provenance.txt).Evaluated in two TandemKit rounds by Claude Opus and Codex gpt-6-astra. Bench tools, fixtures and temp directories have been removed.
Refs #797
🤖 Generated with Claude Code
https://claude.ai/code/session_01H2a7QjQLd9pG9qVFdqbo5d