Skip to content

Evaluate shared reader code without regressing point lookups #797

Closed as not planned

Description

@adityamukho

Follow-up to #772. OnDiskReader<B> repeats read logic for file and memory backends even though lower tree/cache primitives already use dyn StorageBackend. A disposable prototype stores a Box<dyn StorageBackend> containing the existing MutexStorageBackend<B> and makes the reader itself non-generic.

At v3 baseline 3d19abbce5dd8d5244641ff0d7ba8bad5424d07f, Rust 1.99.1, the audit lock and unchanged profiles:

Artifact Before Prototype Saved
Default native CLI 1,263,592 1,256,144 7,448
WASI 879,785 879,687 98
Packaged browser Wasm 994,445 993,809 636
Stripped opt-level-3 library consumer 2,161,280 2,140,888 20,392

The cost is one extra allocation per reader plus backend trait-object dispatch. Native savings do not transfer directly to single-backend builds.

Runtime evidence is conditional, not a no-regression claim. Four socket-swapped pairs for each of 17 warm-read shapes at 100K facts preserved expected/equal row counts in all 68 comparisons. Most median p50 changes were within 2.3%, but the short point probe initially showed +5.08%. A focused 32-pair rerun of three short workloads gave median changes of +3.91% point, +0.84% attribute/value, and +2.02% as-of; individual short-run noise remains large. The fixture was in bench tmpfs, so this does not measure fsync or cold disk performance.

The prototype passed the two existing reader-unit tests, native/browser/WASI builds, wasm-bindgen packaging, and cargo +1.89.0 check --all-features --locked. That is not sufficient coverage to ship a reader refactor.

Deliver a small evaluated change only if its tradeoffs are acceptable:

  • Compare this prototype with extracting shared non-generic read helpers, which may retain specialization without another allocation.
  • Resolve the point-lookup slowdown with longer controlled runs and measure reader creation, cold/cache-disabled reads, concurrent queries, cursor reuse and browser/WASI costs.
  • Preserve lock lifetimes, cache ownership, error ordering, UUID batching, temporal visibility, snapshot pinning and all public API/format behavior.
  • Run the full relevant storage/query/transaction/fault/concurrency suites, platform and coverage gates on the bench before an implementation PR is accepted.
  • Keep the existing size cap, coverage floors, MSRV and portability. Reject the change if reliable performance is compromised for the size saving.

Activity

  1. added this to the v3.0.0 milestone on Oct 29, 2026
  2. adityamukho commented on Nov 3, 2026

    @adityamukho
    CollaboratorAuthor

    Both candidates failed the gates; evidence in #807 (docs/audits/797/).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceBenchmark performance regressions

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions