Bench: timing and release sign-off for JVM, Android, Swift/iOS and WASI #839

Follow-up of #776: X04 in the remaining exclusions and follow-ups table, finalized in #834.

Disposition: MANUAL binding checks — completed and accepted on 2026-11-05.

Milestone: v3.0.0 — Distinct host-runtime evidence and explicit manual release sign-offs are complete for the approved scope.

Evidence and both maintainer sign-offs accepted (2026-11-05)

Core PR #852 now retains three series each for JVM, Android, Swift/iOS and WASI: 45 cases, 1,350 samples and 135 case/run summaries. All samples, hashes and summaries were validated. Physical iOS oracle/smoke and all three timing series passed; exact-result checks reject silent errors. Companion harness PRs and their build/correctness CI are linked from the evidence PR.

Core evidence PR #852 merged on 2026-11-05 as cdad3e7270d0e5e791cd48f47a4ed35ff50df6a4. All PR checks passed, including the retried macOS correctness job. The merge message contains this acceptance text:

Accepted PR #852’s benchmark evidence, documented variability and Device Farm limitations, and all proposed API/lifecycle exclusions.

On 2026-11-05, the requesting maintainer explicitly confirmed that the merge-message acceptance is to be recorded in both binding-maintainer and release-maintainer roles (reply: “yes, record it”). Both sign-offs are therefore recorded for evidence revision 10ff8003b86c68697201662dd51999a61bc4b20c, merged by #852, including its documented variability, Device Farm model/OS limitations, and all proposed API/lifecycle exclusions in the dispositions. This supersedes the pending-sign-off status in the pre-merge report. Source-build versus package scope, timing boundaries, iOS filter/file-attachment deviations and artifact retention remain documented limitations; acceptance does not turn excluded costs into measurements.

All companion harness PRs were verified merged on 2026-11-05:

• JVM #21 — 8490cb41686e9fa3ab1ac18177aadd4cad4963ce.
• Android #17 — 5bb49dc37038679775aecfdd1749b37cdb6a1576.
• Swift/iOS #16 — 12ca81102227cf84eb813894049c685093784791.

Core evidence/WASI and all three companion harnesses are merged. Both maintainer sign-offs and acceptance of every proposed exclusion are recorded above. All acceptance criteria are satisfied; #839 is complete. Original benchmark artifacts remain retained, and the remote bench scratch was cleaned after verified archival.

Approved measurement scope (2026-11-04)

New timing evidence published for this issue must come only from AWS Device Farm or the dedicated bench machine:

• AWS Device Farm: physical Android and Swift/iOS devices.
• Dedicated bench machine: Java/JVM, WASI and applicable other host/browser API-cost measurements.
• macOS timings are explicitly excluded from this issue because no sufficiently stable, repeatable macOS benchmark platform is available. A macOS timing run is not required to close this issue.
• GitHub-hosted runners may build packages and run correctness checks, including preparing iOS test artifacts, but their timings do not qualify as new benchmark evidence for this issue.

This restriction applies only to #839's measurement and publication scope. Existing nightly benchmark workflows on GitHub-hosted runners continue unchanged, including their normal timing collection and publication. No nightly workflow is disabled or migrated by this decision.

This is the user-approved platform scope. The separate binding/release maintainer sign-offs are recorded above; the platform decision alone did not supply acceptance. Device Farm model/OS selection does not guarantee the same physical unit; record that limitation, the actual available device conditions and repeatability evidence for maintainer assessment. Retain historical CI evidence as historical context; do not mix it into the accepted platform histories or remove it retrospectively.

Original scope recorded by the audit (historical context; measurements and accepted dispositions are recorded above):

JVM/JNA, Android ART/ABI/device storage, Swift ARC/iOS storage and WASI need distinct runtime evidence/release sign-off. Options/error paths, foreign FactLog/LogWriter conversion, foreign preparation/registration latency and browser pinned-reader churn remain unmeasured. Python shared UniFFI coverage does not measure other host runtimes; no binding RSS bound is asserted.

C32/C34/C36 and Appendix B establish 26 foreign-runtime IDs across Python (six), browser (eight), Node (six) and C (six), not timing for every binding. Core measured revision was 0394c90ad90e9b95d416f4a0ec4a4c78b16e5b57; source builds patched the binding to that checkout and retained Cargo metadata, not proof of published package artifacts.

Baseline shortlist suffixes are prepared_execute/1k, cursor_batch_1/1k, cursor_batch_256/1k, set_roundtrip/2, predicate/1k, aggregate/1k; browser also has indexeddb_write/1k and indexeddb_reopen/1k. Fixtures are 1k deterministic scalar facts plus a two-element set. Timed work includes host/JSON result conversion and full cursor draining. Fixture creation, query preparation and callback registration are untimed. Browser reopen is within the same page/context, not cold startup.

Runtime evidence required by the original audit (now measured and accepted above):

• Java/JVM: JNA crossing, JVM allocation and JVM callback costs on the dedicated bench machine. Python's shared UniFFI adapter does not measure these host costs.
• Android: ART, ABI and storage timings on physical AWS Device Farm devices; hosted emulators are not substitute evidence. Record model/OS/ABI and assess repeatability and device-selection limitations.
• Swift/iOS: ARC, callbacks and device-storage timings on physical AWS Device Farm iOS devices; Linux UniFFI execution is insufficient. macOS timing is excluded by the approved platform scope above.
• WASI: separate runtime and REPL startup overhead on the dedicated bench machine. The audited WASI package has no matching foreign prepared-object or synchronous host-callback API; do not invent equivalent cases.

API/persistence timing identified by the original audit (now measured or explicitly excluded with sign-off above): options validation and host error conversion, foreign FactLog/LogWriter conversion, foreign preparation and callback registration, and browser pinned-reader churn. Verify option availability per binding: the audited UniFFI/Node/C options expose read-only and query-memory limits but not core transaction_chunk_facts. Browser migration is coordinated with X03. Deliberate callback failures remain correctness tests (X08); no binding RSS bound is established here.

Why outside the nightly shortlist: runtime/device/ABI-specific timing needs distinct stable environments and manual sign-off, and the 26 fixed cases intentionally omit these APIs and lifecycle costs. Related #824 adds transaction APIs; it does not cover this timing/sign-off scope.

Acceptance criteria:

• Publish distinct JVM, Android, Swift/iOS and WASI timing evidence from the approved platforms above, with exact core/binding/build/runtime revisions, recorded hardware/device/ABI conditions and repeatability evidence, plus explicit sign-off from the applicable binding maintainer and release maintainer.
• Measure applicable prepared execution, cursor batches, set conversion and callbacks per host, and WASI startup/runtime under its actual REPL contract; identify unavailable APIs explicitly.
• Measure or explicitly disposition each remaining options/error-conversion, FactLog/LogWriter, preparation/registration and browser pinned-reader cost with binding maintainer and release maintainer sign-off and rationale.
• Verify results with counts/sums/exact sets, separate setup from timed work, and separate source-build evidence from published-package verification.
• Retain raw evidence and runtime-specific histories without merging native or other-host latency series; state that binding RSS bounds remain unverified unless independently measured.

Run heavy builds, enumeration, measurements and comparisons on idle hardware; retain reproducible evidence and clean temporary scratch afterward. This is benchmark/verification work aligned with PHILOSOPHY.md's reliability, portability and simplicity principles; any production change needs its own review.

Evidence: docs/bench-coverage-v3.md; validation run 41642356471. The validation run establishes only its recorded representative latency coverage; it does not complete this follow-up.