Skip to content

ci: enforce named v3 acceptance tests and RSS evidence (#837) - #849

Merged
adityamukho merged 2 commits into
mainfrom
ci/837-acceptance-route-main
Nov 4, 2026
Merged

adityamukho merged 2 commits into
mainfrom
ci/837-acceptance-route-main

Conversation

@adityamukho

@adityamukho adityamukho commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The default-branch smoke workflow currently accepts successful Cargo commands even when they run zero intended tests or skip RSS assertions. Require the exact v3 acceptance inventory and measured RSS records. Run memory-sensitive suites serially and retain Cargo.lock with acceptance output.

Verification runs on a fresh runner that checks out only the workflow revision and reads current-run artifacts as data. This prevents the selected source from overwriting verifier code and resolves the two CodeQL cache-poisoning findings on the previous layout. Cache access remains disabled.

Closes #837. The prerequisite tests in #848 and historical audit in #847 are merged into v3. #850 mirrors the verifier isolation on v3.

Type of change

  • Documentation / tests only

Validation

  • Eight Python regressions passed, covering empty/partial/renamed/skipped output and missing RSS evidence.
  • The isolated job command accepted all five retained outputs and rejected missing output and Python text posing as output without executing it.
  • Smoke workflow and verifier match the v3 counterpart; main's existing lint branch filters are preserved.
  • Run 41788127874 passed the long-haul smoke suite, all five acceptance suites and the fresh verifier job. It used workflow 66962f56ddc75303eebe247159d915afa4420a23 and source a2ac9545be7ee9bb62ca2d50ccd7b8bd6241ee3f; the verifier confirmed ten passing tests and nine measured RSS records.
  • CodeQL passed at 66962f56d with no new alerts.
  • Original measurements and raw evidence are retained on v3 in docs/benchmarks/837-acceptance-gaps.md and its evidence directory; ci: isolate acceptance verification on a fresh runner #850 records the isolation validation.

Notes for reviewer

The exact inventory intentionally fails against older v3 sources. The verifier explicitly rejects failed acceptance jobs and missing artifacts. The dedicated dispatch validates the proposed workflow; scheduled activation follows merge. No main runtime, API, file format, or release changes.

@codecov

codecov Bot commented Nov 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@adityamukho
adityamukho merged commit e7923eb into main Nov 4, 2026
27 of 28 checks passed
@adityamukho
adityamukho deleted the ci/837-acceptance-route-main branch November 4, 2026 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bench: verify v3 memory and scaling acceptance execution

1 participant