Repository navigation
docs,ci: land the CI strategy and test-extension contract - #742
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 53 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThis change adds CI JUnit report publication and documents the repository’s CI architecture, test tiers, platform requirements, build and cache model, workflow constraints, and test-extension conventions. ChangesCI strategy and test reporting
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Nextest
participant JUnitReport
participant CoreTestsAction
participant GitHubStepSummary
Nextest->>JUnitReport: write CI test results
CoreTestsAction->>JUnitReport: parse aggregate attributes
CoreTestsAction->>GitHubStepSummary: append Markdown summary
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
b217938 to
9afbcc6
Compare
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
CONTRIBUTING.md (1)
18-24: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winKeep the validation command consistent with the repository contract.
The documented Clippy command omits
--allow-dirty --fix, unlikeCLAUDE.mdand the required repository command. This can cause contributors to skip the mandated fix pass.Proposed fix
- cargo clippy --all-targets -- -D warnings + cargo clippy --allow-dirty --fix --all-targets -- -D warningsAs per coding guidelines, files matching
**/*must runcargo fmt && cargo test -- --include-ignoredandcargo clippy --allow-dirty --fix --all-targets -- -D warnings.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@CONTRIBUTING.md` around lines 18 - 24, Update the Clippy validation command in the “Before opening a PR” checklist to include the required `--allow-dirty --fix` flags, preserving the existing `--all-targets -- -D warnings` arguments and keeping the documented command consistent with the repository contract.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CONTRIBUTING.md`:
- Around line 46-47: Update the integration harness discovery documentation to
exclude *_root_integration binaries from the generic binary(/_integration$/)
filter, and document a separate binary(/_root_integration$/) match for the
privileged lane. Apply the same correction to the duplicate CI strategy guidance
in docs/ci-strategy.md.
In `@docs/ci-strategy.md`:
- Around line 302-306: Update the branch-protection guidance in the CI strategy
documentation to match the repository’s actual required-check topology: verify
whether the five lane aggregators or only ci-ok is required, then revise the
conflicting statements consistently throughout the document while preserving the
described preflight and join-job behavior.
- Around line 48-53: Update the tiered fan-out paragraph to limit archive
creation and replay claims to the macOS and Linux-KVM lanes, which use cargo
nextest run with --archive-file. Explicitly note that core-tests and the
Linux-native, CI, release, and nightly lanes still run cargo nextest run
directly, and remove the claim that every downstream test job reuses archived
artifacts.
- Around line 157-160: Update the “Tagging” guidance to explicitly use cargo
nextest run for the default and VM test commands, since default-filter and the
referenced filtersets apply only to nextest. Preserve the existing VM test
exclusion and explicit selection behavior, and clarify that plain cargo test
does not honor this filtering.
- Around line 361-376: Update the “Local/CI parity: cargo xtask” section to use
the repository’s existing justfile and scripts/ extension points instead of
cargo xtask commands or tests/smoke/ paths. Replace the example commands and
test-location guidance with the actual documented commands and directories used
by this repository, while preserving the goal of keeping local and CI execution
aligned.
- Around line 446-452: Clarify the naming-convention sentence so the per-crate
meta-test applies only to VM harness candidates, not every top-level file under
crates/<crate>/tests/. Preserve the existing suffix requirements and allowlist,
while distinguishing ordinary integration tests from harness files.
---
Outside diff comments:
In `@CONTRIBUTING.md`:
- Around line 18-24: Update the Clippy validation command in the “Before opening
a PR” checklist to include the required `--allow-dirty --fix` flags, preserving
the existing `--all-targets -- -D warnings` arguments and keeping the documented
command consistent with the repository contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 60194908-af2c-4c40-b30d-1a21e3fd1d4e
📒 Files selected for processing (5)
.config/nextest.toml.github/actions/core-tests/action.ymlCLAUDE.mdCONTRIBUTING.mddocs/ci-strategy.md
9afbcc6 to
864d0c3
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CONTRIBUTING.md`:
- Line 45: Update the Doctest row in CONTRIBUTING.md to use the workspace-wide
command cargo test --workspace --doc, matching CI and covering doctests across
all crates.
In `@docs/ci-strategy.md`:
- Around line 488-493: Correct the local-proof artifact claim in the “Provide
proofs” section: either document the local nextest invocation that produces
JUnit XML, or remove JUnit XML from the artifacts attributed to just up and
scripts/session-e2e.sh. Keep the existing CI artifact description accurate and
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2f22772d-3d61-4943-bfd6-8d687914bad7
📒 Files selected for processing (5)
.config/nextest.toml.github/actions/core-tests/action.ymlCLAUDE.mdCONTRIBUTING.mddocs/ci-strategy.md
🚧 Files skipped from review as they are similar to previous changes (2)
- .github/actions/core-tests/action.yml
- .config/nextest.toml
Documents the frozen-CI contract for contributors and agents (N8): - CONTRIBUTING.md — the test-extension table (unit / in-process integration / doctest / *_integration.rs harness / scripts/ e2e) with the discriminator between an integration harness and a full-CLI e2e script; adding a test never requires a workflow edit. - docs/ci-strategy.md — the full CI design, with a preamble mapping the strategy's terms to this repo (five lane workflows; justfile + scripts, not xtask; _integration binary-suffix filtersets, not a vm_e2e_ test-name prefix) and the section 10 contract table updated to the shipped convention. - CLAUDE.md — a pointer so agents inherit the "do not edit .github/workflows/" rule. - .config/nextest.toml + core-tests — JUnit output plus a $GITHUB_STEP_SUMMARY pass/fail table, so per-PR test evidence is visible without artifact downloads. Reconciled against what already merged: the nightly tier, soak script, and promote timeout landed with #720, and a repo-wide CODEOWNERS landed with #728 (already gating .github/ via the minimalists team), so the parked branch's copies of those are dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
864d0c3 to
b948df9
Compare
Lands the CI-strategy documentation and the test-extension contract for
contributors and agents — the final docs piece of the CI migration
(#687). Docs-mostly, plus a
small test-evidence plumbing addition.
What
CONTRIBUTING.md— the contract: adding a test never requires a CI edit.A table maps each test kind to where it lives and how the lane finds it, with
the discriminator between an integration harness and a full-CLI e2e
script: does it drive the
minimalCLI through the whole system?tests/*.rs////crates/<crate>/tests/*_integration.rs(*_root_integration.rsif it needs root)binary(/_integration$/)— no CI editscripts/script (e.g.session-e2e.sh)docs/ci-strategy.md— the full CI design, with a preamble mapping thestrategy's terms to this repo (five lane workflows;
justfile+scripts/,not
xtask;_integrationbinary-suffix filtersets, not avm_e2e_prefix).The §10 contract table is the shipped convention.
CLAUDE.md— a pointer so agents inherit the "do not edit.github/workflows/" rule..config/nextest.toml+core-tests— JUnit output and a$GITHUB_STEP_SUMMARYpass/fail table, so per-PR test evidence is visiblewithout downloading artifacts.
Reconciliation
This supersedes the older parked
ci/docs-contractbranch, which had drifted:nightly-tests.yml,soak-session-e2e.sh, and the promote-timeout alreadymerged with #720 — dropped.
CODEOWNERSmerged with #728(
* @gominimal/minimalists), which already gates.github/. The parkedbranch's
/.github/ @two-peoplerule would have narrowed ownership, so it'sdropped — the freeze is already enforced.
_integrationnaming that#732 shipped, not the old
_e2edraft.Sequencing
Per the plan, this pairs with the ruleset flip (which turns on code-owner
review that this contract references); the flip is gated on the current soak
re-maturing. This PR itself is docs-only and can merge independently.
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation