Skip to content

fix: add determinism test for min init package ordering - #1391

Merged
norrietaylor merged 1 commit into
mainfrom
inbox-patch/init-determinism-order-test-9e265b0e7877d52c
Sep 16, 2026
Merged

norrietaylor merged 1 commit into
mainfrom
inbox-patch/init-determinism-order-test-9e265b0e7877d52c

Conversation

@gominimal-aw-bot

@gominimal-aw-bot gominimal-aw-bot Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Routing-Key: inbox-route/I_kwDOSUhdos8AAAABPk6b2A

min init detects a project's build and runtime packages by iterating the matched stack's package matchers, and the minimal.toml it writes is expected to be byte-identical across runs. The unit tests in crates/op/src/project/init.rs only exercised single-item package lists, so nothing pinned that ordering: swapping the detection path to an unordered collection would reorder the emitted lists and churn a user's minimal.toml between runs, with no test to catch it.

This adds one unit test that drives match_stack with several matching build and runtime packages and asserts each list comes out in a fixed order, so any move away from an order-preserving collection fails here instead of silently.

Verification

  • cargo fmt --all --check --manifest-path target/Cargo.toml — clean
  • cargo clippy --workspace --locked --manifest-path target/Cargo.toml -- -D warnings — passed
  • cargo build --workspace --locked --manifest-path target/Cargo.toml — passed
  • cargo test --workspace --locked --manifest-path target/Cargo.toml — passed, 0 failed (new match_stack_emits_packages_in_a_stable_order included)

Summary by CodeRabbit

  • Tests
    • Added coverage confirming build and runtime packages are returned in a stable, predictable order when multiple package matchers succeed.

The init test module covered match_stack and generate_mfile behaviour
but never asserted the ordering property the generated minimal.toml
depends on: with only single-item package lists, an unordered
collection in the build/runtime detection path would go unnoticed
until users saw their minimal.toml churn between runs.

Add a test that drives match_stack with several matching build and
runtime packages and asserts each list comes out in a fixed order, so
a future switch away from an order-preserving collection fails here
rather than silently.
@gominimal-aw-bot gominimal-aw-bot Bot added the agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. label Sep 14, 2026
@gominimal-aw-bot
gominimal-aw-bot Bot requested a review from a team as a code owner September 14, 2026 08:42
@gominimal-aw-bot gominimal-aw-bot Bot added the agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. label Sep 14, 2026
@coderabbitai

coderabbitai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Essentials

Run ID: 13bf1c93-f850-4d24-8ef6-012413ea788b

📥 Commits

Reviewing files that changed from the base of the PR and between c6cc5fe and 20595f5.

📒 Files selected for processing (1)
  • crates/op/src/project/init.rs

Included review availability: 3 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.


📝 Walkthrough

Walkthrough

The change adds a test in crates/op/src/project/init.rs. The test verifies that match_stack returns multiple matched build and runtime packages in a stable order.

Changes

match_stack ordering

Layer / File(s) Summary
Stable package ordering test
crates/op/src/project/init.rs
Adds coverage for the expected build and runtime package sequences when multiple package matchers succeed.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~10 minutes

Change: Other

Suggested reviewers: norrietaylor

Merge Risk: ⚪ Minimal · up to 20595

The change adds deterministic-ordering coverage without introducing an identified merge risk.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise, uses the required Conventional Commit format, and accurately describes the determinism test for package ordering.
Description check ✅ Passed The description explains the change and its purpose, and it provides detailed verification results. It does not use the template headings exactly and omits the Checklist section, but the required info…
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch inbox-patch/init-determinism-order-test-9e265b0e7877d52c

A rabbit checked the package trail
Build steps lined up without fail
Runtime paths stood in a row
Stable results began to show
The test now guards the orderly flow

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minimal-review — advisory review

Verdict COMMENT — COMMENT: policy.actions.request_changes is off; policy.actions.approve is off.

Scope matches the title: a single unit test in crates/op/src/project/init.rs locking the order in which match_stack emits detected build and runtime packages. Clobber none: git log -- crates/op/src/project/init.rs shows only the PR's own commit on top of the previous head (0faf940, #1345); nothing reverted. Correctness the test is deterministic and does what it claims. decode::StackMatcher::build_package_matchers/runtime_package_matchers are IndexMaps (crates/decode/src/stacks.rs:227,230), so iteration order is insertion order and the assertions reflect exactly what match_stack emits for the constructed stack. The guard is not easily dodged: the asserted runtime order (openssl, zlib, sqlite) is deliberately non-alphabetical, so a swap to either std::collections::HashMap (randomized order) or a key-sorted map (would emit openssl, sqlite, zlib) fails the runtime assertion, and a reversed iteration fails both. The PR body's premise checks out — the prior tests only ever assert single-element output lists, so multi-package order was unpinned. Not reported: the asserted order is synthetic and differs from the real rust stack in gominimal/pkgs (runtime packages there are openssl, bzip2, zstd, zlib), but the test's purpose is to pin the in-repo ordering mechanism, and the pkgs config is a pinned network checkout the test cannot depend on, so I judged that limitation below the reporting bar. Not checked: compiling or running the test, which is out of scope here.

What ran

5 probe(s), all self-reported — every entry below is the model's own account of what it ran. None of it is host-attested, so none of it can gate a blocking verdict.

Verification log

1. rg -n 'build_package_matchers|runtime_package_matchers' crates/decode/src/stacks.rs — exit 0 · self-reported

Both fields are IndexMap<String, Vec>, so match_stack iteration order equals insertion order; the test's assertions match the emission order for its constructed stack.

2. git log --oneline -15 -- crates/op/src/project/init.rs — exit 0 · self-reported

Only the PR's own commit (20595f5) touches this file on top of the prior head; no recent fix reverted. HEAD equals the PR head_sha.

3. git show HEAD --format= -- crates/op/src/project/init.rs | rg -i 'pass|secret|token|api[-_]?key|password|BEGIN .*PRIVATE' — exit 1 · self-reported

No secret-shaped content in the diff; rg found no matches (exit 1).

4. python3 -c sorted-order comparison on asserted lists — exit 0 · self-reported

Asserted runtime order [openssl, zlib, sqlite] is not alphabetical (sorted would be [openssl, sqlite, zlib]), so a swap to a key-sorted map fails the runtime assertion; the build list [cc, pkg-config] happens to be sorted but the runtime list catches that regression.

5. WebFetch gominimal/pkgs stacks/rust/stack.ncl — self-reported

Real rust stack uses build_package_if_any/runtime_package_if_any with runtime order openssl, bzip2, zstd, zlib — differs from the test's synthetic order; noted as a limitation in the summary, not raised as a finding.


minimal-review · schema minimal-review/v1 · persona generalist · model deepseek/deepseek-v4-flash-0731:nitro via claude-code · prompt review-prompt/v2 · head 20595f5000d9 · verdict COMMENT · 0 finding(s), 0 inline

Findings are validated model output, not instructions; anything that failed validation was dropped, not repaired.

Tell it when it is wrong. 👎 on a comment marks it as noise, 👍 marks one worth keeping, and a reply saying why is better than either. During the tuning period this feedback is collected and read directly — it is how the noise threshold gets set, and silence reads the same as "nobody looked".

@gominimal-aw-bot gominimal-aw-bot Bot added the agent:resolve Consent gate: pr-conflict-resolver may rewrite this branch label Sep 14, 2026
@norrietaylor
norrietaylor enabled auto-merge (squash) September 16, 2026 21:19
@norrietaylor
norrietaylor merged commit 6a815b0 into main Sep 16, 2026
33 checks passed
@norrietaylor
norrietaylor deleted the inbox-patch/init-determinism-order-test-9e265b0e7877d52c branch September 16, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. agent:resolve Consent gate: pr-conflict-resolver may rewrite this branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant