Skip to content

fix: fix(op): stop the cache clean deleting never-read entries on write time - #3501

Open
gominimal-aw-bot[bot] wants to merge 1 commit into
mainfrom
inbox-patch/fix/cache-clean-never-read-entries-f9b71ddddd2a45cf
Open

gominimal-aw-bot[bot] wants to merge 1 commit into
mainfrom
inbox-patch/fix/cache-clean-never-read-entries-f9b71ddddd2a45cf

Conversation

@gominimal-aw-bot

@gominimal-aw-bot gominimal-aw-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Routing-Key: inbox-route/I_kwDOSUhdos8AAAABgGpBtg

The cache clean deleted entries with no recorded read regardless of age.
Writing an entry records no read — finalizing it into the cache does not count as use — so a build's freshly written output was reclaimed minutes later by the daemon's periodic maintenance. Never-read entries are now judged by their write time (the entry directory's mtime) against the same cutoff: a fresh entry survives, an old never-read one is still reclaimed, and an entry with any recorded read keeps the old behavior.

Verification

  • cargo fmt --all --manifest-path target/Cargo.toml -- --check — clean
  • cargo clippy --workspace --all-targets --locked --manifest-path target/Cargo.toml -- -D warnings — clean
  • cargo test --locked --manifest-path target/Cargo.toml -p op — 59 passed
  • cargo test --locked --manifest-path target/Cargo.toml -p lcache — 15 passed
  • cargo test --locked --manifest-path target/Cargo.toml -p minimald --lib maintenance — 6 passed
  • cargo test --locked --manifest-path target/Cargo.toml -p minimald --lib clean_cache — 2 passed

The selection test drives CleanCache::run over five entries and proves all three arms: a never-read entry written minutes ago survives; one written 30 days ago with no read is reclaimed by its write time; a control entry with an old recorded read is reclaimed as before. The minimald suite's pre-existing network/box failures were reproduced on the clean base checkout and are unrelated to this change.

Summary by CodeRabbit

  • New Features
    • Added a way to locate a cache entry’s filesystem path, whether or not it exists.
  • Bug Fixes
    • Cache cleanup now identifies never-used entries as stale based on when they were written, while preserving recently written entries.
  • Documentation
    • Clarified that cache cleanup considers both last-use time and write time when determining which artifacts are stale.

A cache entry with no recorded read was deleted by a clean no matter how
recently it had been written: finalizing an entry records no read, so a
build's output could be reclaimed minutes later by the daemon's periodic
maintenance. Judge an entry nothing has read yet by its write time — its
directory's mtime — against the same cutoff, so a fresh entry survives
and an old never-read one is still reclaimed. A recorded read still wins
when an entry has one.
@gominimal-aw-bot
gominimal-aw-bot Bot requested a review from a team as a code owner November 17, 2026 17:59
@gominimal-aw-bot gominimal-aw-bot Bot added the agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. label Nov 17, 2026
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Cache cleanup now uses write time to age entries that have no recorded read. The change also adds a method that returns a cache entry path from its hash, and updates cleanup tests and documentation.

Changes: Cache entry path accessor

Layer / File(s) Summary
Resolve cache entry paths
crates/lcache/src/lib.rs
Cache::entry_path returns the filesystem path for a hash using the existing bucket layout. It returns the path even when the entry does not exist.

Cache cleanup age rules

Layer / File(s) Summary
Apply the cutoff to never-read entries
crates/op/src/cache_clean.rs, crates/minimald/src/maintenance.rs, crates/minimald/src/rpc/tests.rs, crates/mip/src/cmd_cache.rs, docs/reference/cli-mip.md
Cleanup uses recorded read time when available and otherwise uses the entry directory’s modification time. Entries with unreadable write times are treated as stale. Tests cover old and fresh never-read entries, and the CLI descriptions state the write-time cutoff.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: bryan-minimal, twitchyliquid64, norrietaylor

Merge Risk: Minimal · up to c388b

Cache cleanup now keeps recently written entries that have never been read and still reclaims old ones. No concrete merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title follows the Conventional Commit form and names the main change: the cache clean no longer deletes never-read entries based on write time. It has a repeated prefix, "fix: fix(op):", which is …
Description check Passed The description explains the bug, the fix, and the rule for never-read entries versus entries with a recorded read. It lists verification commands and their results, and it covers the test cases. It d…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title follows the Conventional Commit form and names the main change: the cache clean no longer deletes never-read entries based on write time. It has a repeated prefix, "fix: fix(op):", which is noise and does not match the template's single type(scope) form. The title is 76 characters, so it is also longer than ideal. The title is still related to the changeset and is clear enough for a teammate scanning history.

Full details: Description check

Explanation

The description explains the bug, the fix, and the rule for never-read entries versus entries with a recorded read. It lists verification commands and their results, and it covers the test cases. It does not use the template's exact headings (Summary, Testing, Checklist). It has no issue link, no checklist items, and no breaking-change note. The Summary and Testing content is present but unlabeled, and the checklist is missing.

Resolution

Optional: add the template headings (Summary, Testing, Checklist). Link the related issue with Refs or Closes if one exists. Tick or address the Checklist items, including whether docs were updated (docs/reference/cli-mip.md appears to be changed) and whether a BREAKING CHANGE footer is needed.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

Vale (3.24.0)

{
"Line": 13,
"Path": ".vale.ini",
"Text": "Style 'ste' isn't on the StylesPath (/styles). If it's a package, run vale sync.",
"Code": "E201",
"Span": 23
}

A rabbit checks the cache at dawn
Old entries hop away, withdrawn
Fresh ones rest and keep their place
Hashes lead to each path’s space
The cutoff’s clear, the tests agree
The burrow stays tidy as can be!

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 6ca1fdfb-6a09-42c2-bb41-3dff2ab3b750
Commits

Reviewing files that changed from the base of the PR and between ae8fcb5 and c388be4.

Files selected for processing: 6

  • crates/lcache/src/lib.rs
  • crates/minimald/src/maintenance.rs
  • crates/minimald/src/rpc/tests.rs
  • crates/mip/src/cmd_cache.rs
  • crates/op/src/cache_clean.rs
  • docs/reference/cli-mip.md

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

@gominimal-aw-bot gominimal-aw-bot Bot added the agent:resolve Consent gate: pr-conflict-resolver may rewrite this branch label Nov 17, 2026

@bryan-minimal bryan-minimal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the direction is right and the main test does what inbox#1270 asked. One change before merge, then notes.

  1. An old read outweighs a fresh write (crates/op/src/cache_clean.rs, the selection around lines 140-145). last_read(hash).or_else(|| written_at(..)) only looks at the write time when there is no read at all. The access log never forgets a hash: invalidate_dir (crates/lcache/src/lib.rs) removes the directory and leaves the log alone. So a package read ten days ago gets cleaned; a task then needs it, so it is fetched or built again; the next clean finds the ten-day-old read, ignores the new write, and deletes the package again. The fresh read that would save it is often still in the daemon's unflushed buffer (see 4). That is the same symptom the issue reported, for the same packages. Please judge by the newer of the two, max(last_read, written_at), and add a case to the test: an old recorded read plus a fresh write must survive.

  2. Acceptance points: met. deletes_only_unused_unkept_entries drives CleanCache::run with a fresh never-read entry (kept), a never-read entry back-dated 30 days (deleted) and a control with an old recorded read (deleted). With the fix reverted, the fresh entry is deleted and the count goes to 3, so the test fails.

  3. The four existing tests were not weakened. Each now back-dates its entries past the cutoff and still asserts the deletion. In clean_keeps_what_a_session_needs_and_drops_the_rest the needed entry is back-dated too, which makes its keep-set check stronger. One gap: no daemon-level test shows a fresh entry surviving maintenance.

  4. The two leads in the issue stay open. Reads are flushed to disk every 15 reads, so a daemon's clean can miss its own recent reads; and the keep set may not cover what task boxes use. The new rule makes both harmless for entries written inside the cutoff. Neither is harmless for an older entry, whose recent unflushed read is invisible, so its stale flushed read decides. Point 1 makes that worse. Please say in the PR that both stay open, and link the issue with Closes gominimal/inbox#1270 only if 1 is fixed.

  5. Directory mtime as the write time holds up. The entry is the pending directory renamed into place; its time is set as children are created; the archive extraction does not set directory times; nothing found that sets an entry's directory time afterwards, including the fixed-epoch handling for reproducible builds. A restore that keeps old times is judged old, which is correct. A future mtime from clock skew keeps the entry until the clock passes it, which is acceptable.

  6. An unreadable mtime means delete. That matches the old behaviour and avoids keeping an entry forever, so it is fine. The comment could say so.

  7. Docs. cli-mip.md, the doc comment and the mip cache clean help all state the new rule. The issue asked for a maintainer to agree the rule first; please get that agreement on this PR.

  8. Nit. entry_path repeats the bucket path code in finalize, dir and invalidate_dir; invalidate_dir could call it.

Not verified: the tests were read, not run (CI is green). Whether a rename into place changes the directory's mtime on every filesystem; if it does, the rule only gets safer.

🤖 Generated with Claude Code

@norrietaylor norrietaylor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direction is right and the write-time rule for never-read entries holds up. One defect blocks merge: an old recorded read still beats a fresh write, so a package that was cleaned once and then rebuilt or refetched is deleted again by the next clean. That is the same symptom this PR sets out to fix. Details inline. The earlier change request on this head (the same point) has not been addressed yet: the PR still has its single original commit.

Also: the PR title reads fix: fix(op): ..., which doubles the type prefix in the squash commit subject.

.filter(|hash| {
atimes
.last_read(hash)
.or_else(|| written_at(cache, hash))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] last_read(hash).or_else(|| written_at(..)) only consults the write time when the hash has no recorded read at all. The access log never forgets a hash: invalidate_dir (crates/lcache/src/lib.rs:292) removes the directory and leaves alog/ alone, and atimes() loads only what is flushed to disk (reads flush every FLUSH_INTERVAL calls). So: an entry last read 20 days ago is cleaned, a task needs it and it is rebuilt or refetched now, and the next clean sees the 20-day-old read, ignores the fresh directory mtime, and deletes it again. Its new read, if any, is often still in the daemon's unflushed buffer.

Fix: judge by the newer of the two, e.g. match (atimes.last_read(hash), written_at(cache, hash)) { (Some(r), Some(w)) => Some(r.max(w)), (r, w) => r.or(w) }, and update the comment above ("A read, however stale, still wins") to match. Add a case to deletes_only_unused_unkept_entries: an entry with an old recorded read and a fresh write must survive.

Comment thread crates/lcache/src/lib.rs
}

/// Invalidates (deletes) a directory cache entry with the given spec hash.
pub fn invalidate_dir(&self, hash: &SpecHash) -> Result<(), CacheErr> {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] invalidate_dir repeats the bucket-path computation that the new entry_path just above does; it could call self.entry_path(hash).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. agent:resolve Consent gate: pr-conflict-resolver may rewrite this branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants