Repository navigation
fix: fix(op): stop the cache clean deleting never-read entries on write time - #3501
gominimal-aw-bot[bot] wants to merge 1 commit into
Conversation
A cache entry with no recorded read was deleted by a clean no matter how recently it had been written: finalizing an entry records no read, so a build's output could be reclaimed minutes later by the daemon's periodic maintenance. Judge an entry nothing has read yet by its write time — its directory's mtime — against the same cutoff, so a fresh entry survives and an old never-read one is still reclaimed. A recorded read still wins when an entry has one.
bryan-minimal
left a comment
There was a problem hiding this comment.
Thanks, the direction is right and the main test does what inbox#1270 asked. One change before merge, then notes.
-
An old read outweighs a fresh write (
crates/op/src/cache_clean.rs, the selection around lines 140-145).last_read(hash).or_else(|| written_at(..))only looks at the write time when there is no read at all. The access log never forgets a hash:invalidate_dir(crates/lcache/src/lib.rs) removes the directory and leaves the log alone. So a package read ten days ago gets cleaned; a task then needs it, so it is fetched or built again; the next clean finds the ten-day-old read, ignores the new write, and deletes the package again. The fresh read that would save it is often still in the daemon's unflushed buffer (see 4). That is the same symptom the issue reported, for the same packages. Please judge by the newer of the two,max(last_read, written_at), and add a case to the test: an old recorded read plus a fresh write must survive. -
Acceptance points: met.
deletes_only_unused_unkept_entriesdrivesCleanCache::runwith a fresh never-read entry (kept), a never-read entry back-dated 30 days (deleted) and a control with an old recorded read (deleted). With the fix reverted, the fresh entry is deleted and the count goes to 3, so the test fails. -
The four existing tests were not weakened. Each now back-dates its entries past the cutoff and still asserts the deletion. In
clean_keeps_what_a_session_needs_and_drops_the_restthe needed entry is back-dated too, which makes its keep-set check stronger. One gap: no daemon-level test shows a fresh entry survivingmaintenance. -
The two leads in the issue stay open. Reads are flushed to disk every 15 reads, so a daemon's clean can miss its own recent reads; and the keep set may not cover what task boxes use. The new rule makes both harmless for entries written inside the cutoff. Neither is harmless for an older entry, whose recent unflushed read is invisible, so its stale flushed read decides. Point 1 makes that worse. Please say in the PR that both stay open, and link the issue with
Closes gominimal/inbox#1270only if 1 is fixed. -
Directory mtime as the write time holds up. The entry is the pending directory renamed into place; its time is set as children are created; the archive extraction does not set directory times; nothing found that sets an entry's directory time afterwards, including the fixed-epoch handling for reproducible builds. A restore that keeps old times is judged old, which is correct. A future mtime from clock skew keeps the entry until the clock passes it, which is acceptable.
-
An unreadable mtime means delete. That matches the old behaviour and avoids keeping an entry forever, so it is fine. The comment could say so.
-
Docs.
cli-mip.md, the doc comment and themip cache cleanhelp all state the new rule. The issue asked for a maintainer to agree the rule first; please get that agreement on this PR. -
Nit.
entry_pathrepeats the bucket path code infinalize,dirandinvalidate_dir;invalidate_dircould call it.
Not verified: the tests were read, not run (CI is green). Whether a rename into place changes the directory's mtime on every filesystem; if it does, the rule only gets safer.
🤖 Generated with Claude Code
norrietaylor
left a comment
There was a problem hiding this comment.
The direction is right and the write-time rule for never-read entries holds up. One defect blocks merge: an old recorded read still beats a fresh write, so a package that was cleaned once and then rebuilt or refetched is deleted again by the next clean. That is the same symptom this PR sets out to fix. Details inline. The earlier change request on this head (the same point) has not been addressed yet: the PR still has its single original commit.
Also: the PR title reads fix: fix(op): ..., which doubles the type prefix in the squash commit subject.
| .filter(|hash| { | ||
| atimes | ||
| .last_read(hash) | ||
| .or_else(|| written_at(cache, hash)) |
There was a problem hiding this comment.
[medium] last_read(hash).or_else(|| written_at(..)) only consults the write time when the hash has no recorded read at all. The access log never forgets a hash: invalidate_dir (crates/lcache/src/lib.rs:292) removes the directory and leaves alog/ alone, and atimes() loads only what is flushed to disk (reads flush every FLUSH_INTERVAL calls). So: an entry last read 20 days ago is cleaned, a task needs it and it is rebuilt or refetched now, and the next clean sees the 20-day-old read, ignores the fresh directory mtime, and deletes it again. Its new read, if any, is often still in the daemon's unflushed buffer.
Fix: judge by the newer of the two, e.g. match (atimes.last_read(hash), written_at(cache, hash)) { (Some(r), Some(w)) => Some(r.max(w)), (r, w) => r.or(w) }, and update the comment above ("A read, however stale, still wins") to match. Add a case to deletes_only_unused_unkept_entries: an entry with an old recorded read and a fresh write must survive.
| } | ||
|
|
||
| /// Invalidates (deletes) a directory cache entry with the given spec hash. | ||
| pub fn invalidate_dir(&self, hash: &SpecHash) -> Result<(), CacheErr> { |
There was a problem hiding this comment.
[nit] invalidate_dir repeats the bucket-path computation that the new entry_path just above does; it could call self.entry_path(hash).
Routing-Key: inbox-route/I_kwDOSUhdos8AAAABgGpBtg
The cache clean deleted entries with no recorded read regardless of age.
Writing an entry records no read — finalizing it into the cache does not count as use — so a build's freshly written output was reclaimed minutes later by the daemon's periodic maintenance. Never-read entries are now judged by their write time (the entry directory's mtime) against the same cutoff: a fresh entry survives, an old never-read one is still reclaimed, and an entry with any recorded read keeps the old behavior.
Verification
cargo fmt --all --manifest-path target/Cargo.toml -- --check— cleancargo clippy --workspace --all-targets --locked --manifest-path target/Cargo.toml -- -D warnings— cleancargo test --locked --manifest-path target/Cargo.toml -p op— 59 passedcargo test --locked --manifest-path target/Cargo.toml -p lcache— 15 passedcargo test --locked --manifest-path target/Cargo.toml -p minimald --lib maintenance— 6 passedcargo test --locked --manifest-path target/Cargo.toml -p minimald --lib clean_cache— 2 passedThe selection test drives
CleanCache::runover five entries and proves all three arms: a never-read entry written minutes ago survives; one written 30 days ago with no read is reclaimed by its write time; a control entry with an old recorded read is reclaimed as before. The minimald suite's pre-existing network/box failures were reproduced on the clean base checkout and are unrelated to this change.Summary by CodeRabbit