Repository navigation
Description
Problem
A WAL entry appended after a torn entry is lost on recovery, although its commit was fsynced and reported as successful.
WalReader::read_entries stops at the first entry that is truncated or fails its CRC. WalWriter::open_or_create_at seeks to the end of the file and appends there. So when the WAL ends in a partial entry, every entry written after it is unreachable on the next replay.
Two ways to get there:
- Crash during an append. A kill or power loss in the middle of
append_entryleaves a partial entry. The next open replays the entries before it, then appends new commits after the partial bytes. If the process stops again before a checkpoint, those commits are gone on the next open. - A failed append in a running process.
write_allfails partway (short write, ENOSPC, EIO). The commit returns an error and is not applied, but the partial bytes stay in the file, and the next commit on the same handle is appended after them.
Found while working on #390.
Repro (v3 at 0aa813d and main at v2.0.4)
Three processes, each ending in std::process::exit(0) so nothing checkpoints:
- Open a new file,
(transact [[:a :p/n 1]]), exit. - Append 7 bytes to
<db>.wal(a torn entry). Open: 1 fact.(transact [[:b :p/n 2]])returnsOk. 2 facts. Exit. - Open: 1 fact. The acknowledged commit from step 2 is lost.
Affected
Every release with the WAL, v0.5.0 through v2.0.4, file-backed databases only. A clean close checkpoints and deletes the WAL, so data is lost only when a second crash (or a kill) follows a first one before any checkpoint, or after a failed append in a running process.
Fix
- On a read-write open, cut the WAL back to the end of its last valid entry and fsync it before appending anything.
- After a failed append, refuse further writes on the handle (part of test: drive the fault-injection backend through the full save/WAL/recovery paths #390: the handle is marked failed and must be reopened).
Fix on v3, then a backport to main for a v2.x patch release.
Activity
Merge pull request #514 from project-minigraf/fix/513-wal-torn-tail
test: drive faults through the whole save/WAL/recovery path; never re…
Merge pull request #516 from project-minigraf/fix/513-wal-torn-tail-v2
adityamukho commentedon Oct 10, 2026
Fixed on
v3in #514 and onmain(v2.x) in #516. The fault matrix from #390 (#515) also covers it. This issue stays open until the fix ships in a published release (the next v2.x patch).release: v2.0.5
adityamukho commentedon Oct 11, 2026
Fixed in v2.0.5 (released 2026-10-11, PR #516); also fixed on
v3(PR #514) for v3.0.0. Moved to "Fixed in v2.0.5" in #421.