Skip to content

fix(minimald): fork a task run's container on the sandbox fork thread, not a runtime worker - #2572

Merged
bryan-minimal merged 1 commit into
mainfrom
fix/710-task-fork-thread
Oct 25, 2026
Merged

bryan-minimal merged 1 commit into
mainfrom
fix/710-task-fork-thread

Conversation

@bryan-minimal

@bryan-minimal bryan-minimal commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes gominimal/inbox#710: idle task sessions SIGKILLed together, seconds after unrelated activity.

Mechanism. hakoniwa arms PR_SET_PDEATHSIG(SIGKILL) in every container it forks, and Linux delivers that signal when the forking thread exits. #1748 moved the session launch and Sandbox::run_with_cancel onto sandbox::forker's never-retired thread for that reason. The task-run path in exec.rs was the third spawn site, and it kept forking a task's container on whichever tokio worker ran the exec. A later block_in_place on that worker (an activation's configure_loadout) displaces it, tokio retires it after its 10 s keep-alive, and every idle task session forked on it dies. hakoniwa reports the container's death as exit 125 (Child::FAILURE, "the Container itself fails"), with no client disconnect.

Fix. The task-run spawn goes through on_fork_thread like the other two. One line plus a comment; the caller still blocks for the fork-and-exec handshake as it did inline.

Evidence

The lab scenario 710-pdeathsig-sibling-kill (four idle min task run sessions, eight min session activates, 25 s idle, six rounds per cell) fails on every main built this week: 18 of 18 cells on ccc4f95c (six distros, three rounds, anvil job 01243), and on fb92f871, 608ab9c9 and 230709d3 before it. In every killed session the forking thread is a minimald-worker that is gone by the time of the check, never sandbox-fork. The daemon log of one cell (arch, round 3):

07:10:03.334 exec request            5be34b41ab9f  min://task/run --owns-box t
07:10:05 .. 07:10:09                 eight `min session activate`s
07:10:20.213 run box ended           5be34b41ab9f  exit=125      (no "ssh client disconnected")
07:10:30.218 run box ended           863de6f73711  exit=125      (10.005 s later; the next worker's keep-alive)

With the fix, on the same main (230709d3), six distros, three rounds, anvil job 01257:

candidate 710-pdeathsig-sibling-kill 000-baseline
main 230709d 0 / 18 pass 18 / 18
this branch 18 / 18 pass 18 / 18

A family check (690, 690b, 710d, 710g, 732, 789b, 789t, 789y, 789z on four distros, two rounds; job 01258) is 32 / 32 on both, so nothing else in the exec and teardown paths moved.

Testing

  • just fmt-check, just clippy, and the workspace tests on anvil through prepush.sh (the CI mirror); the only failure is the environment-only sandbox::caps_root_integration::boxes_lack_cap_net_raw.
  • No unit test: the fork-thread guarantee itself is covered by sandbox::forker's tests (a_fork_on_the_fork_thread_outlives_the_caller), and proving the task path uses it needs a real sandbox, which is what the lab scenario does.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Improvements
    • Task process launches now run through the sandbox-managed execution path instead of directly on the async runtime. Launch failures continue to use the same error reporting. This is a change to how task startup is performed; no new user-facing controls or task configuration options are included in this update.

…, not a runtime worker

hakoniwa arms `PR_SET_PDEATHSIG(SIGKILL)` in every container it forks, and Linux delivers that
signal when the forking *thread* exits. #1748 moved the session launch and
`Sandbox::run_with_cancel` onto `sandbox::forker`'s never-retired thread for exactly that reason,
but the task-run path in `exec.rs` kept forking its container on whichever tokio worker ran the
exec. Any `block_in_place` on that worker (a later activation's `configure_loadout`) displaces it,
tokio retires it after its 10 s keep-alive, and every idle task session forked on it dies:
hakoniwa reports the container's death as exit 125, with no client disconnect, which is the
signature in every lab run of inbox#710.

The spawn goes through `on_fork_thread` like the other two. Nothing else changes: the caller
still blocks for the fork-and-exec handshake as it did inline.

Refs: gominimal/inbox#710, #1748

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@bryan-minimal
bryan-minimal requested a review from a team as a code owner October 25, 2026 17:18
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 296d4185-ed7d-469e-8be3-b22ab3527eae
Commits

Reviewing files that changed from the base of the PR and between e40e028 and 89350fe.

Files selected for processing: 1
  • crates/minimald/src/exec.rs

Included review availability: This review used your included allowance. Your plan provides up to 5 included reviews per hour; 0 remain after this review.

Walkthrough

Task invocation spawning moves from the async runtime worker to the sandbox fork thread. Spawn errors continue through the existing command-launch error mapping.

Changes: Task Process Spawning

Layer / File(s) Summary
Move task spawning to the fork thread
crates/minimald/src/exec.rs
Task invocation uses sandbox::forker::on_fork_thread to run cmd.spawn(). The existing command-launch error mapping remains unchanged.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Possibly related issues

Merge Risk: Minimal · up to 89350

The task spawn is moved to the intended fork thread without an identified functional regression.

Pre-merge checks: Passed 3 · Failed 2

Failed Failed checks: 2 warnings

Check name Status Explanation Resolution
Description check Warning Description fails because it is incomplete. It includes Summary and Testing sections. The Summary explains the cause, including PR_SET_PDEATHSIG and fork-thread exit, describes the fix, and links `g… Add the Checklist section from the repository template. State whether documentation was updated. State whether the change is breaking. If the repository requires issue-reference keywords, add Closes: gominimal/inbox#710 or `Refs: gominima…
Linked Issues check Warning The change in crates/minimald/src/exec.rs moves task-container spawning to sandbox::forker::on_fork_thread. The reported lab results support this as a fix for the reproduced task-session failure. … Implement the #710 diagnostic-coverage and abnormal-session-state changes, or provide updated issue requirements that explicitly limit #710 to the fork-thread fix.
Passed Passed checks: 3 passed
Check name Status Explanation
Title check Passed Title passes. It uses Conventional Commit format, fix(minimald): ..., and states the main change: the task run container is forked on the sandbox fork thread instead of a runtime worker. The title m…
Out of Scope Changes check Passed The only code change moves the task-run cmd.spawn() call to sandbox::forker::on_fork_thread and documents the PR_SET_PDEATHSIG failure mode. This directly supports the pull request objective and…
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Full details: Title check

Explanation

Title passes. It uses Conventional Commit format, fix(minimald): ..., and states the main change: the task run container is forked on the sandbox fork thread instead of a runtime worker. The title matches the summary and objectives. It is 91 characters long and does not identify the PDEATHSIG cause, but it remains specific and accurate.

Full details: Out of Scope Changes check

Explanation

The only code change moves the task-run cmd.spawn() call to sandbox::forker::on_fork_thread and documents the PR_SET_PDEATHSIG failure mode. This directly supports the pull request objective and the reported #710 symptom. No unrelated implementation change is shown.

Full details: Description check

Explanation

Description fails because it is incomplete. It includes Summary and Testing sections. The Summary explains the cause, including PR_SET_PDEATHSIG and fork-thread exit, describes the fix, and links gominimal/inbox#710. The Testing section lists formatting, Clippy, workspace tests, a lab scenario with results, and the reason no unit test was added. The template Checklist section is missing. The description does not state whether documentation was updated or whether the change is breaking. It also lacks a Closes: or Refs: line for gominimal/inbox#710.

Resolution

Add the Checklist section from the repository template. State whether documentation was updated. State whether the change is breaking. If the repository requires issue-reference keywords, add Closes: gominimal/inbox#710 or Refs: gominimal/inbox#710.

Full details: Linked Issues check

Explanation

The change in crates/minimald/src/exec.rs moves task-container spawning to sandbox::forker::on_fork_thread. The reported lab results support this as a fix for the reproduced task-session failure. However, open issue #710 also identifies two coding follow-ups: extend incident-time kmsg diagnostic coverage and clear or mark session records after abnormal death. This pull request changes neither diagnostic collection nor session-record handling. The issue does not establish that the original incident had this cause, so the lab result does not replace those stated coding requirements.

Resolution

Implement the #710 diagnostic-coverage and abnormal-session-state changes, or provide updated issue requirements that explicitly limit #710 to the fork-thread fix.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit watched the task take flight
From fork-thread paths into the night
The spawn call hopped along its way
Its error map stayed just the same
Then soft paws stamped the diff: “All right!”

Comment @coderabbitai help to get the list of available commands.

@minimal-foundry

minimal-foundry Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Reviewed: 1 finding, none blocking at 89350fe475f8.

Address the comments on the diff.

Clobber: none seen.

@minimal-foundry minimal-foundry Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minimal-review — advisory review

Verdict COMMENT — 1 finding, 1 inline.

Panel: 3 of 3 personas ran (correctness, security, supply-chain).

Comment thread crates/minimald/src/exec.rs Resolved
@bryan-minimal
bryan-minimal added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit cafa674 Oct 25, 2026
36 checks passed
@bryan-minimal
bryan-minimal deleted the fix/710-task-fork-thread branch October 25, 2026 22:36
norrietaylor pushed a commit that referenced this pull request Nov 4, 2026
…#2572)

hakoniwa arms PR_SET_PDEATHSIG(SIGKILL) in every container it forks, and Linux delivers that
signal when the forking thread exits. The task-run path in exec.rs forked its container on
whichever tokio worker ran the exec. Any block_in_place on that worker displaces it, tokio
retires it after its 10 s keep-alive, and every idle task session forked on it dies with exit
125. The spawn now goes through on_fork_thread, as the session launch and run_with_cancel do.

Backport of #2572. Adapted to the line: sandbox:: reads sandbox2::.

(cherry picked from commit cafa674)
norrietaylor pushed a commit that referenced this pull request Nov 5, 2026
…#2572)

hakoniwa arms PR_SET_PDEATHSIG(SIGKILL) in every container it forks, and Linux delivers that
signal when the forking thread exits. The task-run path in exec.rs forked its container on
whichever tokio worker ran the exec. Any block_in_place on that worker displaces it, tokio
retires it after its 10 s keep-alive, and every idle task session forked on it dies with exit
125. The spawn now goes through on_fork_thread, as the session launch and run_with_cancel do.

Backport of #2572. Adapted to the line: sandbox:: reads sandbox2::.

(cherry picked from commit cafa674)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants