Skip to content

fix(minimald): keep exec output relays live while a stdin write is blocked - #2148

Merged
norrietaylor merged 3 commits into
mainfrom
inbox-patch/exec-bridge-stdin-parking-a26b2d69531fe2bc
Oct 10, 2026
Merged

norrietaylor merged 3 commits into
mainfrom
inbox-patch/exec-bridge-stdin-parking-a26b2d69531fe2bc

Conversation

@gominimal-aw-bot

@gominimal-aw-bot gominimal-aw-bot Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Routing-Key: inbox-route/I_kwDOSUhdos8AAAABU46TDw

In the exec bridge, each stdin chunk was written to the child inside the
read branch's handler, parked on a nested select that raced only client
loss. While that write was blocked on a full child stdin pipe, the outer
select never polled the child's stdout or stderr, so a child that
produced output before consuming its stdin (a large cat, say)
deadlocked the bridge: the session stalled until the client's keepalive
timed out.

Queue the chunk instead, and drive the child-stdin write from its own
branch of the outer select. That branch uses the cancel-safe write and
resumes from a saved offset, so the output relays, wait, and client
loss keep being polled while a write is parked, and no bytes are lost
when another branch wins the race.

Verification

  • cargo fmt --all --check --manifest-path target/Cargo.toml — clean
  • cargo clippy --workspace --all-targets --locked --manifest-path target/Cargo.toml -- -D warnings — no warnings
  • cargo test --locked --manifest-path target/Cargo.toml -p minimald — 915 passed; the 11 failures (net::switch, connection, rpc) are pre-existing, environment-dependent, and fail identically on pristine main
  • New unit test bridge_relays_output_while_a_stdin_write_is_blocked proves child output still flows while a stdin write is blocked, and times out on the old code; all 39 exec tests pass
  • live-session: native-daemon-e2e — runs on this PR; not runnable in the fixer sandbox
  • Real-host repro: the 5 MB session-exec repro was not rerun on a real host. Coverage is
    bridge_relays_output_while_a_stdin_write_is_blocked (256 KiB each way through mock pipes),
    bridge_streams_megabytes_through_cat (4 MiB through a real cat child on the Linux CI lanes),
    and bridge_reports_exit_when_child_exits_with_stdin_queued (child reaped with a chunk queued:
    exit status kept, queued chunk dropped, no kill).

Summary by CodeRabbit

  • Bug Fixes
    • Prevented commands from stalling output when they are slow to accept input. Output continues while input is pending, and queued input is delivered when the command can read it.
    • Stopped sending input to a command after its input stream closes, while keeping the SSH input stream available to later commands.
    • Preserved command exit status when the command exits with input still queued.

…ocked

bridge_one wrote each stdin chunk to the child inside the read branch's
handler, parked on a nested select that raced only client loss. While
that write was blocked on a full child stdin pipe, the outer select
never polled the child's stdout or stderr, so a child that produced
output before consuming stdin (for example a large cat) deadlocked
the bridge: the session stalled until the client's keepalive timed out.

Queue the chunk instead, and drive the child-stdin write from a
dedicated branch of the outer select. The branch uses the cancel-safe
write and resumes from a saved offset, so the loop keeps polling
stdout, stderr, wait, and client loss while a write is parked, and no
bytes are lost when another branch wins the race.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@gominimal-aw-bot
gominimal-aw-bot Bot requested a review from a team as a code owner October 10, 2026 01:05
@gominimal-aw-bot gominimal-aw-bot Bot added the agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. label Oct 10, 2026
@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 024799b7-cd7b-4915-9941-5e1cf1467414

📥 Commits

Reviewing files that changed from the base of the PR and between bdd6664 and 669939f.


📒 Files selected for processing (1)
  • crates/minimald/src/exec.rs

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.



📝 Walkthrough

Walkthrough

The SSH bridge now reads SSH stdin and writes child stdin in separate select! branches. It tracks partial writes and continues to process child output and exit events while input is queued. Tests cover blocked writes, concurrent streaming, and child exit with queued input.

Changes

SSH bridge I/O

Layer / File(s) Summary
Cancelable child-stdin flow
crates/minimald/src/exec.rs
bridge_one queues SSH input and writes it incrementally. It continues polling output and process exit while input is pending. A write failure or zero-byte write closes that child’s stdin and discards pending bytes. EOF or an SSH stdin read error closes stdin for the sequence. The bridge documentation describes child-stdin writes as cancel-safe.
Queued-input regression tests
crates/minimald/src/exec.rs
Tests verify output relay and later input delivery during backpressure, concurrent streaming of 4 MiB through cat, and return of exit status 3 when the child exits with input queued.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant SSHClient
  participant bridge_one
  participant ChildProcess
  SSHClient->>bridge_one: provide input
  bridge_one->>ChildProcess: write pending input
  ChildProcess-->>bridge_one: produce stdout or stderr
  bridge_one-->>SSHClient: relay child output
  ChildProcess-->>bridge_one: exit with status
Loading

Suggested reviewers: norrietaylor


Merge Risk: ⚪ Minimal · up to 66993

No identified issue blocks merging; live-session behavior remains unconfirmed.

Pre-merge checks | Passed 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 1 files.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly and concisely describes the main change and follows the required Conventional Commit format.
Description check Passed The description provides a clear change summary and detailed verification evidence. It does not use the template's exact Summary, Testing, and Checklist headings, and it omits the checklist, but the r…


✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR


  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit sends a stream along,
While buffered bytes keep moving on.
The child may pause; the output flows,
Then queued input onward goes.
Four megabytes pass through with care,
And exit status finds the air.

Comment @coderabbitai help to get the list of available commands.

…n queued

Add a real-process regression test that pipes 4 MiB through cat in both
directions at once, and a mock test that a child exiting with a stdin
chunk still queued reports its own status. Seed child_stdin_open from
whether the step has a child stdin at all.
@norrietaylor norrietaylor changed the title fix: fix(minimald): keep exec output relays live while a stdin write is blocked fix(minimald): keep exec output relays live while a stdin write is blocked Oct 10, 2026
…ectly

select! evaluates every precondition before it builds any branch future,
so the gates can read child_stdin and pending themselves; the mirror
flags only had to be kept in step by hand. Note at the loop exit that
stdin still queued when the loop ends is dropped, and make the queued-
stdin exit test keep every pipe end open so the wait branch is the only
way out, asserting the child saw exactly its pipe's worth of stdin.
@gominimal-aw-bot gominimal-aw-bot Bot added the agent:resolve Consent gate: pr-conflict-resolver may rewrite this branch label Oct 10, 2026
@norrietaylor

Copy link
Copy Markdown
Member

/review

@minimal-foundry

minimal-foundry Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Approved: no findings at 31dd4ffa6136.

Nothing to do.

Clobber: none: git log -- crates/minimald/src/exec.rs shows only this PR's own commits (bdd6664 → 669939f → 31dd4ff); nothing reverts recent work.

@minimal-foundry minimal-foundry Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minimal-review — approved

Verdict APPROVE — No findings.

@minimal-foundry minimal-foundry Bot added the review:approved minimal-review approved the current head label Oct 10, 2026
@norrietaylor
norrietaylor added this pull request to the merge queue Oct 10, 2026
Merged via the queue into main with commit f80d920 Oct 10, 2026
37 checks passed
@norrietaylor
norrietaylor deleted the inbox-patch/exec-bridge-stdin-parking-a26b2d69531fe2bc branch October 10, 2026 02:38
bryan-minimal added a commit that referenced this pull request Oct 13, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains: the channel is read as data arrives, a writer of its own feeds the command from what
is held, and the held amount is capped at 256 MiB, past which the exec ends with a notice on
its stderr rather than hanging (an active reject). A command that closed its stdin or exited
is not charged for input it never reads.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bryan-minimal added a commit that referenced this pull request Oct 13, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains: the channel is read as data arrives, a writer of its own feeds the command from what
is held, and the held amount is bounded twice, 256 MiB per exec and 512 MiB across the
daemon's execs, past which the exec that would cross the bound ends with a notice on its
stderr rather than hanging (an active reject). A command that closed its stdin or exited is
not charged for input it never reads.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bryan-minimal added a commit that referenced this pull request Oct 13, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains: the channel is read as data arrives, a writer of its own feeds the command from what
is held, and the held amount is bounded twice, 256 MiB per exec and 512 MiB across the
daemon's execs, past which the exec that would cross the bound ends with a notice on its
stderr rather than hanging (an active reject). A command that closed its stdin or exited is
not charged for input it never reads.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bryan-minimal added a commit that referenced this pull request Oct 14, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains. The pump takes each packet off the channel into a queue the bridge reads as the
command's stdin; a byte is charged to the exec's and the daemon's held counts from push to
read, as a plain Vec (not russh's power-of-two locked buffer). The counts are bounded, 256 MiB
per exec and 512 MiB across the daemon's execs: past either, the queue is cleared and released
at once and the exec ends under its own reason (not as a lost client), with a notice on its
stderr naming the bound. For a single program, input arriving after the command closed its
stdin is read and dropped, never held against it; a sequence keeps it for the next command.

At a bound the pump stops taking from the channel instead of ending the exec
at once: before the command starts that is the wait for it (a fast client puts
the cap in flight before a command spawns on a loaded host), and once it runs
a bound with no byte read for 10 s is the stall that ends it. The connection
is parked for at most that long per bound; a reading command is never ended.

The nsenter shim inherited the exec's stdin pipe as its own fd 0 and held it while
waiting on the command, so a command that closed its stdin never closed the pipe:
the bridge saw no EPIPE and took it for a command that had stopped reading. The
shim never reads fd 0; after the spawn it points fd 0 at /dev/null.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bryan-minimal added a commit that referenced this pull request Oct 14, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains. The pump takes each packet off the channel into a queue the bridge reads as the
command's stdin; a byte is charged to the exec's and the daemon's held counts from push to
read, as a plain Vec (not russh's power-of-two locked buffer). The counts are bounded, 256 MiB
per exec and 512 MiB across the daemon's execs: past either, the queue is cleared and released
at once and the exec ends under its own reason (not as a lost client), with a notice on its
stderr naming the bound. For a single program, input arriving after the command closed its
stdin is read and dropped, never held against it; a sequence keeps it for the next command.

At a bound the pump stops taking from the channel instead of ending the exec
at once: before the command starts that is the wait for it (a fast client puts
the cap in flight before a command spawns on a loaded host), and once it runs
a bound with no byte read for 10 s is the stall that ends it. The connection
is parked for at most that long per bound; a reading command is never ended.

The nsenter shim inherited the exec's stdin pipe as its own fd 0 and held it while
waiting on the command, so a command that closed its stdin never closed the pipe:
the bridge saw no EPIPE and took it for a command that had stopped reading. The
shim never reads fd 0; after the spawn it points fd 0 at /dev/null.

What the daemon holds is never overstated: a push that races the reader's drop is
dropped uncharged (clear discards), the pump is awaited after its abort, and an
exec releases what it still counts when it is dropped; a client that goes releases
its held input at once. At the daemon bound every exec's releases wake the waiting
pumps, and an exec holding nothing of its own waits for the others rather than
being ended for their hoarding.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bryan-minimal added a commit that referenced this pull request Oct 15, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains. The pump takes each packet off the channel into a queue the bridge reads as the
command's stdin; a byte is charged to the exec's and the daemon's held counts from push to
read, as a plain Vec (not russh's power-of-two locked buffer). The counts are bounded, 256 MiB
per exec and 512 MiB across the daemon's execs: past either, the queue is cleared and released
at once and the exec ends under its own reason (not as a lost client), with a notice on its
stderr naming the bound. For a single program, input arriving after the command closed its
stdin is read and dropped, never held against it; a sequence keeps it for the next command.

At a bound the pump stops taking from the channel instead of ending the exec
at once: before the command starts that is the wait for it (a fast client puts
the cap in flight before a command spawns on a loaded host), and once it runs
a bound with no byte read for 10 s is the stall that ends it. The connection
is parked for at most that long per bound; a reading command is never ended.

The nsenter shim inherited the exec's stdin pipe as its own fd 0 and held it while
waiting on the command, so a command that closed its stdin never closed the pipe:
the bridge saw no EPIPE and took it for a command that had stopped reading. The
shim never reads fd 0; after the spawn it points fd 0 at /dev/null.

What the daemon holds is never overstated: a push that races the reader's drop is
dropped uncharged (clear discards), the pump is awaited after its abort, and an
exec releases what it still counts when it is dropped; a client that goes releases
its held input at once. At the daemon bound every exec's releases wake the waiting
pumps, and an exec holding nothing of its own waits for the others rather than
being ended for their hoarding.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bryan-minimal added a commit that referenced this pull request Oct 17, 2026
…and cannot park the connection

The second half of gominimal/inbox#866: with #2148 in, an exec that streams enough data both
ways still hangs, and the whole connection with it. minimald's input pump wrote each
CHANNEL_DATA into the command's stdin pipe before reading the next; a command blocked on its
own stdout left the pump waiting, russh's session loop parked on that channel's bounded queue,
and a parked loop delivers nothing to any channel, including the command's output, whose
draining is what would let it read its stdin again.

russh grants the client's send window on receipt and gives the server no way to hold it back
(Eugeny/russh#730 was declined upstream: "the application must always drain"), so the daemon
drains. The pump takes each packet off the channel into a queue the bridge reads as the
command's stdin; a byte is charged to the exec's and the daemon's held counts from push to
read, as a plain Vec (not russh's power-of-two locked buffer). The counts are bounded, 256 MiB
per exec and 512 MiB across the daemon's execs: past either, the queue is cleared and released
at once and the exec ends under its own reason (not as a lost client), with a notice on its
stderr naming the bound. For a single program, input arriving after the command closed its
stdin is read and dropped, never held against it; a sequence keeps it for the next command.

At a bound the pump stops taking from the channel instead of ending the exec
at once: before the command starts that is the wait for it (a fast client puts
the cap in flight before a command spawns on a loaded host), and once it runs
a bound with no byte read for 10 s is the stall that ends it. The connection
is parked for at most that long per bound; a reading command is never ended.

The nsenter shim inherited the exec's stdin pipe as its own fd 0 and held it while
waiting on the command, so a command that closed its stdin never closed the pipe:
the bridge saw no EPIPE and took it for a command that had stopped reading. The
shim never reads fd 0; after the spawn it points fd 0 at /dev/null.

What the daemon holds is never overstated: a push that races the reader's drop is
dropped uncharged (clear discards), the pump is awaited after its abort, and an
exec releases what it still counts when it is dropped; a client that goes releases
its held input at once. At the daemon bound every exec's releases wake the waiting
pumps, and an exec holding nothing of its own waits for the others rather than
being ended for their hoarding.

Refs: gominimal/inbox#866, Eugeny/russh#730
Fixes: gominimal/inbox#1089

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:fix Agentic fix PR eligible for review iteration; min-aw's worker-iterate gates on it. agent:resolve Consent gate: pr-conflict-resolver may rewrite this branch review:approved minimal-review approved the current head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant