Repository navigation
fix: bound minimald host mailbox sends so a wedged host can't hang attach or stop #1369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
5e083ab
348f2f3
5b9bec7
47fab2f
257879c
6cc7493
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -18,7 +18,7 @@ use std::os::fd::{AsRawFd, FromRawFd, IntoRawFd, OwnedFd, RawFd}; | |||||||
| use std::time::SystemTime; | ||||||||
| use tokio::io::AsyncWriteExt; | ||||||||
| use tokio::io::unix::AsyncFd; | ||||||||
| use tokio::sync::mpsc::error::SendError; | ||||||||
| use tokio::sync::mpsc::error::{SendError, SendTimeoutError}; | ||||||||
| use tokio::sync::{mpsc, oneshot}; | ||||||||
| use tokio::task::JoinHandle; | ||||||||
| use tracing::Instrument as _; | ||||||||
|
|
@@ -1350,9 +1350,20 @@ impl HostHandle { | |||||||
| } | ||||||||
|
|
||||||||
| pub async fn kill(&self, for_shutdown: bool) -> Result<(), ()> { | ||||||||
| match self.sender.send(Message::Kill(for_shutdown)).await { | ||||||||
| match self | ||||||||
| .sender | ||||||||
| .send_timeout( | ||||||||
| Message::Kill(for_shutdown), | ||||||||
| crate::session::HOST_PROBE_TIMEOUT, | ||||||||
| ) | ||||||||
| .await | ||||||||
|
Comment on lines
+1355
to
+1359
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🌐 Web query:
💡 Result: In Tokio's tokio::sync::mpsc::Sender::send_timeout API, receiving Ok() confirms only that the message was successfully enqueued into the channel [1][2]. It does not guarantee that the receiver has processed the message [1][2]. The function is designed to wait until there is capacity available in the channel's buffer to store the value [1][2]. Once the message is placed into the channel, the function returns Ok(), regardless of whether a receiver has retrieved or processed it at that time [1][2]. If the channel is full and cannot accommodate the new message within the specified duration, the function returns an error [1][3]. Citations:
🤖 get_repo_knowledge executed:
Length of output: 2949 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- session_host.rs relevant methods ---'
sed -n '1280,1420p' crates/minimald/src/session_host.rs
printf '%s\n' '--- session.rs shutdown path ---'
sed -n '1585,1665p' crates/minimald/src/session.rs
printf '%s\n' '--- host message definitions and receiver loop ---'
rg -n -C 6 'enum Message|Message::Kill|Message::Attach|send_timeout|HOST_PROBE_TIMEOUT|recv\(|try_recv\(' crates/minimald/src/session_host.rs crates/minimald/src/session.rs crates/minimald/src/session.rs
printf '%s\n' '--- channel declarations and host operation callers ---'
rg -n -C 5 'mpsc::|Sender<|Receiver<|Host::|host\.attach|host\.kill|attach\(' crates/minimald/src/session_host.rs crates/minimald/src/session.rsRepository: gominimal/minimal Length of output: 50374 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- exact changed diff summary ---'
git diff --stat
git diff -- crates/minimald/src/session_host.rs crates/minimald/src/session.rs | sed -n '1,260p'Repository: gominimal/minimal Length of output: 190 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- HostHandle probe tests ---'
sed -n '2960,3045p' crates/minimald/src/session.rs
printf '%s\n' '--- HostHandle-related tests ---'
rg -n -C 8 'HostHandle::|wedged|attach.*timeout|kill.*timeout|probe_host|HOST_MAILBOX_CAPACITY' crates/minimald/src/session_host.rs crates/minimald/src/session.rs
printf '%s\n' '--- host construction and loop ownership ---'
rg -n -C 10 'fn build|async fn mainloop|JoinHandle|HostHandle \{|mpsc::channel\(HOST_MAILBOX_CAPACITY' crates/minimald/src/session_host.rs
printf '%s\n' '--- host message handling and internal attach ---'
sed -n '3225,3310p' crates/minimald/src/session_host.rs
rg -n -C 8 'async fn attach\(' crates/minimald/src/session_host.rsRepository: gominimal/minimal Length of output: 45426 Wait for host-operation completion, not only mailbox enqueue.
📍 Affects 2 files
🤖 Prompt for AI Agents
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [worker-iterate:addressed] The attach-side acknowledgement (attach returning
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The attach path remains open. Please add a per-operation acknowledgement for attach in this PR, or open a follow-up issue for the deferred attach acknowledgement. I can review the follow-up change separately. You are interacting with an AI system. |
||||||||
| { | ||||||||
| Ok(()) => Ok(()), | ||||||||
| Err(_e) => Err(()), // closed | ||||||||
| // A closed channel, or a wedged loop that never drains its mailbox | ||||||||
| // so the send cannot be queued before the deadline: either way the | ||||||||
| // kill did not land, and the caller must not block awaiting a loop | ||||||||
| // that will never observe it. | ||||||||
| Err(_e) => Err(()), | ||||||||
| } | ||||||||
|
Comment on lines
1352
to
1367
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. LOW · The same wedge this PR bounds for What the reviewer checked
Hand this to an agent👍 useful · 👎 noise — react to tell the reviewer. Replying works too, and says more. |
||||||||
| } | ||||||||
| /// Binds `c` to this host, carrying the attaching terminal's facts. | ||||||||
|
|
@@ -1373,11 +1384,18 @@ impl HostHandle { | |||||||
| ) -> Result<(), (Channel<Msg>, WinSize)> { | ||||||||
| match self | ||||||||
| .sender | ||||||||
| .send(Message::Attach(c, sz, connection, keys)) | ||||||||
| .send_timeout( | ||||||||
| Message::Attach(c, sz, connection, keys), | ||||||||
| crate::session::HOST_PROBE_TIMEOUT, | ||||||||
| ) | ||||||||
| .await | ||||||||
| { | ||||||||
| Ok(()) => Ok(()), | ||||||||
| Err(SendError(Message::Attach(c, sz, _, _))) => Err((c, sz)), | ||||||||
| // A wedged loop that never drains its mailbox (the send times out) | ||||||||
| // and a loop that has gone (the channel is closed) both hand the | ||||||||
| // channel back; the caller re-mints a host from it either way. | ||||||||
| Err(SendTimeoutError::Timeout(Message::Attach(c, sz, _, _))) | ||||||||
| | Err(SendTimeoutError::Closed(Message::Attach(c, sz, _, _))) => Err((c, sz)), | ||||||||
|
Comment on lines
+1397
to
+1398
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. MEDIUM · The new SendTimeoutError::Timeout arm routes a wedged-but-alive host into the attach re-mint path, where the old host's loop task is silently detached and leaks.
Suggested change
What the reviewer checked
Hand this to an agent👍 useful · 👎 noise — react to tell the reviewer. Replying works too, and says more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [worker-iterate:addressed]
|
||||||||
| Err(e) => unreachable!("{:?}", e), | ||||||||
| } | ||||||||
| } | ||||||||
|
|
@@ -4531,6 +4549,37 @@ mod tests { | |||||||
| ); | ||||||||
| } | ||||||||
|
|
||||||||
| /// A kill aimed at a host whose loop has stopped draining its mailbox | ||||||||
| /// gives up on its own deadline instead of parking forever. Left | ||||||||
| /// unbounded, this send blocked once the mailbox filled — the wedged-host | ||||||||
| /// hang behind a stuck `min stop` and, via the same send, `min session | ||||||||
| /// attach`. | ||||||||
| #[tokio::test(start_paused = true)] | ||||||||
| async fn kill_to_a_wedged_host_gives_up_instead_of_parking() { | ||||||||
| // Holding the mailbox is what makes the host wedged: it accepts | ||||||||
| // messages up to its capacity and never drains one. | ||||||||
| let (host, _mailbox) = HostHandle::wedged(); | ||||||||
|
|
||||||||
| // Queue past the mailbox so the next send has to block trying to | ||||||||
| // enqueue at all — the case an unbounded send never returned from. | ||||||||
| for _ in 0..HOST_MAILBOX_CAPACITY { | ||||||||
| host.kill(false) | ||||||||
| .await | ||||||||
| .expect("queuing into an open mailbox should succeed"); | ||||||||
| } | ||||||||
|
|
||||||||
| // Far longer than the send's own deadline, so under the paused clock | ||||||||
| // the send's timeout is the one that fires; reaching this outer bound | ||||||||
| // would mean the send had no deadline at all. | ||||||||
| let outcome = tokio::time::timeout(Duration::from_secs(600), host.kill(false)) | ||||||||
| .await | ||||||||
| .expect("a bounded kill must return on its own deadline, not park forever"); | ||||||||
| assert!( | ||||||||
| outcome.is_err(), | ||||||||
| "a kill that cannot be queued before the deadline must report failure", | ||||||||
| ); | ||||||||
| } | ||||||||
|
|
||||||||
| /// A [`sandbox2::NetGuard`] that records whether its teardown ran, so a test | ||||||||
| /// can assert the session's network is released exactly when the shell | ||||||||
| /// process ends — and left up while it is merely detached. | ||||||||
|
|
||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LOW ·
style· confidence 0.60 · agreed by 1 personas (generalist)The new comment on the attach-time hook-host replacement narrates the PR to itself: "the very 'attach hangs with no timeout' symptom this change removes" describes the pre-fix behavior and has no referent once merged — the same class of self-referential comment this PR already trimmed out of
stop_running(review thread onstop_running). Keep the durable half (bounds the wait for the hook host to wind down, aborting a wedged loop rather than parking attach) and drop the symptom sentence.What the reviewer checked
Hand this to an agent
👍 useful · 👎 noise — react to tell the reviewer. Replying works too, and says more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[worker-iterate:addressed]
Addressed: dropped the self-referential symptom clause ("the very "attach hangs with no timeout" symptom this change removes") that has no referent once merged, matching the trim already applied to the
stop_runningcomment on this PR. Kept the durable half — bounds the wait for the hook host to wind down, aborting a wedged loop rather than parking the attach; same bounded kill-and-stop as shutdown, seeSession::kill_and_stop_loop. Comment-only, behavior-preserving.