Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 58 additions & 17 deletions crates/minimal/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ use std::time::Duration;

use anyhow::Context as _;
use minimald_rpc::OneshotSshRpc;
use tokio::io::{AsyncReadExt, AsyncWriteExt};

/// Add a spinner-style progress bar to the process-global `MultiProgress`.
/// Bars must be inserted before any style/message/tick configuration —
Expand Down Expand Up @@ -219,35 +218,62 @@ impl Client {
///
/// The type parameter `R` picks the RPC from the wire contract; `request`
/// is serialized to JSON and the response is deserialized from JSON.
///
/// Uses `channel.wait()` rather than `channel.into_stream()` so that
/// extended-data (stream 1) — where the daemon writes handler errors
/// (#901) — is visible instead of silently discarded by the stream's
/// `AsyncRead` impl.
pub async fn oneshot_rpc<R: OneshotSshRpc>(
&mut self,
request: R::Request<'_>,
) -> Result<R::Response, anyhow::Error>
where
<R as OneshotSshRpc>::Response: serde::de::DeserializeOwned,
{
let channel = self
let mut channel = self
.handle
.channel_open_session()
.await
.with_context(|| format!("open channel for {}", R::NAME))?;

send_traceparent(&channel).await;
// want_reply = true so an unknown subsystem (CLI/daemon version
// skew) surfaces as a Failure instead of the client writing into a
// channel nobody serves (#901).
channel
.request_subsystem(false, R::NAME)
.request_subsystem(true, R::NAME)
.await
.with_context(|| format!("request subsystem {}", R::NAME))?;

let body = serde_json::to_vec(&request).context("serialize request")?;
channel.data_bytes(body).await.context("write request")?;
channel.eof().await.context("shutdown write half")?;

let mut rpc = channel.into_stream();
rpc.write_all(&body).await.context("write request")?;
rpc.shutdown().await.context("shutdown write half")?;

// Drain the channel with wait() rather than into_stream() so that
// extended-data (stream 1) — where the daemon writes handler errors
// (#901) — is visible instead of silently discarded by the stream's
// AsyncRead impl.
let mut resp_buf = Vec::with_capacity(256);
rpc.read_to_end(&mut resp_buf)
.await
.context("read response")?;
let mut err_buf = Vec::new();
while let Some(msg) = channel.wait().await {
match msg {
russh::ChannelMsg::Data { data } => {
resp_buf.extend_from_slice(&data);
}
russh::ChannelMsg::ExtendedData { data, ext: 1 } => {
err_buf.extend_from_slice(&data);
}
_ => {}
}
}
Comment on lines +240 to +268

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Handle ChannelMsg::Failure — the loop currently ignores a refused subsystem and can hang.

The comment at Lines 240-242 states want_reply = true so an unknown subsystem "surfaces as a Failure instead of the client writing into a channel nobody serves," but the drain loop's _ => {} arm swallows ChannelMsg::Failure. Since request_subsystem does not consume its reply (it arrives via wait(), as open_exec_channel and download_diag_bundle both demonstrate), and a bare refusal does not close the channel (see the note at Lines 588-593), wait() will block indefinitely on a CLI/daemon version skew instead of surfacing the failure. oneshot_rpc has no surrounding timeout, so this is an unbounded hang on exactly the path this PR aims to fix.

🔒️ Proposed fix: bail on subsystem refusal
         while let Some(msg) = channel.wait().await {
             match msg {
                 russh::ChannelMsg::Data { data } => {
                     resp_buf.extend_from_slice(&data);
                 }
                 russh::ChannelMsg::ExtendedData { data, ext: 1 } => {
                     err_buf.extend_from_slice(&data);
                 }
+                russh::ChannelMsg::Failure => anyhow::bail!(
+                    "daemon refused the {} subsystem (likely a CLI/daemon version skew)",
+                    R::NAME
+                ),
                 _ => {}
             }
         }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// want_reply = true so an unknown subsystem (CLI/daemon version
// skew) surfaces as a Failure instead of the client writing into a
// channel nobody serves (#901).
channel
.request_subsystem(false, R::NAME)
.request_subsystem(true, R::NAME)
.await
.with_context(|| format!("request subsystem {}", R::NAME))?;
let body = serde_json::to_vec(&request).context("serialize request")?;
channel.data_bytes(body).await.context("write request")?;
channel.eof().await.context("shutdown write half")?;
let mut rpc = channel.into_stream();
rpc.write_all(&body).await.context("write request")?;
rpc.shutdown().await.context("shutdown write half")?;
// Drain the channel with wait() rather than into_stream() so that
// extended-data (stream 1) — where the daemon writes handler errors
// (#901) — is visible instead of silently discarded by the stream's
// AsyncRead impl.
let mut resp_buf = Vec::with_capacity(256);
rpc.read_to_end(&mut resp_buf)
.await
.context("read response")?;
let mut err_buf = Vec::new();
while let Some(msg) = channel.wait().await {
match msg {
russh::ChannelMsg::Data { data } => {
resp_buf.extend_from_slice(&data);
}
russh::ChannelMsg::ExtendedData { data, ext: 1 } => {
err_buf.extend_from_slice(&data);
}
_ => {}
}
}
// want_reply = true so an unknown subsystem (CLI/daemon version
// skew) surfaces as a Failure instead of the client writing into a
// channel nobody serves (`#901`).
channel
.request_subsystem(true, R::NAME)
.await
.with_context(|| format!("request subsystem {}", R::NAME))?;
let body = serde_json::to_vec(&request).context("serialize request")?;
channel.data_bytes(body).await.context("write request")?;
channel.eof().await.context("shutdown write half")?;
// Drain the channel with wait() rather than into_stream() so that
// extended-data (stream 1) — where the daemon writes handler errors
// (`#901`) — is visible instead of silently discarded by the stream's
// AsyncRead impl.
let mut resp_buf = Vec::with_capacity(256);
let mut err_buf = Vec::new();
while let Some(msg) = channel.wait().await {
match msg {
russh::ChannelMsg::Data { data } => {
resp_buf.extend_from_slice(&data);
}
russh::ChannelMsg::ExtendedData { data, ext: 1 } => {
err_buf.extend_from_slice(&data);
}
russh::ChannelMsg::Failure => anyhow::bail!(
"daemon refused the {} subsystem (likely a CLI/daemon version skew)",
R::NAME
),
_ => {}
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/minimal/src/client.rs` around lines 240 - 268, Update the
response-draining loop in oneshot_rpc to explicitly handle
russh::ChannelMsg::Failure and return an error immediately when the subsystem
request is refused. Preserve collection of Data and ExtendedData messages, but
do not let the fallback arm swallow Failure; ensure the error identifies the
requested subsystem using R::NAME.


if !err_buf.is_empty() {
anyhow::bail!(
"{} RPC failed on the daemon side: {}",
R::NAME,
String::from_utf8_lossy(&err_buf)
);
}

serde_json::from_slice(&resp_buf)
.with_context(|| format!("decode response for {}", R::NAME))
Expand Down Expand Up @@ -461,15 +487,24 @@ impl Client {
let writer: Box<dyn tokio::io::AsyncWrite + Unpin + Send> = Box::new(channel.make_writer());
stream(writer).await?;

// Wait for the channel to close to signal that unpacking is done.
// The daemon relays any unpack failure on extended-data stream 1 and
// only then closes, so accumulate that stream under the same cap the
// diagnostic download uses — a daemon spraying extended data must not
// be able to balloon the client's memory.
// Drain the channel: collect extended-data (stream 1) errors from
// the daemon under the same cap the diagnostic download uses, and
// track whether we see an explicit Eof/Close to distinguish a clean
// post-unpack close from a dropped connection (#901). Without this
// check a connection drop mid-unpack looks identical to a successful
// close — empty err, Ok(()) — and cmd_activate proceeds with an
// empty workspace.
let mut err = Vec::new();
let mut saw_close = false;
while let Some(msg) = channel.wait().await {
if let russh::ChannelMsg::ExtendedData { data, ext: 1 } = msg {
append_daemon_error(&mut err, &data);
match msg {
russh::ChannelMsg::ExtendedData { data, ext: 1 } => {
append_daemon_error(&mut err, &data);
}
russh::ChannelMsg::Eof | russh::ChannelMsg::Close => {
saw_close = true;
}
_ => {}
}
}
if !err.is_empty() {
Expand All @@ -478,6 +513,12 @@ impl Client {
String::from_utf8_lossy(&err)
);
}
if !saw_close {
anyhow::bail!(
"upload stream ended unexpectedly: the connection to the daemon \
was lost before unpacking completed"
);
}
Ok(())
}

Expand Down
103 changes: 85 additions & 18 deletions crates/minimald/src/rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use std::pin::Pin;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::task::{Context, Poll};
use tokio::io::{AsyncRead, AsyncReadExt, AsyncWriteExt, ReadBuf};
use tokio::io::{AsyncRead, ReadBuf};
use tokio::task::spawn;

use crate::{
Expand All @@ -39,29 +39,54 @@ trait ServeOneshot: OneshotSshRpc {
/// Helper to deserialize the request and serialize the response
/// down the given SSH channel, calling the provided async handler
/// function to compute the response.
async fn handle_channel<F>(&self, c: RuChannel<Msg>, handler: F) -> Result<(), ConnectionError>
///
/// On a handler error the message is written to SSH extended data
/// (stream 1) before the channel is closed, so the client sees a
/// legible error instead of an opaque EOF (#901).
async fn handle_channel<F>(
&self,
mut c: RuChannel<Msg>,
handler: F,
) -> Result<(), ConnectionError>
where
F: for<'a> AsyncFnOnce(Self::Request<'a>) -> Result<Self::Response, ConnectionError>,
{
let mut stream = c.into_stream();

// Read the request by draining Data messages until Eof or the
// channel closes. Using wait() directly (rather than into_stream)
// so the channel remains available for extended_data_bytes() and
// close() on the error path (#901).
let mut buf = Vec::with_capacity(1024);
stream
.read_to_end(&mut buf)
.await
.map_err(russh::Error::from)?;
while let Some(msg) = c.wait().await {
match msg {
russh::ChannelMsg::Data { data } => buf.extend_from_slice(&data),
russh::ChannelMsg::Eof | russh::ChannelMsg::Close => break,
_ => {}
}
}

let request: Self::Request<'_> = serde_json::from_slice(&buf)?;
let response = handler(request).await?;
let response_bytes = serde_json::to_vec(&response)?;
// All error paths — JSON parse, handler, serialization — are funneled
// through the same match so the client always sees a legible message
// on extended data instead of an opaque EOF (#901).
let result: Result<Vec<u8>, ConnectionError> = async {
let request: Self::Request<'_> = serde_json::from_slice(&buf)?;
let response = handler(request).await?;
Ok(serde_json::to_vec(&response)?)
}
.await;

stream
.write_all(&response_bytes)
.await
.map_err(russh::Error::from)?;
stream.flush().await.map_err(russh::Error::from)?;
stream.shutdown().await.map_err(russh::Error::from)?;
Ok(())
match result {
Ok(response_bytes) => {
c.data_bytes(response_bytes).await?;
c.eof().await?;
c.close().await?;
Ok(())
}
Err(e) => {
let _ = c.extended_data_bytes(1, e.to_string()).await;
let _ = c.close().await;
Err(e)
}
}
}
}

Expand Down Expand Up @@ -1479,6 +1504,48 @@ mod tests {
)));
}

/// When the daemon fails to parse a request (or the handler errors),
/// the error message must reach the client on extended data instead
/// of the client seeing an opaque EOF on the response stream (#901).
#[tokio::test]
async fn oneshot_rpc_surfaces_handler_errors_as_extended_data() {
use russh::ChannelMsg;

let server = TestServer::new().await;
let mut client = server.connect().await;

// Send invalid JSON to a known subsystem: handle_channel will fail
// at serde_json::from_slice, which now writes the error to extended
// data before closing the channel.
let mut channel = client.open_subsystem(GetVersion::NAME, &[]).await;
channel
.data_bytes(b"not valid json".to_vec())
.await
.unwrap();
channel.eof().await.unwrap();

let mut err_buf = Vec::new();
let mut data_buf = Vec::new();
while let Some(msg) = channel.wait().await {
match msg {
ChannelMsg::ExtendedData { data, ext: 1 } => err_buf.extend_from_slice(&data),
ChannelMsg::Data { data } => data_buf.extend_from_slice(&data),
_ => {}
}
}

assert!(
!err_buf.is_empty(),
"expected an error on extended data, got data={:?}, err=empty",
String::from_utf8_lossy(&data_buf),
);
assert!(
data_buf.is_empty(),
"expected no response data on error, got {:?}",
String::from_utf8_lossy(&data_buf),
);
}

#[tokio::test]
async fn get_version_returns_compiled_in_versions() {
let server = TestServer::new().await;
Expand Down
34 changes: 25 additions & 9 deletions crates/minimald/src/test_harness.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ use paths::DaemonAbsPath;
use russh::keys::ssh_key;
use sessions::SessionId;
use tempfile::TempDir;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{UnixListener, UnixStream};

use minimald_rpc::OneshotSshRpc;
Expand Down Expand Up @@ -209,17 +208,34 @@ impl TestClient {
/// Panics on any transport or codec failure — appropriate for unit
/// tests, which want loud failure rather than recovery.
pub async fn call<R: OneshotSshRpc>(&mut self, req: &R::Request<'_>) -> R::Response {
let channel = self.handle.channel_open_session().await.unwrap();
channel.request_subsystem(false, R::NAME).await.unwrap();
let mut channel = self.handle.channel_open_session().await.unwrap();
channel.request_subsystem(true, R::NAME).await.unwrap();

let body = serde_json::to_vec(req).expect("request serializes");
let mut stream = channel.into_stream();
stream.write_all(&body).await.unwrap();
stream.shutdown().await.unwrap();
channel.data_bytes(body).await.unwrap();
channel.eof().await.unwrap();

let mut resp_buf = Vec::with_capacity(1024);
let mut err_buf = Vec::new();
while let Some(msg) = channel.wait().await {
match msg {
russh::ChannelMsg::Data { data } => resp_buf.extend_from_slice(&data),
russh::ChannelMsg::ExtendedData { data, ext: 1 } => {
err_buf.extend_from_slice(&data)
}
_ => {}
}
}

if !err_buf.is_empty() {
panic!(
"{} RPC failed on the daemon side: {}",
R::NAME,
String::from_utf8_lossy(&err_buf)
);
}

let mut response_buf = Vec::with_capacity(1024);
stream.read_to_end(&mut response_buf).await.unwrap();
serde_json::from_slice(&response_buf).expect("response deserializes")
serde_json::from_slice(&resp_buf).expect("response deserializes")
}

/// Opens an SFTP session attached to the given minimald session.
Expand Down
Loading