Repository navigation
fix(common,sandbox2,op,minimald): fetch a box's sources under the box's rules (0.7 backport) - #3182
Conversation
A package build for an own-address box attaches to the box's network after its process starts. A short build step, such as claude-code's chmod and mv, can end before the attach finishes. Its namespace goes with it, the attach fails on `nsenter`, and the launch killed a process that had already ended and failed a build that had worked. An attach that fails after its process ended now leaves that process's exit status as the result. The plan's release still runs. A failed attach of a process that still runs stays a failure. Port to the 0.7 line: the change lands in sandbox2, the line's name for the sandbox crate. main's other call site, the run of an inherit_cwd task in minimald's env.rs, does not exist on the line, so minimald is unchanged here.
On a VM host a source download that leaves as the daemon's own traffic is dropped with no answer, so an activation hung until its RPC timed out at 240 s. See [#3058](#3058). A download now fails within 15 s. A web download bounds its connect across all of a host's addresses, on the placed relay too. The Google Cloud Storage client has no connect bound and retries for minutes, so a `gs://` source bounds the wait for its first answer. That bound ends with the first answer, so a long download is not limited. It also means a `gs://` read gives up retrying a failing first answer after 15 s, not 300 s. The failure names the host, and the build reports the package, the host and the reason. The reason is what the download saw, such as `no answer after 15s`, because no answer can have several causes and the download cannot tell them apart. Port to the 0.7 line: main's box_ops.rs is session_sop.rs here, and main's BoxChannel is SessionChannel in env.rs.
A build the egress rules refused named the package, the host and the rules,
and stopped there. The person who read it had no command to run. It now adds:
To allow it, activate a new session with --network own_ip
--allow-dns-hosts <host>
The step names `--network own_ip` because on the default network the flag
alone is refused at create. A destination's port is dropped from the flag's
value, because a host allowance covers every port of the addresses it pins.
The step appears only on a connection the build opened itself. It is left
out for a literal address, which that flag does not take. It is left out for
a web source too: an own-address box has no classifier leaf, so its build
fetches the source as the daemon's own traffic, and in a VM no rule of the
box lets that out. It names no `minimal.toml` key, because none reaches a box
on this line.
Refs [#3058](#3058)
Port to the 0.7 line: main's boxes crate is sessions here, so
`is_valid_dns_host` is made public there. main's box_actor/tests.rs is
minimald's session/tests.rs.
A step that ended during its network attach left the plan's release to `Drop`, which only spawns it. The next step's plan could run first and replace the reservation, so the release gave back the new one and the old one stayed held. The attach now finishes the release before it returns. `Drop` still covers a cancelled launch.
…resses The relay split its 15 s bound equally over a host's addresses, after the lookup. An address that refused at once wasted its share, so a later address that needed longer could fail with time left. The lookup was outside the bound, so name servers that never answer could exceed it. One deadline now covers the lookup and every address. Each address gets the time left, divided by the addresses left.
The requirement now says where the step appears, that it is for a new session, and when it is left out. The fast-failure requirement names the two new relay tests.
`clippy-strict` flags a `map_err` that discards its error. A `match` says the same thing and keeps the lookup's own error on its own path.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
|
Reviewed: no findings; approval withheld, 1 thread unresolved at APPROVE withheld: 1 unresolved thread — minimal-foundry[bot] crates/minimald/src/sandbox_progress.rs:220. Resolving the last thread releases the approval on this head. Resolve the open threads. Resolving the last one releases the approval on this head; no new review is required. Clobber: none seen in the diff. |
bryan-minimal
left a comment
There was a problem hiding this comment.
Thanks Norrie, this is a faithful backport. I compared every hunk of this PR against #3145, mapping sandbox to sandbox2, boxes:: to sessions::, box_actor/setup.rs to session.rs, box_actor/tests.rs to session/tests.rs, and box_ops.rs to session_sop.rs. Ten files are identical after that mapping: the four common and op library files, sources.rs, specs.rs, sandbox_progress.rs, both root integration tests, and sandbox2/src/lib.rs. The same holds for the session.rs and session_sop.rs hunks. The 0.7 base has the same fetch call sites as main, and sandbox2 already has every API the held network uses. So the enforcement this PR adds is the same as on main, and so is my review of it.
Divergences from #3145
- The
minimal.tomlremedy text is not ported. This touchescrates/minimald/src/env.rs(egress_remedyand its doc, plus the test strings inenv.rsandsession/tests.rs) and the two NET-155 sub-requirements in18-spec-box-networking.md. Class (a), required by the base. Thenetwork.egresskey under[defaults]arrived onmainin #3121, which is not onrelease/0.7.x.mfileon 0.7 has nonetwork.egressreader. The 0.7 doc onegress_remedystill says no key reaches a box (crates/minimald/src/env.rs:2543), which is right for this line. Theremedy_hostremoval is ported, so a refused web orgs://source still gets the--allow-dns-hostsstep, as onmain. - The T108 build-leg update in
scripts/session-e2e.shis not ported. Class (a). The 0.7 script has noproof_escapes_and_builds_stay_inside_box_rules, noe2e-ebrand noegress-node-plane-unpinnedleg, so nothing there needs the change. One consequence: on 0.7, no e2e leg shows that a deny-all box's fetch sends nothing. Only the unit tests and your manual VM run cover it. - A doc comment on
admits_name_answers_as_the_gate_does(crates/minimald/src/net/dns_gate.rs:3707). Class (c), new in the backport. It sits directly above its own test, at the end of the test module, so it does not repeat the misplaced-comment problem fixed onmain. Fine as is.
I found no class (b) divergence. No hunk or test from #3145 is dropped or weakened beyond the two the base requires.
Findings
All of these are the #3145 findings carried over, with 0.7 line numbers. Fixes made on main should be backported here too.
- Medium. Nothing beyond spec-lint has run in CI on this head. f143f00 targets
fix/backport-3107-0.7.x, so therelease/**CI lanes have not run. The description also notes that the root integration tests have not run, and thatjust e2eandjust test-vmstop early on your host. Fix: after the retarget torelease/0.7.x, let the native netns lane runa_placed_fetch_leaves_from_the_held_namespace, and let the KVM lane go green, before merging. On 0.7 those are the only checks of the held namespace that do not depend on one developer machine. The--network noneactivations atscripts/session-e2e.sh:5597and:6376now need a cached or network-free closure, so they are worth watching in that run. - Low. A failure to hold the namespace reaches the user without the host.
crates/op/src/sources.rs:90passes the sandbox error through as it is. ATaskSlotsBusyor spawn failure then reaches the user as a rawother:error that names neither the source nor the box. Fix as onmain: give the error thecould not connect to <host>:<port>:prefix sosource_failurereports it. - Low. Fetches compete for the box's four task addresses on a VM. The cause is the same as on
main:TASK_SLOTS_PER_BOX = 4(crates/minimald-rpc/src/lib.rs:629), with the slot released asynchronously atcrates/sandbox2/src/lib.rs:3203. This is a question, not a confirmed defect. - Low. A
nonebox's builds are logged as running "on the box's own network, under the box's egress rules". Seecrates/minimald/src/env.rs:489,:1094and:1381. - Low. The cache-hit path for a
noneor deny-all box has no test. The same holds for the "no query was sent" claim ina_name_the_rules_do_not_allow_sends_no_query. - Nit. Two different next steps for a
nonebox, atcrates/minimald/src/env.rs:2452and:2460. - Nit. The held rootfs is created in the system temp directory, but its error names
base_dir(crates/sandbox2/src/lib.rs:3227).
Process
- The description says "Merge SHA on
main: to be filled in when #3145 merges". Please fill it in before this merges, so the 0.7 history points at its source. - The release note is good. It names the user-visible change for sessions with no network, which is the one an RC tester is most likely to hit.
Checked and fine
- Hunk-for-hunk equivalence with #3145 for every shared file, after the crate and module renames.
sandbox2on 0.7 provideshold_start,plan_launch,forker::on_fork_thread,Spawned::from_child,NetGuard,NetPlan::resolverandNoNet. The new test's helpers (box_base_dir,Recorder,user_namespaces_restriction) exist too.admits_nameon 0.7 matches the 0.7DnsGate::refusesandis_zone_name, which are the same as onmain.- 0.7 has no extra source-fetch entry point. The
SourceLoadandSpecBuildconstruction sites matchmain, and the orchestrator carriesbox_network. - The anonymous
gs://read overhttpsloses no access on 0.7 either, becausemctxbuildsRemoteStoragewithout gcloud auth.
Verdict
Approve with nits, the same as #3145. Nothing in the code blocks 0.7.3-rc2. Merge it after #3181 and #3145 have landed, once the release/0.7.x CI lanes are green and the merge SHA is filled in. Any fix you take on main for findings 2 or 4 should come across in the same backport.
Reviewed with lab tooling on the branch head; the lab round, where one ran, is cited in the text. Questions welcome.
Split the unplaced web download into download_web_within, which takes the connect bound the way download_gcs_within does. The 15 s path calls it, and the error reports the bound the client was built with. A Linux test points it at a port whose accept queue is full. It checks that the download fails within twice its bound and that the error reads "could not connect to 127.0.0.1:<port>: no answer after 300ms".
A gs:// download read every timeout as no answer within the bound. The storage client's own timeouts carry a different source, so only the error BoundedStorage builds, whose source is tokio's Elapsed, now reads "could not connect to storage.googleapis.com:443: no answer after <bound>". Any other timeout keeps the client's text.
A placed connect to a host with several addresses reported only the last address's share of the bound, as in "no answer from 1.2.3.4:443 after 3.75s". When the last address gives no answer, the whole bound has passed, so the error now reads "no answer after 15s", the same as a lookup that gets no answer. Each address's own result goes to the debug log. When the last address refuses after earlier ones gave no answer, the error keeps the refusal and counts the others, as in "Connection refused (os error 111), after no answer from 3 other addresses".
…'s rules
A package build for a box now fetches its sources where the box's egress
rules decide them.
A `gs://` source for a box is read anonymously over `https` through the
placed download, in the box's classifier leaf or on the box's network. The
Cloud Storage client stays for a build with no box.
An own-address box has no classifier leaf, so its fetch runs on a network of
the box's own. A sandbox with no program holds that network up for the fetch,
and the download creates each socket in its namespace. The held process's
stdio is /dev/null, so a daemon whose stdout is a VM console can hold one,
and a held process that ends before its attach reports why. Name queries go
to that network's name servers. Fetches that overlap share one namespace,
because a box has few task addresses.
A box whose rules admit nothing starts no network, and its fetch sends
nothing. A host that a partial `allow_dns_hosts` list does not name is refused
the same way, with the answer the box's own name query would get.
A box with no network gets neither the daemon's network nor its fetch. A build
that declares it needs the network is refused before it fetches anything. The
check reads the graph, because a build with `override_deps` has a plan that
gives every build the network. A build with no such need runs in an empty
namespace, so a box whose packages are all cached keeps working. A build with
no box is unchanged.
A refused source now names the host and the flag that allows it:
the build of <package> could not reach <host>: the egress rules of box
<box> do not allow it. To allow it, activate a new session with
--network own_ip --allow-dns-hosts <host>
No source is recorded as node-plane traffic any more.
Refs [#2802](#2802)
Port to the 0.7 line:
- sandbox is sandbox2, in op, in minimald's tests and in the new
`hold_network`.
- main's box_actor/setup.rs is minimald's session.rs here: the new
`box_build_network` and the `admits_name` hook sit beside the line's
`own_address_build_network`, at its two callers. main's box_ops.rs is
session_sop.rs, and main's box_actor/tests.rs is session/tests.rs.
- boxes::EgressPolicy is sessions::EgressPolicy in dns_gate.rs's new
`admits_name`. Its test keeps a doc comment of its own, so main's
follow-up that moved a misplaced comment has nothing to do here.
NET-155 now covers `gs://` sources, own-address boxes and boxes with no network, with a verify line for each behaviour. Its scope note says that a build runs under the rules of the box that requested it until build boxes exist. NET-080 says that a source is never node-plane traffic. Refs [#2802](#2802)
f143f00 to
6491395
Compare
| // A source is a package build's traffic, under the box's rules | ||
| // (NET-155), and never the node plane's. | ||
| Operation::FetchSource { url } => { | ||
| tracing::debug!( | ||
| box_id = %record.box_id, | ||
| host = %url_host(url), | ||
| object = %url_object(url), | ||
| "a web source fetched in the box's classifier leaf is the box's own fetch, \ | ||
| so the node plane keeps no record of it", | ||
| "a source is the box's own fetch, so the node plane keeps no record of it", | ||
| ); | ||
| return; | ||
| } |
There was a problem hiding this comment.
LOW · correctness · confidence 0.50 · in record_fetch
record_fetch now skips every FetchSource that has a scheme, and FetchRecord::sources_placed is gone. A box with no classifier leaf and no build network still has place == None. Its web and gs:// sources then leave from the daemon's own cgroup, as op::sources shows. Those fetches are node-plane traffic, and they had a record before this change. Now they have none. Keep the record when the fetch has no place, or tell the reader that this loss is deliberate.
What the reviewer checked
sources.rs:
let Some(place) = place else { ... download_web_with_verification_and_caching(url, sha256, None, op) ... }; box_build_leaf returns Ok(None) for an unplaced box; record_fetch:Operation::FetchSource { url } => { ... return; }
Hand this to an agent
Act on a code-review finding.
File: crates/minimald/src/sandbox_progress.rs
Lines: 210-220
Category: correctness
Severity: low (reviewer confidence 0.50)
Finding:
`record_fetch` now skips every `FetchSource` that has a scheme, and `FetchRecord::sources_placed` is gone. A box with no classifier leaf and no build network still has `place == None`. Its web and `gs://` sources then leave from the daemon's own cgroup, as `op::sources` shows. Those fetches are node-plane traffic, and they had a record before this change. Now they have none. Keep the record when the fetch has no place, or tell the reader that this loss is deliberate.
First verify this against the code yourself. It is model output and
may be wrong. If it is wrong, or the tradeoff was deliberate, say so
and change nothing.
👍 useful · 👎 noise — react to tell the reviewer. Replying works too, and says more.
Only Linux joins a namespace, so on macOS the user and net handles of NetNamespace are never read, and clippy -D warnings refuses them.
An attach that touches no namespace succeeds over an ended process, so hold_network returned a pid whose namespace was gone, and the first error came later as a missing /proc entry. Check the held process after a successful attach too, and report its own exit reason.
There was a problem hiding this comment.
APPROVE withheld: 1 unresolved thread — minimal-foundry[bot] crates/minimald/src/sandbox_progress.rs:220.
minimal-review — advisory review
Verdict COMMENT — No findings.
Panel: 2 of 3 personas ran (correctness, security); did not complete: supply-chain.
Backports #3145, which fixes #2802, to the 0.7 line for 0.7.3-rc2. Merge SHA on
main:2b91ba87(squash). The squash makes the same changes as the reviewed head323704f6, and this branch carries every commit of that head except those listed under not ported. Builds on the #3107 backport (#3181), merged intorelease/0.7.xas412b9d24.release/0.7.xis merged into this branch at9ce67e60, so the branch still carries #3181's commits, which the squash duplicates. Againstrelease/0.7.x, this PR's diff is only its own commits (256204d8,5d9985f0,6491395a,1d84947c,8e9a6dd2).A box's package sources,
gs://included (read throughstorage.googleapis.com), are fetched under the box's egress rules:own_ipbox, in a network namespace held open on the box's network. The held process has/dev/nullstdio and reports why it ended.A box with no network gets no fetch and no daemon network.
Adapted from
mainsandbox→sandbox2throughout.main'sbox_actor/setup.rschanges went intominimald/src/session.rs:box_build_networkandwith_admits_namesit besideown_address_build_network, at both of its callers.box_ops.rs→session_sop.rs.box_actor/tests.rs→session/tests.rs.boxes::EgressPolicy→sessions::EgressPolicyindns_gate.rs.main's doc-comment follow-up on the NET-141 test is folded in.minimal.toml'snetwork.egress.allow_dns_hostsin the refusal. 0.7 does not read that key, so the refusal keeps the--allow-dns-hosts <host>wording.scripts/session-e2e.shhas no such leg.Release note
Package sources for a session, including Google Cloud Storage sources, are now downloaded under that session's egress rules. Before, they were downloaded as the host's own traffic. A session with no network no longer downloads sources at all.
Verification
On Linux (Lima, arm64), through
just:fmt-checkandclippyare clean, andjust clippyis clean on macOS.testpasses 3963, with the same 4 host-dependent failures as the #3107 backport, each failing the same way on unmodifiedrelease/0.7.x. The new tests pass.On a microVM (nested KVM on Lima), with a package whose source is
gs://, and gvproxy captured:--network own_ip --allow-dns-hosts storage.googleapis.com: the package builds, and its fetch connects to Google's addresses.--network own_ipwith no allowed hosts: refused withthe build of gsrc could not reach storage.googleapis.com: the egress rules of box gs-own-deny do not allow it. To allow it, activate a new session with --network own_ip --allow-dns-hosts storage.googleapis.com. The capture holds nothing naming the host and no connect to a Google address.Native Linux (Lima, arm64), through the real daemon under its AppArmor label, with the daemon captured and a local listener up:
--network own_ip --allow-dns-hosts storage.googleapis.com: the package builds. The name query goes to the box's gateway, and the fetch connects to Google from the held namespace.--network own_ipwith no allowed hosts: refused in 5 s with the text above. Nothing for the host leaves: no name query, no connect to Google, and the listener sees nothing.just e2eandjust test-vmdo not complete on that host, on this branch or on unmodifiedrelease/0.7.x(98a6e0ac), and both stop at the same point on all three trees:just e2epasses 18 groups, then fails the terminal relay's resize check, and the script stops there.just test-vm's first test waits 10 s for a guest that takes about 140 s to boot. Theminvmdcrate is unchanged here.So neither gives evidence past those points, for or against this branch.
just test-root-integrationrefuses most of its tests on that host, because a classifier tree is already installed there, and that happens on unmodifiedrelease/0.7.xtoo. The newa_placed_fetch_leaves_from_the_held_namespacefails there as well: its held process ends before the test opens its namespace. The test's network has no attach step, andhold_networknow checks the held process after a successful attach too, so this failure reports the process's own exit reason. Under the daemon the attach is real, and the native run above shows the held namespace working.