Repository navigation
fix(minimald): run every process the daemon injects into a box in the box's leaf - #3593
norrietaylor wants to merge 2 commits into
Conversation
… box's leaf A lifecycle hook now joins the box's classifier leaf before it joins the box's namespaces, as an exec does: both build their command through one placed_injection_command, which also carries the UDP 443 cohort check. The shim's leaf join is now fatal for every leaf, not only a deny-all box's, so an injected process that cannot join its box's leaf stops before its program runs rather than run in the daemon's own leaf. NET-079 gains the two sub-requirements this binds.
|
Warning Review limit reached
This review includes 13 billable files and costs up to $3.25.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Or wait 6 minutes for your next included review. View limit details
|
Spec lint
Advisory. None of this blocks the merge; the review decides. Advisory, and never a merge gate. Linter: |
|
Approved: no findings at Nothing to do. Clobber: none seen. |
|
/arch-review |
There was a problem hiding this comment.
arch-review: PASS · the full ruling
The change conforms to §4.1 line 54. A process outside the boxes cgroup subtree takes the node-plane identity, and minimald must keep a box's placement inescapable. Exec and hook injections now take one path, placed_injection_command, and the shim refuses any failed leaf join before setns. A box with no leaf joins nothing, so the §7.4 exception for a host without enforcement still holds. One small fix: the comment at spec line 717 says the exception is "above", but it is below.
Ruled at 5fefdd0 against gominimal/arch@327e1b57d16b by claude-opus-5-5.
bryan-minimal
left a comment
There was a problem hiding this comment.
Read at 5fefdd0, by tracing the code; no build and no run on a host with the classifier. The placement is sound: every injection (exec, detached exec, the direct-tcpip dial, all four hook events) now builds its command in placed_injection_command, the shim writes its own pid to the leaf first thing in shim_main (single-threaded, before setns and the fork), a failed write exits before setns, and a box with no leaf behaves as before. Nothing blocking. Four should-fix points, mostly about what a failure looks like and what the tests can catch.
1. A failed join reads as the command failing (should-fix). The shim prints minimald: joining the box's classifier leaf <cgroup path>: <errno> (join_classifier_leaf in nsenter.rs), then main returns the error, so a second line Error: Other("joining the box's classifier leaf …") follows and the process exits 1.
min box exec b -- makeon an allow-list box whose leaf write is refused: two lines in the daemon's terms, no next step, and exit 1, which a script cannot tell frommakefailing.- An exec that races
min box stop: the leaf is gone, so the person reads "No such file or directory" about a cgroup path where "the box stopped" is what happened.
This was already so for deny-all leaves. The PR makes it so for every host-address box. Exec propagates the command's exit code, so a refusal by Minimal wants its own code (as the exit 5 refusals have) and one plain sentence.
2. The in-box dial for direct-tcpip now fails without a trace (should-fix). dial_in_box in connection.rs sends the shim's stderr to null. Before, a failed join on a box that is not deny-all was advisory and the dial ran. Now the shim exits 1, the channel opens and closes at once, and the daemon logs nothing because spawn() succeeded. The forward looks dead with no cause. Logging the shim's exit status in the daemon would be enough.
3. The "every verdict" test cannot catch what it is named for (should-fix). an_injection_that_cannot_join_its_box_leaf_is_refused_for_every_verdict calls join_classifier_leaf directly, and that helper no longer looks at the verdict, so the loop over deny and allow adds nothing. Put an is_deny() advisory branch, or a let _ =, back at the call in shim_main and all three new tests still pass. No test goes through shim_main.
4. No test covers the exec half of the new spec line (should-fix).
Host::command_in_sessionexists only outside test builds. Replace itsplaced_injection_command(…)call withinjection.command()and every test passes.- The spec cites
the_classifier_leaf_travels_on_the_shims_argv_when_one_is_setfor exec. That test checks how theInjectioncommand is put together, not that exec uses it. a_hook_injection_joins_the_box_leafcoversInjectedCommands, nothook_planpassingself.leaf, and it runs withhost_ip_enforced: false, so the UDP 443 check never runs for a hook.- Nothing runs against a real cgroup. I can run the host-address scenarios on a classifier host against this head, as offered on #3588, if that helps.
5. The body overstates the gap (nit). Before this PR the headless hooks (activate at finalize, the re-run on resume or restart, detach, destroy) already went through HostHandle and command_in_session, and were placed. The one hook that ran in the daemon's leaf was the attach hook, which runs inside the host loop (run_hooks_for(Attach) → InjectedCommands). Also "attach" in the new spec line has no injection of its own: the attach shell is the launch, which #3588 covers.
6. A question, not about this diff. min task run in a host-address box: the task gets its own sandbox and task_network passes no leaf, while a package build gets one (NET-155). I could not confirm which leaf the task's own process runs in. Does NET-079 mean to cover it?
Also checked and right: a shim that dies on a failed join leaves the box sound (settle_injection gets ESRCH and skips it), and a blocked leaf's directory is never removed, so an exec cannot race a lifted block into ENOENT. I agree with the arch bot's note about where the exception sits in the spec.
Not checked: the real-cgroup refusals (a daemon under systemd Delegate=, for one), the exact text a tty exec prints, and the two test failures the body says also fail on main.
🤖 Generated with Claude Code
|
@bryan-minimal yes please, the same here. A fix for your review is in progress and will push a new head shortly. Running against that head would cover your points 1 and 2 on a real cgroup: an exec whose leaf join is refused, an exec racing |
…words and log the dial's end
|
@bryan-minimal thanks, addressed in 7eda83f:
This head is ready for the classifier-host run you offered: an exec whose leaf join is refused, an exec racing |
|
/review |
|
/arch-review |
There was a problem hiding this comment.
arch-review: PASS · the full ruling
The change conforms to §4.1 of the egress design. Every injected process (exec, hook, forwarded-port dial) joins the box's classifier leaf before it joins the namespaces. A failed join stops the program before it runs, for every verdict. The new exit codes 2 and 5 agree with the exit-code table in architecture.md. Hooks now take the UDP 443 cohort check, as an exec does.
Ruled at 7eda83f against gominimal/arch@8b824707cde6 by claude-opus-5-5.
bryan-minimal
left a comment
There was a problem hiding this comment.
Second round, read at 7eda83f by tracing the code; no build and no run. All four points from the first round are answered. Nothing blocking. Two small should-fix points and three nits.
The first round's points
- The failed join is one line with a code of its own, and
min box execreports it as a refusal only when the code, the first stderr chunk and the absence of earlier output all agree (exec_refusal,reads_as_refusal), so a command that exits 2 with its own stderr stays the command's result. A script that reads only$?still cannot tell the two apart, as with the existing exit-5 refusals; I accept that. watch_in_box_dialreaps the dial's child and logs exits 2 and 5 at WARN with the code.- The every-verdict case runs the real shim (
tests/nsenter_integration.rs): make the join advisory andecho ranexits 0, so it goes red. an_exec_and_a_hook_into_a_placed_box_join_its_leafgoes red ifinjected_commandorhook_commandsdrop the leaf, and the CLI test goes red if the mapping goes.
New
- The cause behind exit 5 is lost for a plain exec (should-fix). The shim prints "Did not run the command: the host could not apply the box's network rules to it ()."
min box execreplaces it with its own line, which has no cause and no next step;relayed_statusreturns the refusal and discards the held line, and the CLI test asserts an empty stderr. The daemon logs onlyexec ended exit_code=5. So for an exec without-tnobody can see whether the write was refused for permission, a busy group or something else. Keep the cause in the CLI's line and namemin bugas the next step, as #3588's refusal does. - A stop race may read as a rules failure (should-fix, one line; the errno is from memory, please check it). ENOENT is mapped to "The box stopped before the command started." If the shim has
cgroup.procsopen and the stop's removal of the leaf lands before the write, I believe the write fails with ENODEV, not ENOENT, and the person gets the exit 5 "network rules" text for what was a stop. If that is right, treat ENODEV as ENOENT injoin_refusal. I will try to catch it in the run on a classifier host. - Nit: the dial's stderr still goes to null, so the WARN has the code but not the shim's one line. Piping it and logging the line would give the cause.
- Nit: a refused
--detachexec shows only as exit 5 frombox wait, not as a refusal. - Nit: two one-line seams are still untested: the call from
command_in_session(built only outside tests) toinjected_command, andChildStdio::newcallingwatch_in_box_dial(the dial test runs the watcher by itself).
Checked and right: ENOENT on the leaf means the box stopped in every path I traced (the leaf is removed only in SandboxBackend::drop, after the box is reaped, and on an abandoned launch; place_pid never creates it); the two new texts say what happened in the person's words.
The run on a host with the classifier installed is being set up against this head and 4435a72 on #3588: an exec whose join is refused, an exec racing min box stop, a forward whose dial cannot join, and a launch whose supervisor placement fails. I will post what it shows on both PRs.
🤖 Generated with Claude Code
What changes
InjectedCommands), now joins the box's classifier leaf before the box's namespaces, as an exec does. The headless hooks (activate at finalize, the re-run on resume or restart, detach, destroy) already went throughcommand_in_sessionand were placed. Exec and hooks now share oneplaced_injection_command, which also runs thehost_ipcohort's UDP 443 check (BEP-018).min box execreports these as " stopped before the command started." (exit 2) and "Did not run the command in : the host could not apply the box's network rules to it." (exit 5).direct-tcpip) ended. A dial that never ran is a warning that gives its exit code.Package builds are out of scope here: main runs them in the box's leaf (NET-155), and #3256 moves them into build boxes. A task's own process in a host-address box is covered by #3610.
Spec
NET-079 gains two sub-requirements. First, a process the daemon starts inside a host-address box that has a classifier leaf, by joining its namespaces (an exec, a lifecycle hook, the dial of a forwarded port), runs in that leaf and never in the daemon's. The attach shell is the box's launch, not an injection. Second, a process that cannot join the leaf is stopped before its program runs, whatever the box's declaration, and ends with an exit code of its own that
min box execreports as a refusal.Tests
nsenter_integration::an_injection_that_cannot_join_its_box_leaf_is_refused_for_every_verdict(drives the real shim)nsenter_integration::an_injection_whose_box_leaf_is_gone_says_the_box_stoppednsenter::tests::a_failed_leaf_join_is_one_line_and_an_exit_code_of_its_ownnsenter::tests::an_injection_joins_its_box_leaf_by_writing_its_pidbox_runtime::tests::an_exec_and_a_hook_into_a_placed_box_join_its_leafhooks::tests::a_hook_injection_joins_the_box_leafhooks::tests::a_hook_into_a_host_address_box_takes_the_udp_443_checkconnection::tests::an_in_box_dial_that_did_not_run_is_logged_with_its_exit_codeminimal tests::box_exec_reports_an_injection_that_never_started_as_a_refusalVerification
Linux (umask 022):
just fixandjust clippyclean;-p minimald1536 passed,-p minimal1195 passed,-p minimald-rpc91 passed, no failures. macOSjust test: 1721 passed. Not run locally:just test-cross,just e2e,just test-root-integration; nothing has run against a real cgroup yet.