Repository navigation
T71 — Hold the box-zone answerer as a host service installed by the privileged step - #1924
Conversation
… both The daemon connects to the answerer channel first and publishes its zone rows there — to the installed host service when the privileged step put one in, or to another VM host daemon holding the port as the recorded single-operator interim — and hosts the answerer itself only when no channel socket exists and the hook port is free. A channel that is present but refuses or times out is a surfaced error, never a reason to host, and a node that loses its connection reconnects with backoff and re-publishes, so a service restart is a short absence window, never a session restart. The new answerer subcommand is the one the service manager runs, as the operator's uid, over the sockets the manager holds and hands it (systemd LISTEN_FDS on Linux, launchd socket activation on macOS): it answers the box zone from the names it holds itself plus the rows nodes publish over the channel, gating each peer on its uid before a byte is read, refusing a name a connected node already holds and any address outside the host-answerable range, and holding each node's rows only while its connection is up.
…ox-zone-answerer-as-a-host-service-ins
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
…ox-zone-answerer-as-a-host-service-ins
The interim holder's death leaves its channel socket file on the disk with nothing listening behind it, and the next daemon read that file as a machine that has an answerer: the connect is refused, the channel is 'present but not answering', and the daemon sat in the surfaced error forever, never hosting again — the KVM lane's fresh min ls after a stopped session stopped naming the VM host daemon as the zone's answerer. A refused connect now names the corpse for what it is: a service manager's socket never refuses a connect (the manager accepts it and starts the answerer) and a live holder never refuses one either, so the file is nobody's and the daemon takes the machine's answerer, replacing the leftover when it binds the channel. The refusal a live answerer answers the hello or a publish with is PermissionDenied now, so ConnectionRefused stays the connect's own kind. The macOS arm of the channel's uid gate also never compiled: libc does not export getpeereid on apple targets (the unit lane's Clippy step exited 101 on it). It reads LOCAL_PEERCRED with an xucred instead — the kernel's own answer for the peer's uid, decided at connect time — the same gate the Linux arm keeps. The pinned test keeps its name and its contract — a channel present that refuses the hello is an error surfaced at start, never a reason to host — against a channel that is alive and says no, the wire shape a version-mismatched or misconfigured service answers with; the corpse gets its own proof that the machine recovers.
…s ran out the coder's uncommitted work as it stood when its 7200s ran out The coder was stopped at the end of its time with these changes not committed. The runtime committed its working tree as it stood so the work is kept: this commit is a snapshot, not a finished step, and may not build or pass its tests.
…ox-zone-answerer-as-a-host-service-ins
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
3 similar comments
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
Round 1 of 10 on a failed check suite, pushed as The coder ran out of its 7200s before it finished. If it left work uncommitted, its last commit is the runtime's snapshot of that work as it stood — the commit message says so — and not a finished step: it may not build or pass its tests.
Verify Outside the task's
Converted to a draft: something above needs a person before review. Two VM nodes on one host answer their names from the one answerer the service manager holds, a stopped session leaves that answerer answering, a node that finds the hook port held publishes its rows over the channel instead of binding and says so at start, a channel peer with another uid is refused, a second node's claim on a held name is refused and reported, an address outside the answer rule is refused, both names answer again after a service restart with no session action, and the advisory's privileged step installs the service as the operator from a root-owned copy. Commit headers reworded by the runtime to fit commitlint. The header the coder wrote is the first paragraph of each of these commits' bodies:
|
…visory The coder's snapshot left the crate unbuildable. The answerer step now reaches the advisory, and the clashes it left are resolved. - The installed copy's protocol probe bound (1 s) is renamed ANSWERER_VERSION_PROBE_BOUND. It no longer collides with ANSWERER_PROBE_BOUND, the 250 ms liveness-datagram bound T65 shipped, which keeps its value. - session_advisory_at takes the AnswererStep. Session start reads it beside the detection and logs whether the answerer is manager-held. - The program path and unit paths split per platform, so both command renders compile under test without a duplicate definition. - The two CLI tests the plan names are added. Tests that pinned the old lead-ins and the old Linux command now pin the new ones. - The macOS-only build no longer warns on the LISTEN_FDS helpers that only Linux uses. - The e2e lead-in greps follow the new wording.
… service NET-122 gains the sub-requirement that the privileged step installs the box-zone answerer as a host service. The service manager holds the service's sockets, it runs as the operator, and it runs from a root-owned copy. NET-138's interim sentence now points at that step.
The advisory command now also installs the box-zone answerer host service. Each e2e case that runs it as root records the channel path it binds first. The teardown and the trap then remove the service at the exact paths the command writes. On macOS that is the LaunchDaemon and the program copy. On Linux it is the socket and service units and the program copy. The channel socket goes on both. The Linux cases read only the first line of the command. That line has an unterminated quote now that unit files ride inside the command as heredocs. They now read the whole command, the way the range case does.
The answerer service is now its own program, `min-answerer`, which the privileged step copies to a root-owned path. The units run it. Its two client verbs, `release` and `release-cancel`, run as root and talk to the operator's VM host daemons over their control sockets. - The channel is machine-global: /run/minimal/answerer.sock on Linux, the socket unit's RuntimeDirectory. On macOS it is /Library/Application Support/minimal/run/answerer.sock. Test and debug builds may override it by env; release builds never do. - "Installed" is decided by the unit file, never by the socket path. A stale socket with no unit file is logged and treated as absent. - Nodes use the global channel when the service is installed. Otherwise they use the interim's per-state-dir channel, which only the hosting node binds. - The node id is the canonical state dir plus the VM name, so name ownership is per node. - The control socket gains release and release-cancel, accepted from root or the operator only. Release frees the port and waits up to 15 s for the service's channel. Cancel re-binds at once, and the daemon re-binds on its own when the window runs out. - The advisory offers the step only on VM-backed hosts. It installs the units, releases this CLI's own daemons, then starts the unit. On either timeout it rolls back, cancels, and exits non-zero.
…rer is manager-held
…nd count not-run cases
Spec lint
Advisory. None of this blocks the merge; the review decides. Advisory, and never a merge gate. Linter: |
The address book refuses another node's address but let a node's second row take the same address as its first, overwriting the first box's holder record; releasing the second would then free an address the first still answers at. A publish now refuses the later row.
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was the runtime's review findings). A person decides now. Remove |
The channel's uid gate closed a foreign peer's connection without a reply, so another operator's daemon surfaced only a closed channel. The gate now drains the peer's hello on a thread of its own and answers a refusal naming the uid the service serves, the peer's uid, and that multi-operator hosts are not supported.
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was the runtime's review findings). A person decides now. Remove |
With no min-answerer to copy, or a path the command cannot quote, the advisory rendered without the service step yet still said the command installs or reinstalls the service. The state's fact now names the remedy only when the step is carried, and the missing-step fact names which of the two left it out.
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
1 similar comment
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was the runtime's review findings). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was the runtime's review findings). A person decides now. Remove |
Each foreign-uid connection to the world-connectable channel got its own refusal thread for up to a second, so a non-operator process could grow the service's threads and fds without bound. At most four refusals now answer at once; past that a foreign peer is closed bare.
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
norrietaylor
left a comment
There was a problem hiding this comment.
Reviewed against T71's plan block. The real-Mac privileged e2e answerer_survives_session_stop passes (handover, two nodes, restart, sibling stop, later publish); KVM proves the Linux service and its loopback-only bind. ARCH PASS. Five review findings fixed in this PR. Follow-ups: gominimal/inbox#889, gominimal/inbox#894, gominimal/inbox#883.
T71 — Hold the box-zone answerer as a host service installed by the privileged step
slice: S9
kind: task
covers: NET-122
size: M
est: 360 code, 320 tests
executor: agent
difficulty: needs care
model: strongest
diagnostics: the bundle's VM host daemon log tail says at start whether the node published its rows into the manager-held answerer or hosted the answerer itself as the interim, with the hook port and the channel path; the answerer service's own log names each node that connected and each row it published or withdrew
observability: one info line at daemon start naming publish-or-host and the hook port; one info line per node connection and disconnection at the answerer service; one warn line per refused channel peer naming its uid
layers:
- vmhost:
- crates/minvmd/src/cmd/answerer.rs (new): the answerer subcommand the service manager runs as the operator's uid, never root: the host's one box-zone answerer on the host loopback at the hook port, its listener socket and its channel socket both held by the service manager and inherited (launchd socket activation on macOS, a systemd system socket unit with User set to the operator on Linux, never a user-session unit), answering with the shared zone decision from rows that node helpers write over the channel, a unix socket whose peer uid must match the service's; the hello carries a node id and a channel protocol version; a name belongs to the connection that first published it and a second node's publish of the same name is refused and reported; every published A address must fall inside the reserved range, the node's addresses or 127.0.0.1 and is refused otherwise; rows live only while the node's connection is up, so a restart holds nothing until nodes reconnect and re-publish
- crates/minvmd/src/net/answerer.rs: hosting the answerer in the daemon becomes the recorded interim: at start the daemon connects to the channel socket first and publishes its rows there (a connect starts a socket-activated service, so a present socket path that refuses or times out is an error surfaced at session start, never a reason to host), hosts the answerer itself only when the channel socket path is absent and the hook port is free (a port held by anything but the service is a surfaced collision, not a fallback), never both; it keeps the channel connection for the session, reconnects with backoff on loss and re-publishes idempotently, so a service restart is a short absence window and never a session restart; one publish path either way
- crates/minvmd/src/cmd/run.rs: makes the publish-or-host choice at start and surfaces the interim at session start beside the resolver advisory
test: cargo nextest run -p minvmd answerer_service_answers_rows_from_two_nodes
test: cargo nextest run -p minvmd helper_publishes_when_service_holds_the_port
test: cargo nextest run -p minvmd answerer_channel_refuses_foreign_uid
test: cargo nextest run -p minvmd answerer_service_forgets_rows_of_a_gone_node
test: cargo nextest run -p minvmd second_node_publishing_a_held_name_is_refused
test: cargo nextest run -p minvmd answerer_refuses_an_address_outside_the_answer_rule
test: cargo nextest run -p minvmd nodes_republish_after_a_service_restart
test: cargo nextest run -p minvmd present_channel_that_refuses_is_an_error_not_a_host
- cli:
- crates/minimal/src/resolver.rs: the advisory's privileged step also copies the answerer program to a root-owned, non-user-writable path and installs the service unit pointing at that copy (a launchd plist with socket activation on macOS, a systemd system socket and service pair running as the operator on Linux), never at a user-writable binary; the hook probe reports whether the answerer is manager-held or session-held and compares the installed copy's channel protocol version with the daemon's, re-surfacing the advisory on a mismatch so an upgrade re-runs the step
test: cargo nextest run -p minimal advisory_installs_manager_held_answerer
test: cargo nextest run -p minimal hook_probe_resurfaces_advisory_on_protocol_mismatch
- docs:
- docs/specs/18-spec-box-networking/18-spec-box-networking.md: NET-122 gains the sub-requirement "WHERE the host is hooked THE SYSTEM SHALL install, by the privileged step, the box-zone answerer as a host service whose listener and channel sockets the service manager holds, running as the operator, from a root-owned non-user-writable program", and NET-138's interim sentence points at it
- e2e:
- scripts/session-e2e.sh: adds the case answerer_survives_session_stop, run where a lane can install the unit (the KVM lane as root with the systemd variant; the macOS lane only if it can install a LaunchDaemon, else it fails rather than self-skips so the pin is never silent): with the service installed, stopping the session leaves a sibling node's name answering, a service restart under two connected nodes answers both names again without a session action, and a session started afterwards publishes instead of binding
test: ./scripts/session-e2e.sh answerer_survives_session_stop
produces:
- crates/minvmd/src/cmd/answerer.rs
consumes:
- crates/minvmd/src/net/answerer.rs
- crates/minimal/src/resolver.rs
proof: Two VM nodes on one host answer their names from the one answerer the service manager holds, a stopped session leaves that answerer answering, a node that finds the hook port held publishes its rows over the channel instead of binding and says so at start, a channel peer with another uid is refused, a second node's claim on a held name is refused and reported, an address outside the answer rule is refused, both names answer again after a service restart with no session action, and the advisory's privileged step installs the service as the operator from a root-owned copy.
depends on:
- blocked by #1710
Out of time
The coder ran out of its 7200s before it finished. If it left work uncommitted, its last commit is the runtime's snapshot of that work as it stood — the commit message says so — and not a finished step: it may not build or pass its tests. So this PR is a draft.
Status at 12f51da (local fixer, architecture conditions applied)
/run/minimal/answerer.sockon Linux,/Library/Application Support/minimal/run/answerer.sockon macOS.AddressBookincrates/minvmd/src/net/answerer.rs.two_nodes_never_share_a_box_address.min-answereris a separate crate (crates/min-answerer, minvmddefault-features = false), so it doesn't link libkrun.just answerer-link-check, part ofjust ci, accepts only system libraries and no RPATH, RUNPATH or LC_RPATH. Its rule set is identical to T91's (T91 — Check that min-answerer links only system libraries #1947).releaseandrelease-cancelgo over the control socket, accepting only uid 0 or the operator (peer credentials). The daemon waits 15 s for the channel, and every failure path re-binds.min lsand the session-start line distinguish three states: manager-held service, hosting the interim, and publishing into another node's interim.MINIMAL_E2E_PRIVILEGED=1, it printsanswerer_survives_session_stop: NOT RUN (no privileged LaunchDaemon install on this lane)and counts it in the NOT RUN summary.Release-blocking follow-ups: T91 #1947 (link check) and T92 #1948 (dist build, staging, packages, installer, advisory signature check, release.yml patch). These supersede gominimal/inbox#869.
Outside the plan's layers:
crates/min-answerer/*Cargo.tomlandCargo.lockcrates/minimald-rpc(release variants andManagerHeld)crates/minimal/src/cmd/session.rsandtests/cli.rscrates/minvmd/src/box_registry.rs(allocated addresses)justfile(answerer-build,answerer-link-check)scripts/check-answerer-links.shSpec: host-wide allocation and the reuse quarantine are added as a NET-010 sub-requirement in spec PR #1946, which merges after this PR. The operator decided on 2026-10-04 that this PR merges ahead of #1946, as T79 does.
Pending before merge: one run of
answerer_survives_session_stopon a real Mac withMINIMAL_E2E_PRIVILEGED=1(the launchd socket-activation handover).Proof
Two VM nodes on one host answer their names from the one answerer the service manager holds, a stopped session leaves that answerer answering, a node that finds the hook port held publishes its rows over the channel instead of binding and says so at start, a channel peer with another uid is refused, a second node's claim on a held name is refused and reported, an address outside the answer rule is refused, both names answer again after a service restart with no session action, and the advisory's privileged step installs the service as the operator from a root-owned copy.
Tests
cargo nextest run -p minvmd answerer_service_answers_rows_from_two_nodescargo nextest run -p minvmd helper_publishes_when_service_holds_the_portcargo nextest run -p minvmd answerer_channel_refuses_foreign_uidcargo nextest run -p minvmd answerer_service_forgets_rows_of_a_gone_nodecargo nextest run -p minvmd second_node_publishing_a_held_name_is_refusedcargo nextest run -p minvmd answerer_refuses_an_address_outside_the_answer_rulecargo nextest run -p minvmd nodes_republish_after_a_service_restartcargo nextest run -p minvmd present_channel_that_refuses_is_an_error_not_a_hostcargo nextest run -p minimal advisory_installs_manager_held_answerercargo nextest run -p minimal hook_probe_resurfaces_advisory_on_protocol_mismatch./scripts/session-e2e.sh answerer_survives_session_stopVerify
min task run agent-verifyexited 101, so this PR is a draft.Base
The base branch moved while the coder worked; it was merged in before the tests ran.
Out of scope
These files are outside the task's
layers:, so this PR is a draft:crates/minvmd/src/cmd/mod.rscrates/minvmd/src/main.rsCloses #1803
Local fixer round (head a465194)
Reuse quarantine: a released box address is held back for the answer TTL (
REUSE_QUARANTINE = ANSWER_TTL_SECS, 15 s). Allocation takes the lowest free address that is not quarantined. The box that held an address may take it back. When the range is exhausted, allocation fails naming the wait. Test:released_address_is_not_reused_within_the_answer_ttl.Per-user interim channel: every state dir of one user resolves the same interim answerer channel:
$XDG_RUNTIME_DIR/minimal/answerer-interim.sock, falling back to/tmp/minimal-<uid>/(created 0700; refused if it has another owner or a wider mode).~/Library/Application Support/minimal/run/answerer-interim.sock.The peer credential check accepts the same uid, plus root for release. Tests:
second_state_dir_publishes_into_the_first_ones_interim,interim_dir_is_created_private_and_a_wide_one_is_refused.Channelless hook-port holder: registration fails loudly when port 7656 is held by something with no channel. The error names the holder and the remedy. Test:
registration_fails_naming_a_channelless_hook_port_holder.Handover: an allocation that arrives during a handover waits for the rest of the window instead of failing. Test:
allocation_during_handover_waits_then_proceeds.e2e: the case builds
min-answererwhen it is not besidemin(just answerer-build, in its own invocation), so the case never reports NOT RUN on the Linux KVM lane. It also asserts that node B, from a second state dir, publishes into node A's interim before the service install.Behaviour change and release note
On a Linux host that runs native
minimaldand VMs together without the answerer service installed, own-address box registration now fails with an error naming the hook port's holder and telling the operator to install the answerer service. Before this change such boxes self-assigned addresses that could collide. T90 closes this once native daemons publish into the channel.Link check: T91 (#1954) owns
scripts/check-answerer-links.sh. When main is merged into this branch, this PR takes T91's script and drops its own copy, and theanswerer-link-checkrecipe keeps calling it.just testanswerer_survives_session_stop(withmin-answererabsent at start)Merge with main: the run-base privacy fix
Merging main brought in T93's node-port reservation (#1957), which created the per-user run base with mode 0755 before the zone answerer started. The interim answerer refuses a base that loose, so it held the hook port with no channel, and a second state dir's node could not publish into it. This happened on every run, not intermittently.
sock::restrict_owned_dir.Folding both callers into one helper is tracked in gominimal/inbox#883.
Follow-ups from review
not_run, which is how the plist's missingSockTypeand itsSockNodekey reached a real Mac.