Repository navigation
T79 — Drop a box's flows to the switch's own addresses in the shared verdict - #1943
norrietaylor merged 21 commits into
Conversation
NET-062's sub-clause: the shared frame verdict grows the switch's own addresses — the gateway, the host alias, the node address and the BEP peer the caller hands it — as an opt-in set attached by with_switch_own_addresses, subtracted from whatever the box's egress.allow_subnets admits, as the infrastructure deny set design §5.3 names, and dropped ahead of the declared dimensions under the egress-infrastructure-destination rule string, the one the VM host's gate refuses the same set with. The three port-scoped openings fall through to the box's own rules: the resolver's port on the gateway over TCP or UDP (the UDP carve-out ahead of the set already admits the query under any declaration, and a TCP query keeps the verdict its rules give it); the proxy's listener for a box on a credentialed lane (the NET-134 arms ahead of the set, which decide the peer's whole address once a listener is compiled); and the host alias reached as local reach under the box's rules (NET-003) — the whole alias while no host exposure is configured, the shape the VM host's gate holds and the session e2e proves, and exactly the configured exposure ports once one is, via with_host_exposure_ports. The node address has no opening. The set is absent from EgressRules::new and from_policy, so the Kani oracle's shape over new and the proxy builders still holds and the VM host gate's rows — compiled without it — decide as they did; the gate's own copy in minvmd is not edited.
NET-062's sub-clause on the native relay — the only leg a native host runs, with no minvmd gate beside it: compiled_egress now hands the verdict the switch's own addresses, resolved from the subnet it already carries — the gateway, the host alias, the node address and the BEP peer — so a box whose egress.allow_subnets cover the switch subnet still cannot open a flow to them outside the three port-scoped openings. No host exposure is configured anywhere today, so the set is handed with none and the host alias stays local reach under the box's own rules (NET-003's shape, the one the VM host's gate holds and the session e2e proves); the gateway outside the resolver's port and the node address at every port are the relay's own drops, named by the existing rate-limited warn line under the infrastructure deny set's rule. The same compilation is the hostname proxy's caller check (NET-070), so no reach the relay would refuse can be asked of the proxy instead, and the DropReason arm drop_transport reads carries the new reason's protocol. The VM host gate's own copy in minvmd is not edited.
The set's doc said the resolver carve-out and the proxy listener arms sit 'above' its decision — true of the verdict's order, not of the file, where both live below the set's definition. Name the order instead.
…-s-flows-to-the-switch-s-own-addresses
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Box-to-host reach over the host alias is default-deny except the host's configured exposures (design §7.1, NET-062). The box's rules decide reach within an exposure but never open the alias by themselves, so with no exposure configured the alias now drops at every port. The sessions tests pin the drop for an allow-all box with no exposure, the exposure port falling through to the box's rules, and a non-listener port at the BEP peer dropping for an allow-all box on a credentialed lane. The native relay test moves the alias frame into the dropped set.
The switch lanes run minimald's relay in the guest, so the host alias is default-deny there too (design §7.1). The own-address NET-003 leg now asserts resolution of host.min.internal to the alias only, and the NET-004 leg asserts a silent drop: no reset, an elapsed floor, and the relay's egress-infrastructure-destination line where the daemon log is readable. The deprecation notice is no longer asserted: the relay emits it only for a frame its verdict admits. Both spots carry TODO(host-exposure) for the positive reach arms.
…terim The in-guest hostname proxy listens on the node address, which the own-address set dropped at every port, so a box's proxied request never reached it. While the interim :7654 Host-header proxy is served (design §7.1), TCP to its port on the node address falls through to the box's own rules; every other port, and UDP, stays the set's drop. The opening retires with the proxy. The port is the one the proxy's first bind asks for: the configured or host-handed port, else the documented default. It rides on the switch client into every box's compiled rules and the proxy's caller check alike.
The KVM lane's two controls asserted the shape NET-062's sub-clause replaced: the no-lane case's control read a reset at the daemon's own address (100.64.255.253), and the proxy parity case's pair 5 read the in-guest proxy's 403 through that same address. The shared verdict now drops a box's flows to the switch's own addresses ahead of its rules, silently and never as a reset, so both controls fail a correct lane. The no-lane control now probes the host alias instead — the address gvproxy NATs to the host's loopback for every port, local reach an allow-all row admits on both legs — at the proxy's port, where nothing on the host listens, so the host's kernel resets the connect. That covers every leg the silence probe crosses: resolution, the fabric, both gates' admit path. Pair 5 keeps its caller's literal allow (the subtracted destination, spelled on purpose so the proxied leg's silence is the deny set's, not the rules') and now asserts the closure: the caller's own relay drops its connect to the machine's address, curl 28 with no status, so the request never reaches the proxy — no reach a direct connection would not have. The proxy's caller check itself is unchanged: it runs the same compiled rules at the target's lease, which no member of the deny set is, and its 403 stays pinned by the unit proofs.
…-s-flows-to-the-switch-s-own-addresses
|
Round 1 of 1 on a failed check suite, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. On a native host a box whose allow_subnets cover the switch subnet still cannot open a flow to the switch's own addresses outside the three port-scoped openings — dropped silently, never reset, one rate-limited warn line naming the infrastructure deny set, each opening admitting exactly its port, the VM host gate's own copy untouched — and the KVM e2e lane now agrees: its two controls no longer expect the retired reset at the node address (the no-lane control reads the host alias's reset through gvproxy's loopback NAT instead) nor the in-guest proxy's 403 through that address (pair 5 asserts the caller's own relay drops the connect, curl 28 with no status, so the request never reaches the proxy). Notes from the agent: The directive's failed checks (test-kvm, ci-linux-kvm-success) were caused by two controls in scripts/session-e2e.sh asserting the pre-change shape: the no-lane case's fabric-alive control expected a reset at the node address 100.64.255.253 (the shared verdict now drops it as egress-infrastructure-destination, so socat ran out its window), and the proxy-parity case's pair 5 expected the in-guest proxy's 403 through 100.64.255.253:7654 (the caller's connect is now dropped at its own relay, so no request arrives). The fix belongs in that file, which is OUTSIDE the task's layers (crates/sessions/src/core/egress.rs, crates/minimald/src/net/switch.rs): the layers' unit tests pin the new drop, and no in-layers change can make a lane green whose controls assert the old reset — so I made the two surgical edits there rather than leave the lane red, and name it plainly. The no-lane control now probes the host alias at the proxy's port (gvproxy NATs the alias to the host loopback for every port, nothing listens at 8118 there, so the host kernel resets — the same alias route the NET-004 leg already proves live end to end on every VM lane, and it covers every leg the silence probe crosses). Pair 5 keeps the caller's literal --allow-subnets 100.64.255.2…[truncated] |
This reverts commit 5a219af. Pair 5 keeps reading the in-guest proxy's 403 through the node address, which the interim hostname-proxy opening admits; the no-lane control is moved by the next commit instead.
The no-lane case's control read a reset from the node address at the BEP proxy's port. The own-address set now drops the node address outside the hostname proxy's port, and the host alias at every port, so neither can answer. The control now connects to a sibling own-address box's lease at a port the sibling publishes and nothing listens on, whose kernel resets it. The probe's meaning is unchanged: silence at the BEP proxy's address.
Every box's own-address set opens the node address at the port the proxy's first bind asks for. An unpinned daemon that finds the default busy binds an OS-picked port instead, which no box can reach. The proxy now says so once when it starts serving, naming both ports and telling the operator to pin hostname_proxy_port. Tracked in gominimal/minimal#1952.
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
Box-to-host reach over the host alias is default-deny except configured host exposures (design §7.1, gominimal/inbox#867), so the boxes' own relays drop the escape harness's positive controls at the alias. The harness now binds its listener on the runner's non-loopback interface address, reached through the switch's NAT as ordinary egress and declared by both boxes as a /32. It fails if that address is loopback, inside 100.64/10, or in the reserved local range. It also asserts that the gate logs no infrastructure drop at the address, so its refusals stay the resident union's.
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
|
Approved: no findings at Nothing to do. Verdict APPROVE, posted as a comment: the App cannot review its own pull request; the check and the label carry it. Clobber: none: |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was the runtime's review findings). A person decides now. Remove |
|
The runtime will not revise this pull request again on its own: round 2 would exceed the cap of 1 (the last trigger was a failed check suite). A person decides now. Remove |
norrietaylor
left a comment
There was a problem hiding this comment.
Gates met on 7ac770e: ARCH-EXPERT PASS (C1/C2 for both escape proofs, cross arms on the per-row private-space rule), minimal-review approved with no open threads, every required lane green. KVM evidence is in the PR description (run 37203233656).
Three conflicts, each a collision between this branch's host-alias default-deny work and main's T83/T89: - minimald/src/net/listeners.rs: main's T83 removed the staged-plan table and its four tests outright (the plan now rides Launched, no process-global table), so its side of the region is taken whole; the branch's extra hostname-proxy-port argument survives in the call sites the auto-merge already carried. - minvmd/tests/vm_escape_integration.rs: the escape-union harness keeps the branch's runner-address destination (the host alias is default-deny for a box now, so the positive controls cannot reach it) and takes T89's always-drop posture for the in-plan unregistered source: the admit arm is gone, the Ok arm panics, and the stale phase language in the declaration comment goes with it. - scripts/session-e2e.sh: same reconciliation in the escape-union case — T89's always-refused arms, the branch's listener address as their destination, no esu_alias (undefined in the merged case), no phase keying; the case's own-address legs in min_internal_names_through_proxy are untouched by the merge.
Two helpers the merge brought in from main call SessionGate::for_session with the four arguments main's signature has; this branch's interim node-address opening (16c32e3) made the fifth, the hostname proxy port, and the merged switch.rs carries it. Both helpers take None like every other test call site: no proxy opening is under test in either.
The merge took T89's always-drop posture for the made-up-lease arms and kept this branch's runner-address destination, so the case's descriptive header and its OK line still narrating the shipped interim admit (and the alias destination main's arms used) were two bullets and one clause out of date. They now say what the merged arms assert.
|
Round 1 of 1 on a conflict with the base branch, pushed as Resolved conflicts in:
Verify Outside the task's
Converted to a draft: something above needs a person before review. With main merged in, the branch still drops a box's flows to the switch's own addresses on a native host whatever its allow_subnets say, outside the three port-scoped openings, silently and never reset, named in the existing rate-limited warn line, with each opening admitting exactly its port: all four of the task's test lines pass on the merged tree, and none of the conflict resolutions touched the shared verdict (sessions egress.rs), the native relay's own change (beyond the verdict it is handed), or the VM host gate's own copy (egress_gate.rs). Notes from the agent: Conflict round, no review threads in the directive. Resolved the three markers and completed the merge (10ccfbf), plus two follow-up commits. listeners.rs: took main's T83 side whole — T83 deleted the staged-plan table and its four tests (the plan now rides Launched), so the branch's None-argument edits inside them went with it; the branch's fifth hostname_proxy_port argument survives at every remaining call site. vm_escape_integration.rs: T89's always-drop posture for the in-plan unregistered source (the interim admit arm and the phase keying are gone) with the branch's runner-address destination kept — esu_alias/alias no longer exist in the merged case, so the panics and rows name esu_host/host; that harness is #![cfg(minvmd_libkrun)] plus #[ignore] MINVMD_E2E=1, so it compiles to nothing on this Linux box: verified by forcing the cfg through clippy (compile and lint, no link, no run); its runtime proof stays with the VM lanes. session-e2e.sh: the escape-union case takes T89's always-refused arms with the branch's listener address as their destination, and the case's header bullets and OK line — which still narrated the shipped interim admit and the alias destination — were settled with them; the min_internal_names_through_proxy case the test line runs was untouched by the merge (all the merge's hunks fall inside the escape-union case). Two auto-merge compile breaks fixed in a follow-u…[truncated] |
norrietaylor
left a comment
There was a problem hiding this comment.
Reviewed at 32a2cb1: the coder's merge matches the locally verified one; 46d6389 threads the proxy-port argument through main's new gate call sites; the e2e escape-union header is now accurate. ARCH PASS (KVM evidence run 37203233656 recorded). All lanes green on head; the verify draft was agent-runtime#278 skew outside the diff.
|
/arch-review |
|
/arch-review c6815c4 |
|
/arch-review |
1 similar comment
|
/arch-review |
|
/arch-review c6815c4 |
T79 — Drop a box's flows to the switch's own addresses in the shared verdict
slice: S9
kind: task
covers: NET-062
size: S
est: 120 code, 200 tests
executor: agent
difficulty: needs care
model: strongest
diagnostics: the daemon log tail carries each such drop in the shared refusal line format with the reason naming the switch's own address, so a box that cannot reach its gateway on a port it expected reads why in a bundle
observability: the existing rate-limited egress-drop warn line carries the drop, its reason naming the infrastructure deny set; no new line
layers:
- sessions:
- crates/sessions/src/core/egress.rs: the shared frame verdict subtracts the switch's own addresses (the gateway, the host alias, the node address and the BEP peer address the caller hands it) from whatever a box's
egress.allow_subnetsadmits, as the infrastructure deny set design §5.3 names, except at the three port-scoped openings NET-062's sub-clause lists: the resolver's port on the gateway address; the proxy's address for a box on a credentialed lane; configured host exposures over the host alias, reached as local reach under the box's rules. The verdict for such a flow is a drop, never a reset. The native relay in crates/minimald/src/net/switch.rs is the only leg on a native host and takes the verdict through its existing call; the VM host gate keeps its own copy (egress_gate.rs box_cannot_reach_switch_api) and is not editedtest: cargo nextest run -p sessions switch_addresses_egress_dropped_not_reset
test: cargo nextest run -p sessions switch_address_openings_admit_only_their_ports
- daemon:
- crates/minimald/src/net/switch.rs: hands the verdict the switch's own addresses for the native relay's placement; no other change
test: cargo nextest run -p minimald native_relay_drops_flows_to_switch_addresses
produces: none
consumes:
- crates/sessions/src/core/egress.rs
- crates/minimald/src/net/switch.rs
proof: On a native host, a box whose allow_subnets cover the switch subnet still cannot open a flow to the switch's own addresses outside the three port-scoped openings; the flow is dropped silently and never reset, the drop shows in the existing rate-limited warn line naming the infrastructure deny set, and each opening admits exactly its port; the VM host gate's own copy is unchanged.
depends on:
- blocked by #1800
Behaviour change (read before release)
Box-to-host reach over the host alias (
host.min.internal, and the deprecated literal100.64.255.254) is now default-deny, except configured host exposures (design §7.1). The relay that enforces it also runs inside the VM, so this applies on every host kind as soon as this merges. There is no host-exposure configuration yet (gominimal/inbox#867). Until there is, an own-address box can't reach any host service over the alias, for example a dev database on the host. The owner accepted the gap for this merge.Release note: "Boxes with their own address no longer reach host services through
host.min.internalor100.64.255.254unless the host exposes them. The deprecated literal follows the name's rule."Interim opening: own-address boxes still reach the
:7654Host-header proxy at the node address, on TCP at the configured proxy port only and under the box's own rules (NET-070/071 parity). This opening retires with the proxy. The spec amendment is in #1946.Spec ordering: merged ahead of spec #1946 by user decision. The NET-062 fourth opening and the NET-003/004 clarifications ship before their spec text. #1946 stays open as the follow-up.
Follow-ups: gominimal/inbox#867 (host exposures) and gominimal/inbox#868 (VM host gate parity, and the NET-004 deprecation notice on a dropped connection).
Out of the plan's layers
scripts/session-e2e.sh: e2e layer added to the plan by architecture ruling.crates/minimald/src/{server.rs,session.rs,net/mod.rs,net/gvproxy_network.rs,net/dns.rs,net/proxy.rs,net/listeners.rs}andcrates/minimald/tests/netns_root_integration.rs: thread the configured hostname-proxy port intocompiled_egress, so the relay and the proxy's caller check compile the same rules.Known limitation: an unpinned daemon that finds 7654 busy and binds an OS-chosen port keeps the opening at 7654. That fails closed: own-address boxes can't reach that proxy.
Proof
On a native host, a box whose egress.allow_subnets cover the switch subnet still cannot open a flow to the switch's own addresses outside the three port-scoped openings: the relay (the only leg a native host runs) drops the gateway outside the resolver's port and the node address at every port ahead of the box's rules, silently and never reset, and the drop shows in the existing rate-limited warn line under rule_matched="egress-infrastructure-destination" naming the box, the direction, the switch address and port, and the protocol; each opening admits exactly its port — the resolver's port on the gateway over UDP and TCP alike (falling through to the box's rules), the proxy's listener for a box on a credentialed lane (the NET-134 arms ahead of the set), and the host alias as local reach under the box's rules; and the VM host gate's own copy in minvmd is byte-for-byte unchanged.
Notes from the agent: Design reading that needed deciding: NET-062's third opening says 'configured host exposures over the host alias, reached as local reach under the box's rules', but no host-exposure configuration surface exists anywhere in the tree today. Subtracting the alias with an empty exposure list would drop every host-alias flow and break the spec's own proofs: the un-edited minvmd gate treats the alias as 'local reach, decided by the row's CIDR rules' (its infrastructure_destinations_drop_on_the_host_for_every_row test asserts an allow-all row's alias frame reaches the switch), and the session e2e's NET-003/NET-004 case proves an allow-all own-address box reaches host.min.internal and the deprecated literal at an arbitrary host port through the shared verdict. So the set carries the exposure ports as an explicit parameter (SwitchOwnAddresses::with_host_exposure_ports, defaulting to none): with none configured the alias falls through to the box's own rules — the shape the gate and the e2e hold — and once a host configures exposures, exactly those ports fall through a…[truncated]
Tests
cargo nextest run -p sessions switch_addresses_egress_dropped_not_resetcargo nextest run -p sessions switch_address_openings_admit_only_their_portscargo nextest run -p minimald native_relay_drops_flows_to_switch_addressesVerify
min task run agent-verifyexited 101, so this PR is a draft.Base
The base branch moved while the coder worked and was merged in before the tests ran. These files changed on BOTH sides since the branch point — read them first, because a rewrite from a stale reading can undo a merge without a conflict:
crates/sessions/src/core/egress.rsCloses #1874
CI KVM evidence at 7ac770e (run 37203233656)
minvmd::vm_escape_integration vm_escape_bounded_to_resident_unionPASS (22.6 s).escape reaches only the declared union OK.10.1.0.189, declared as a /32 by both boxes.egress-infrastructure-destinationline named it.egress-unknown-source.egress-undeclared-subnet.egress-undeclared-subnet); pair 4 had no ingress mapping.Rounds after the T79 head stabilised