Repository navigation
T86 — Let the switch's forwarder reach a box's published ports through the egress gate - #1905
Conversation
The host egress gate's control-surface rule refused every frame a box sent to the plan's gateway, resolver queries excepted, so the one box-to-gateway flow a published port carries — the reply half of a connection the switch's forwarder opened to it — was dropped on the host and a host request through the node's published hostname proxy never got its answer (NET-059). The gate now keeps a ledger of those connections (ForwarderFlows): the ingress leg records a flow from the dial's SYN alone — a gateway-to-row TCP SYN whose destination port the row's admitted set holds at that moment, never adopted mid-stream — and the control-surface check admits a gateway-bound frame only when it matches the exact reverse tuple of a live recorded flow, never by port. The carve-out falls through to the decision behind it, exactly as the resolver's does, so a row's own rules still decide the reply. An entry ends on either side's FIN or RST, on the idle bound, and on revocation — a port leaving its row's admitted set or the row's withdrawal purges its flows — and the table is bounded: full of live flows it refuses a new one rather than evicting, after first reclaiming what idleness already ended, with one rate-limited warn line per interval. One debug line records each flow the gate records and each that expires, beside the existing per-refused-frame drop line and a test-facing gauge of live flows. A frame a box sends to the gateway on its own, from any port, its admitted ports included, is still refused unless it is a resolver query.
Adds the published_proxy_routes_from_host case to scripts/session-e2e.sh (NET-059/081): a host curl through the VM's published loopback proxy port to a box's min.internal name must come back 200 with the box's marker. The reply leg is the one the forwarder flow table in the egress gate now admits — the proxy answers the switch's dial from the gateway address, which the control-surface rule refuses on sight unless the dial's SYN recorded the flow. The unit tests beside the gate pin the refusals; this case pins the delivery. The case takes the same two VM-lane gates the box-registration proof takes (the CLI VM-backed, a switch present) and runs last in the whole-lane order, because it stops and respawns both daemons for its own run.
…itch-s-forwarder-reach-a-box-s-publish
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
Approved: no findings at Nothing to do. Verdict APPROVE, posted as a comment: the App cannot review its own pull request; the check and the label carry it. Clobber: none: |
Main's reply-flow records (ReplyTables, delivered packets toward a registered box at an inside port a publish dials) and this branch's forwarder flow table (ForwarderFlows, the gateway's own dials at a row's admitted ports) both answer the control-surface refusal, each at a different end of a published connection, so the resolution keeps both mechanisms side by side: - the reply record admits its exact reverse ahead of every rule; - the forwarder tuple is the control surface's second exception, and falls through to the row's rules for everything else. Every relay and verdict signature now carries both tables, the retire paths drain them together, and the e2e script carries both sides' cases.
|
Round 1 of 6 on a conflict with the base branch, pushed as Resolved conflicts in:
Verify On this merged branch the switch's forwarder can reach a box's published ports through the egress gate: the switch-to-guest leg records a TCP forwarder flow only from a gateway-to-row SYN at a port in that row's admitted set (no mid-stream adoption), the control-surface check admits a gateway-bound frame only on the exact reverse tuple of a live recorded flow, and a box-initiated frame to the gateway from any port is still refused unless it is a resolver query — all seven gate tests plus the relay-level proof node_proxy_reply_to_forwarder_reaches_switch are green, and the end-to-end case published_proxy_routes_from_host sits in scripts/session-e2e.sh for the VM-backed lanes, self-skipping on this KVM-less native host. Notes from the agent: Revise round 1 was conflict-only: the directive named no review threads, so there are none to judge and this account carries no 'threads' array. The conflict was semantic, not textual — this branch's T86 forwarder flow table and main's T77 reply-flow records independently added reply admission to the same gate. The resolution keeps both mechanisms side by side rather than folding one into the other: ReplyTables records a delivered opening packet toward a registered box at the inside port one of its applied publishes dials and its record admits the exact reverse ahead of every rule; ForwarderFlows records the gateway's own dials at a row's admitted (external) ports and its tuple is the control surface's second exception, falling through to the row's rules for everything else. Every relay and verdict signature now carries both tables (replies: ReplyTables ahead of flows: ForwarderFlows), and the retire paths drain both together. Two signatures that crossed clippy's 7-argument threshold in the merge (relay_switch_frames_to_guest, gate_verdict) gained #[expect(clippy::too_many_arguments, reason = …)] consistent with the file's existing expects. The merge commit (ba3f9e5) also carries main's changes in crates/minimald/src…[truncated] |
|
/revise Round 2: reuse the reply-flow table main already has, and keep only what it lacks. Merge main first. T77 #1839 landed
Out of scope: UDP, and any change to |
T77 (#1839) landed the reply-flow tables this gate already holds, and design 7.1 wants one decision: the forwarder's reply now rides ReplyTables::reply_admits, as any other inbound flow's does, and the parallel ForwarderFlow, ForwarderTable and ForwarderFlows go, with their constants, their wiring and the ingress and egress blocks that fed them. The one gap the shared table did not answer stays answered: an applied unexpose retraction ends the box's reply-flow records at the retracted publication's inside port in the same step the ledger drops its attribution, so the next reply frame on them is the rules' again. The end itself is the new shared ReplyFlows::end_port, which retains the records whose protocol or destination port differ. The control-surface comment keeps its round-1 corrections on main's text: the reply the record above admits never reaches the check, and the record opens at the publish's inside port, not the row's declared one.
The seven proofs the plan named, re-pointed at the reply-flow tables the merge brought in, with no table of the forwarder's own: the forwarder's dial is recorded by the same shared decision every other inbound flow is, and the box's answer to it rides `reply_admits`, ahead of the control-surface check — so the two `gate_verdict` units pin the order (the answer passes flat past a deny-all row and past the switch's own address rule; the node proxy's own connects to the gateway, and the answerer port, stay refused; the resolver carve-out is unchanged). The windows and the cap the shared table keeps are the forwarder's too: the expiry proof drives the hand-held clock the real-clock decision cannot take, shows the expired record is removed rather than refused, and the re-dial that records again; the cap proof drives `shrink_cap_of` to one flow, pins the refusal at ingress — the connect fails, the frame never reaches the box, the live record is kept, nothing evicted — and the refused flow's answer refused in turn while the live flow answers. The retraction takes the publication's records with it: two mappings, two records, an applied unexpose of one — the row withdrawn first, because a retraction at a live row is refused until the row's published set exists — and the count drops by exactly the retracted mapping's record. The mid-stream proof keeps the SYN-only rule on the gateway source and the FIN-and-ends read, and the node-proxy proof is the e2e case's unit at relay level. The purge's frame-level difference is not here to be seen in today's tree — an applied retraction never co-occurs with a live row, and the row's absence refuses the answer on its own — so that proof pins the record-level half and says so in place.
…itch-s-forwarder-reach-a-box-s-publish
|
Round 2 of 6 on a /revise note, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. On a VM lane, the host request through the node's published hostname-proxy port reaches a box by name and gets its answer (the kept e2e case published_proxy_routes_from_host, which this box could not run for want of KVM), because the gate now answers the forwarder with the reply-flow table main already has: the delivered opening dial at a published inside port records by the same shared rule as any other inbound flow, and the box's answer rides reply_admits ahead of the control-surface check — a frame a box sends to the gateway on its own, from any port, is still refused unless it is a resolver query. The publication's retraction ends its records in the same step (egress::ReplyFlows::end_port), a full table refuses the next connect instead of evicting a live one, and the seven gate tests show each half. Notes from the agent: Round 2 of the directive, on the merged tree. The parallel forwarder table (ForwarderFlow, ForwarderTable, ForwarderFlows and wiring) is deleted; no fix to observe_delivered's admission was needed, because a forwarder dial already reaches it today — its destination is a row and its port is the applied publish's inside port, the exact bound the caller checks, which the tests prove. Out of layer, as the directive's item 2 sanctions: egress::ReplyFlows::end_port added to crates/sessions/src/core/egress.rs (main lacked it after the merge; it retains the records whose proto or destination port differ) — name it in the PR body. The plan's observability line said 'a gauge of live forwarder flows'; that gauge is gone with the table it belonged to, and live flows are reported through the existing per-box record count (ReplyTables::record_count_of, test-only) and the shared table's own transition lines (first record, first fill, cap refusals) — likewise the diagnostics line's 'one debug line per forwarder flow recorded / expired': the shared table's own lines are the ones the daemon log keeps. The purge's frame-level effect is unreachable in …[truncated] |
Main gained the same method when the listen-publish task merged, so the branch defined it twice (E0592). Keep main's copy and drop this branch's; both end every record at the given protocol and destination port.
# Conflicts: # scripts/session-e2e.sh
|
Round 4 of 6 on the runtime's review findings, pushed as
Verify Outside the task's
Converted to a draft: something above needs a person before review. Review threads the coder judged:
At this branch the seven minvmd gate proofs pass: the forwarder's dial to a box's published port records a reply flow on the shared table and the box's exact-reverse answer passes the egress gate, while a frame a box sends to the gateway on its own, from any port, is still refused unless it is a resolver query. The e2e case published_proxy_routes_from_host is on the branch for the VM lanes (it self-skipped here; this box has no KVM). This round closes the review thread's lockstep: the gate's udp arm reads egress::IPPROTO_UDP, now pub beside IPPROTO_TCP, so a retraction's purge is keyed by the same constant the recording keyed its records by and no protocol-number divergence can silently leave a box's reply-flow records standing. Notes from the agent: Revise round 4: the only open thread, and the only thing changed. crates/sessions/src/core/egress.rs is this round's one touch outside the task's layers: the thread's stated remedy ('if the shared constant is ever made pub, reference it') needs the shared module's IPPROTO_UDP visible, a one-word change (const -> pub const) plus its doc; the sibling IPPROTO_TCP is already pub 'for the gates' record and reply arms' — the same consumer class the gate belongs to — so this finishes what that doc already names. No behavior change anywhere; the workaround comment in the gate goes with it. Checks beyond the test lines: cargo clippy -p minvmd -p sessions --all-targets --locked -- -D warnings (the justfile recipe's pinned flags at package scope, not --workspace) and cargo fmt --check -p sessions -p minvmd both pass. The e2e line passed here only as a documented self-skip (exit 0): the case is a VM-lane proof and this box has no KVM, so its substance — the host curl through the published proxy port reaching a box by name — still needs a VM lane to be seen. Round 3 answered this same thread with a comment explaining the literal; the reviewer correctly re-raised the duplication itself, so this round removes it rather th…[truncated] |
|
Un-drafting. The one file outside this task's layers is |
published_protocol is called only from relay_control's Expose arm, where the body is an expose body — the same body its sibling published_inside parses as ExposeBody — but it deserialized it as UnexposeBody. That returned the right value only while the two structs happened to share a 'protocol' field; any divergence there (a rename, an enum, deny_unknown_fields) would make it return None, so the publish is never noted, a later retraction reads None from note_retracted, and end_port_at is never called — the records keep admitting a withdrawn publication's replies until their idle cap expires. Parse the body as ExposeBody, the shape the decision (summarize_expose) already admitted and checked.
…itch-s-forwarder-reach-a-box-s-publish # Conflicts: # scripts/session-e2e.sh
|
Round 5 of 6 on the runtime's review findings, pushed as Resolved conflicts in:
Verify Outside the task's
Converted to a draft: something above needs a person before review. Review threads the coder judged:
On a VM-backed host, a host request through the node's published hostname-proxy port reaches a box by name and gets its answer — the gate admits the guest's reply to a connection the switch's forwarder opened, while a frame a box sends to the gateway on its own, from any port, is still refused unless it is a resolver query; this round keeps those replies bounded by making the publish ledger's protocol note read the expose body it is taken from, so a retraction always ends the reply-flow records its publication opened instead of silently failing open if the retraction's body type ever diverges. Notes from the agent: Revise round 5: the directive's one review thread, fixed inside the task's layers (egress_gate.rs, one line): published_protocol deserialized its body as UnexposeBody though its only caller passes an expose body; it now parses ExposeBody like published_inside. The e2e line ./scripts/session-e2e.sh published_proxy_routes_from_host could not be run on this box: there is no /dev/kvm and no vmx/svm CPU flag (jq and cpio are also absent), so no VM can boot — the case exists in the script from an earlier round and stays owed to the VM CI lanes. cargo clippy -p minvmd --all-targets is clean. |
|
Un-drafting. Round 5 (82457fd) parses the publish's protocol from the expose body, and merges main with both e2e case names kept. The only out-of-layer file is still the |
…itch-s-forwarder-reach-a-box-s-publish
…itch-s-forwarder-reach-a-box-s-publish
…flow count The plan's diagnostics line asks for one debug line per flow the gate records and per flow that ends; its observability line asks for a gauge of live flows. Each recorded flow and each end (window passed, close, retraction, row withdrawal) now says so at debug, and live_flows() reads the count across boxes beside refused_at_cap_of.
|
Round 6 of 10 on a failed check suite: the coder changed nothing. every commit this round put on the branch above |
norrietaylor
left a comment
There was a problem hiding this comment.
Approved at aca4669. ARCH PASS (final), minimal-review approved this head, and no open threads. The residual sibling-publish case is tracked in gominimal/inbox#862.
T86 — Let the switch's forwarder reach a box's published ports through the egress gate
slice: S9
kind: task
covers: NET-081, NET-059
size: M
est: 170 code, 260 tests
executor: agent
difficulty: needs care
model: strongest
diagnostics: the VM host daemon's log keeps one egress-switch-control-surface drop line per refused frame, and gains one debug line per forwarder flow the gate records and per flow that expires
observability: the gate's existing drop counters, plus a gauge of live forwarder flows
layers:
- vmhost:
- crates/minvmd/src/net/egress_gate.rs: the switch-to-guest leg records a TCP forwarder flow only from a gateway-to-row SYN whose destination port is in that row's admitted set at that moment (no mid-stream adoption), keyed per row by the exact 5-tuple (gateway address and port, row address and admitted port, TCP); the control-surface check admits a gateway-bound frame only when it matches the exact reverse tuple of a live recorded flow, never by port alone; an entry ends on FIN or RST, on idle timeout, and on revocation, so a port leaving the row's admitted set or the row's withdrawal purges its flows and the next reply frame is dropped; the table is bounded and fails closed, refusing a new forwarder flow when full rather than evicting a live one, with a rate-limited log line; a box-initiated frame to the gateway, from any port, is still refused unless it is a resolver query; UDP is out of scope; the comment that says no box-to-gateway flow at admitted ports exists is corrected
test: cargo nextest run -p minvmd gate_verdict_admits_forwarder_reply_from_admitted_port
test: cargo nextest run -p minvmd gate_verdict_refuses_gateway_dial_from_admitted_port_without_inbound_flow
test: cargo nextest run -p minvmd forwarder_flow_expires_and_table_is_bounded
test: cargo nextest run -p minvmd forwarder_flow_table_full_refuses_new_flow
test: cargo nextest run -p minvmd forwarder_reply_dropped_after_admitted_port_withdrawn
test: cargo nextest run -p minvmd forwarder_flow_not_adopted_mid_stream
test: cargo nextest run -p minvmd node_proxy_reply_to_forwarder_reaches_switch
- e2e:
- scripts/session-e2e.sh: adds the case published_proxy_routes_from_host, a host curl through the node's published 127.0.0.1 proxy port to a box's name, run on the VM lanes with one VM
test: ./scripts/session-e2e.sh published_proxy_routes_from_host
produces: none
consumes:
- crates/minvmd/src/net/egress_gate.rs
proof: On a VM-backed host, a host request through the node's published hostname-proxy port reaches a box by name and gets its answer, because the gate admits the guest's reply to a connection the switch's forwarder opened; a frame a box sends to the gateway on its own, from any port, is still refused unless it is a resolver query.
Proof
On a VM-backed host, a host request through the node's published hostname-proxy port now reaches a box by name and gets its answer, because the host egress gate records the switch forwarder's dial to a row-admitted port as a live flow and admits the guest's reply by the exact reverse tuple; a frame a box sends to the gateway on its own — from any port, at any admitted port included — is still refused unless it is a resolver query.
Notes from the agent: Layer 1 (crates/minvmd/src/net/egress_gate.rs): the switch-to-guest leg records a TCP forwarder flow only from a gateway-to-row SYN whose destination port is in that row's admitted set at that moment (no mid-stream adoption), keyed per row by the exact 5-tuple; the control-surface check admits a gateway-bound frame only when it matches the reverse tuple of a live recorded flow, never by port alone, and still falls through to the unknown-source check, the infrastructure deny set and the row's own rules; entries end on FIN/RST, on a 5-minute idle timeout, and on revocation (a port leaving the admitted set or the row's withdrawal purges its flows); the table is bounded at 1024 and fails closed — a full table refuses a new flow rather than evicting a live one, with a 60s rate-limited warn line — and gains one debug line per recorded flow and per expired flow, which the expiry test asserts. Diagnostics and observability: the existing per-drop egress-switch-control-surface lines are unchanged, the new at-cap warn and record/expire debug lines are pinned by the unit tests, and the live-flow count is readable as a gauge seam (live(), cfg(test) reader like dns_pins'). Layer 2 (scripts/session-e2e.sh): the published_proxy_routes_from_host case gates on the CLI being VM-backed and a switch existing (the box-registration proof's two gates), activates one default-network box with a socat 200-marker responder, reads the published proxy port from min ls's HOSTNAME PROXY line (a parse that takes both the VM and native sp…[truncated]
Tests
cargo nextest run -p minvmd gate_verdict_admits_forwarder_reply_from_admitted_portcargo nextest run -p minvmd gate_verdict_refuses_gateway_dial_from_admitted_port_without_inbound_flowcargo nextest run -p minvmd forwarder_flow_expires_and_table_is_boundedcargo nextest run -p minvmd forwarder_flow_table_full_refuses_new_flowcargo nextest run -p minvmd forwarder_reply_dropped_after_admitted_port_withdrawncargo nextest run -p minvmd forwarder_flow_not_adopted_mid_streamcargo nextest run -p minvmd node_proxy_reply_to_forwarder_reaches_switch./scripts/session-e2e.sh published_proxy_routes_from_hostVerify
min task run agent-verifypassed ([tasks.verify]).Base
The base branch moved while the coder worked; it was merged in before the tests ran.
Closes #1902
Known residual (tracked in gominimal/inbox#862)
Retracting a publish ends the box's reply records at (addr, inside port, protocol). The forwarder's dials come from a switch-owned address that NET-062 drops box traffic to, so after retraction every box's replies on the retracted listener's connections are dropped.
There is one exception. While a sibling publish still dials the same (addr, inside port, protocol), the shared records are kept, and connections accepted through the retracted listener keep working until their reply window passes. The cause is in the forwarder: the pinned gvproxy v0.8.9
Unexposecloses only listeners, not accepted connections (design §7.1). The fix belongs there, and the sibling-publish pin lands with it.Diagnostics and observability (aca4669)
ReplyTables::live_flows()is the live-flow gauge seam, read by the recording test. A minvmd status or metrics surface reads it when one lands, and itsexpect(dead_code)comes off then.