Skip to content

feat(otel): guest telemetry rides the VM channel (7/7, spec 25) - #2039

Merged
bryan-minimal merged 1 commit into
mainfrom
otel/7-vsock
Oct 14, 2026
Merged

bryan-minimal merged 1 commit into
mainfrom
otel/7-vsock

Conversation

@bryan-minimal

@bryan-minimal bryan-minimal commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

feat(otel): guest telemetry rides the VM channel (7/7, spec 25)

Stack: spec #1988 → test hardening → mlog export → CLI+daemon → minvmd+guest → diag → requirement tests → this → (docs in the spec PR). Base: otel/6-tests.

Answers the spec review's first blocker (guest export rode the node network's temporary allow-all and would stop silently once minimal#1786 enforces the baseline; no network-mode hole covers every box). The guest never uses its network for telemetry:

  • Guest side (minimald in guest mode, mlog): with MINIMAL_OTEL_FORWARD=vsock:7353 on the boot line, mlog queues each spool line (the same OTLP-JSON line the spool writes, so TEL-014/015 hold) for a sender thread that writes length-delimited frames to the host over vsock port 7353 (CID 2). Bounded queue (1024 lines / 4 MiB), 5 s reconnect, drop-and-count when the host is away; nothing in the guest waits on the host (TEL-046). No OTLP exporter is built in the guest.
  • Host side (minvmd): the run supervisor binds an owner-only telemetry door beside the control socket; the VMM child wires port 7353 for that VM only (the same shape as the READY marker and the report door). The door checks each line strictly (one OTLP request per line, ≤ 1 MiB, ≤ 4096 lines/s per connection; malformed → dropped and counted, never logged), stamps it with minimal.forwarded_by=minvmd and minimal.vm=<name>, writes it to the host spool (so min bug and the lab see guest records) and forwards batches as OTLP/JSON under the host's switches — host off or DO_NOT_TRACK ⇒ the door never opens; signal off ⇒ spooled only. Forwarding runs on its own thread behind a bounded queue, so a hung collector never stops the door reading.
  • Boot line: MINIMAL_TELEMETRY, MINIMAL_OTEL_FORWARD, the filter and the exporter-off switches, TRACEPARENT — no endpoint and no header, ever. The endpoint-shape and secret-shape code and the url dependency leave minvmd (TEL-033).
  • Peer binding: the port exists only for that VM, and sandbox2's socket-family seal admits no AF_VSOCK in any box plan — pinned by sandbox2::no_seal_admits_af_vsock — so only the guest's pid-1 daemon can reach the door. Works for every box networking mode, including network = none, and identically on Linux (KVM) and macOS (HVF): both use libkrun's single implicit vsock device.
  • Trace shape unchanged: guest.ready under vm.boot, one trace id from cmd to the guest.

Evidence: VM-lane test crates/minvmd/tests/otel_vsock_integration.rs (boots a guest under MINVMD_E2E=1, checks guest.ready in the host spool with the stamps, the forwarded JSON at /v1/traces, and a boot line with FORWARD and no endpoint); unit tests for framing, drop/reconnect, strict parsing, oversize, batching and the stuck-forwarder case; lab (Cloud Hypervisor, debian-12): 715b/715f/715g/715h pass with the dual-shape scenarios, plus 715i (relay bounds: malformed, flooded and oversized frames refused and counted, no guest bytes logged), 715j (collector down: the guest never blocks, spans still land stamped), 715k (graceful stop: guest flushes before the Shutdown ack) and 715l (provenance stamps on every guest span) — all eight pass. Design note: docs/specs/25-spec-telemetry/guest-vsock.md. Cargo.lock: +vsock (minimald), −url (minvmd).

Plan: #1992. Plan tasks in this PR: the guest-telemetry path (a new task under the VM slice; the plan comment will be amended).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Guest trace and log telemetry can now be forwarded to the host and sent to configured telemetry services. Forwarded records include VM identification and host provenance.
    • Diagnostic bundles now include guest telemetry spool records.
  • Bug Fixes
    • After a guest acknowledges shutdown, the VM gets time to exit cleanly before termination signals are sent.
    • Telemetry queues receive a bounded drain period during shutdown so pending records can be forwarded.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review:approved minimal-review approved the current head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants