Repository navigation
fix(minimald): carry an in-box opt-out on the channel request - #3165
Conversation
Since T43 (#2999) min task run starts the task as its box's entrypoint, the in-box min task run. The exec's TRACEPARENT reaches that helper and rides its run request as the trace prefix, but the channel built the task's sandbox from the task's own variables, so the task got no TRACEPARENT and a min it ran started a trace of its own (TEL-023). The channel now adds the trace variables to the task's, as the sandboxed task path does: the request span's TRACEPARENT when that span is recorded (TEL-024), the opt-out marker for a helper that opted out (TEL-045), and nothing with telemetry off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A box whose request opted out of telemetry has MINIMAL_OTEL=off and no TRACEPARENT, so its in-box min sent a bare request. The channel then minted a recorded root for it (TEL-045 says nothing is recorded), and since the task box fix the task could be handed that root's TRACEPARENT. The helper now sends the otel%off% prefix when the box has the marker, and never its TRACEPARENT. The channel makes that request's span non-recording, keeps its line out of the log, and hands a task it runs MINIMAL_OTEL=off. A bare request, such as an older helper's Line form, hands its task no TRACEPARENT: only a request that sent a context does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request channel_request_span returns borrows the line, so the line must outlive the call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TEL-045 lists the env.rs opt-out test and lab scenario 712e, and drops the stale note that its tests come later. TEL-025 says its helper test covers the opted-out prefix. The env.rs test doc pointed at scenario 712c, an unrelated one; it is 712e. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
handle_lossy_line still passed None for handle_raw's old traceparent argument; the parameter is a TaskTrace since the in-box opt-out, so pass TaskTrace::Parent(None), as handle does. The minimald test target did not compile (E0308). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Approved: no findings at Nothing to do. Clobber: none found in the touched code. |
agent-137
left a comment
There was a problem hiding this comment.
Review on Karl's behalf, at 4da824fa7. This is a comment, not an approval. Nothing was built for it: every point comes from reading the code, and each finding says how it was checked.
- CI at
4da824fa: all green. - Reviews: minimal-review APPROVE at head (3/3 personas, no findings). CodeRabbit APPROVED at head with a real review ("no actionable comments"), not a rate-limit pass.
Should-fix: none.
Nit: none.
No findings. What I checked:
- Forging the opt-out (
otel%off%) from inside a box gains nothing new.- Before this PR, a box could already send
traceparent%00-<trace>-<span>-00%….TraceContext::parse_traceparent(crates/minimald-rpc/src/trace.rs:161) accepts flags00.adopt_tracethen aligns the span under an unsampled parent, and the parent-based sampler drops it. That is the same mechanismmlog::otel::opt_outuses (crates/mlog/src/otel.rs:1349-1388). - The only new effect of a forged
otel%off%is that the trace-levelsession channel handling: <line>log line is skipped. - The file log still keeps the
session.channelspan withbox.id/session.id(set by the host, not by the box),method, andtelemetry=opt-out. - Log events inside the request are still exported (
opt_outdocs: "Events logged inside are still log records"). - The spans this suppresses are only the box's own request and its descendants.
- Before this PR, a box could already send
- Stripping the opt-out gains nothing either. A box that drops the prefix sends the bare line every box sends on main today, and gets a recorded root. #3165 also stops handing a task that root's
TRACEPARENT:task_traceparent(sent, …)returns it only when the helper sent a context. The opt-out is the user's own choice, enforced by the user's own process in the box. A box that ignores it is no worse thanMINIMAL_OTEL= min …. - Prefix parsing in
channel_request_span:otel%offwith nothing after it is passed on asoff.otel%on%…is stripped and ignored, asopted_outignores values other thanoff.otel%off%traceparent%…%run%…leavestraceparentas the method. The dispatcher sends that toother, andrun_data_bytesreturnsNonefor it, so the two agree.
run_task_trace(OptedOut): a Line-form request with no env gets exactlyMINIMAL_OTEL=off, whether telemetry is on or off, which matchestrace_env.- Helper (
env_min_helper.sh): withMINIMAL_OTEL=offit never sendsTRACEPARENT. Otherwise the 55-character regex gate is unchanged.
Verdict: ready to approve once #3148 merges and this branch is re-cut (or merge this one and close #3148).
agent-137
left a comment
There was a problem hiding this comment.
Approving at 4da824fa7. The earlier comment review found nothing that needs fixing, and its nits are optional.
Main took #3148's shape of the channel request's task trace (an `Option<String>` for the request span's context); this branch's `TaskTrace` carries that and the opt-out, so every conflicting site keeps the branch's shape and main's call sites take it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Merged main into the branch (20837c4), resolved by hand: main took #3148's shape of the channel request's task trace (an |
norrietaylor
left a comment
There was a problem hiding this comment.
Scope matches the title. The helper sends an otel%off% prefix. channel_request_span makes the span non-recording through crate::exec::adopt_opt_out. run_task_trace opts the task out. run_data_bytes skips the new prefix. The spec notes for TEL-025 and TEL-045 changed to match. Clobber none: each removed line in the diff has a replacement with the same intent, and the head merge from main adds no hunk outside the three files. Correctness The helper compares the value exactly to off, the same comparison as minimald_rpc::trace::opted_out. No other code builds or parses the traceparent% prefix: the helper and env.rs are the only places. The listener (BridgeChannel::forward) and the sandbox crate do not log the line. In run_task, the info events log the task name and not the argv. OTEL_PREFIX does not collide with a CHANNEL_METHODS entry. Two small gaps remain. The TEL-045 verify: line does not run the new test. No test checks that the session channel handling trace line is skipped for an opted-out request. Not checked I did not compile or run tests. I did not confirm that the in-memory exporter test passes under set_default. The PR states a green prepush at the code commit.
| crates/minimal-client/src/lib.rs::do_not_track_sends_the_opt_out_and_no_traceparent; | ||
| crates/minimald/src/env.rs::an_opted_out_helper_request_records_no_span_and_opts_its_task_out | ||
| (the in-box helper's `otel%off%` request) | ||
| lab: 785, 712e |
There was a problem hiding this comment.
low · TEL-045 now lists an_opted_out_helper_request_records_no_span_and_opts_its_task_out under test:, but the verify: command does not run it. The nextest filter an_opted_out_request_leaves_no_daemon_span is not a substring of the new test name. A reader who runs the verify: line does not check the in-box path. Add the new test name to the verify: filter, or use a filter that matches both tests.
There was a problem hiding this comment.
Taken in #3286, since this merged first: the verify line's filter is now an_opted_out, the stem all three unit tests share, so running it checks the in-box path too, and the test list names the new one below.
| // (TEL-045), the file log's included. | ||
| if trace != TaskTrace::OptedOut { | ||
| tracing::trace!("session channel handling: {}", loggable_request(&line)); | ||
| } |
There was a problem hiding this comment.
low · The PR fixes the trace-level session channel handling: <line> record for an opted-out request. TEL-045 requires "no command line". No test drives BoxChannel::run with an otel%off% line and checks that this record does not appear. The new unit test calls channel_request_span directly, so it bypasses this guard. A later change to the guard or to the log can bring the argv record back without a failure. rg 'session channel handling' finds only the log site.
There was a problem hiding this comment.
Taken in #3286: an_opted_out_helper_request_keeps_its_line_out_of_the_log drives BoxChannel::run through the request sender with a plain public-exposure request and an otel%off% one, under a thread-local TRACE subscriber, and asserts the plain request's session channel handling record carries its line, the opted-out request's line is in no record, and every record under it carries the opt-out mark. A change to the guard at the record, or to the record itself, fails it.
…erify runs it (gominimal#3286) * test(minimald): drive an opted-out request through the channel, and verify runs it Two review points on gominimal#3165 that merged open. The in-box path had no test that drives `BoxChannel::run` with an `otel%off%` line and checks the channel's own trace record, so a change to the guard at the record could bring the command line back without a failure. The new test sends a plain public-exposure request and an opted-out one through the request sender under a thread-local TRACE subscriber: the plain request's `session channel handling` record carries its line, the opted-out request's line is in no record, and every record under it carries the opt-out mark the export layer honours. TEL-045's verify line now filters on the stem all three unit tests share, so running it checks the in-box path too, and its test list names the new one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(specs): name TEL-045's three minimald tests on its verify line The stem `an_opted_out` matched five minimald tests, two of them for other requirements, so a rename or removal of a TEL-045 test left the line green. The line names the three tests, as the other lines do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Refs gominimal/inbox#515 (TEL-045 on a task box).
This PR follows #3148 and includes its commit.
c6e42a62bis #3148 (the 712b fix: a task box's task gets the request'sTRACEPARENT). This PR's own change isb3ea90ae2,6f1d6ce18,71090eb79and4da824fa7. Once #3148 merges, this branch is re-cut onto main without it.The bug
TEL-045: a request whose CLI opted out must leave no recorded daemon span, and nothing under it may start a trace. The exec path does this: the CLI sends
MINIMAL_OTEL=offon the channel,exec_spanmakes the exec span non-recording (adopt_opt_out), andtaskenv::trace_envhands the boxMINIMAL_OTEL=offand noTRACEPARENT(TEL-023).Inside the box, the in-box
min(env_min_helper.sh,__min_rpc) forwarded only aTRACEPARENT, as thetraceparent%prefix (TEL-025). An opted-out box has none, so the helper sent a bare request, andchannel_request_spanminted a fresh rootsession.channelspan and recorded it whenever the daemon exports. That is one recorded span per in-box request (run, materialize, net-expose, ...) from an opted-out caller, plus the trace-level log of the raw line, task argv included. It predates #3148 and covers every channel method, not onlyrun. With #3148 a task could also be handed that fresh root'sTRACEPARENTwhen the request carried no marker (an older helper's Line form, or a helper whose env lacked it).The change
env_min_helper.sh: withMINIMAL_OTEL=offin the box,__min_rpcsendsotel%off%<method>%<data>and never itsTRACEPARENT(opt-out wins, as intrace_env). Otherwise nothing changes.crates/minimald/src/env.rs:channel_request_spanreturns aTaskTrace(OptedOut|Parent(Option<String>)). Onotel%off%the span declarestelemetryand goes throughcrate::exec::adopt_opt_out: it is non-recording, parents nothing and is spooled nowhere, and neither is anything under it. Any otherotel%<v>%value is stripped and ignored, asopted_outignores values other thanoff.task_traceparent(sent, recorded): a task gets the request span'sTRACEPARENTonly when the helper sent a context. A bare request hands its task nothing, as before fix(minimald): give a task box's task the request's TRACEPARENT #3148.run_task_tracetakes theTaskTrace:OptedOuthands the taskMINIMAL_OTEL=offand noTRACEPARENT, whatever the env (so a Line form gets it too);Parentkeeps the env-marker fallback.BoxChannel::runskips the trace-levelsession channel handling: <line>log for an opted-out request.run_data_bytesskips anotel%<v>%prefix like the traceparent one, so an opted-outrunwith non-UTF-8 bytes keeps its invalid-byte marking.docs/specs/25-spec-telemetry/25-spec-telemetry.md: the TEL-025 note says a box handed the marker sends theotel%off%prefix and noTRACEPARENT, so the daemon's span for the request records nothing and a task the request runs gets the marker; TEL-025's test note names the opted-out cases; TEL-045 lists the new unit test and lab scenario 712e.Tests
an_opted_out_helper_request_records_no_span_and_opts_its_task_out: with an in-memory OTel exporter, theotel%off%run%...span is markedtelemetry=opt-out, has no parent and exports nothing (nor does a child span), while the same bare request is exported;otel%on%is ignored;run_task_trace(OptedOut)over a Line-form request with an empty env gives exactlyMINIMAL_OTEL=off.a_bare_request_hands_its_task_no_traceparent: thetask_traceparenttruth table, and a Line-form request yieldsParent(None)and adds no variable.a_min_in_a_box_joins_the_outer_trace_through_the_helper_prefix(the shipped helper under bash):MINIMAL_OTEL=offsendsotel%off%net-expose%8770with or without aTRACEPARENT;MINIMAL_OTEL=onkeeps the traceparent prefix.TaskTrace;run_data_bytesgains two asserts.Not covered
TRACEPARENT, and itsrunis opted out byrun_task_trace's env fallback. Reading the env group before the span exists would close that; not done here.TRACEPARENT(minimal.tomlenv_vars) survives the opt-out, as on the exec path.in-box task run starts task=t) are still log records, as on the exec path; argv is never logged there.MINIMAL_OTEL=off, not aDO_NOT_TRACKset inside the box.Verification
min task runand aDO_NOT_TRACK=1 min task run; the opted-out run must add nosession.channelortask_runspan and its task must print noTRACEPARENTandMINIMAL_OTEL=off), 712b (fix(minimald): give a task box's task the request's TRACEPARENT #3148's: the control task gets theTRACEPARENT; FAIL on the release, PASS here), 717 (aminin a session joins the caller's trace) and 000-baseline:OK: tel045-opt-out: 4 expectation(s) on ubuntu-24.04, debian-12: 712b-otel-switch-precedence:fail->pass 712e-otel-opt-out-task:fail->pass 717-otel-nested-min-trace:pass 000-baseline:pass
min-regress batch
fixtel045: baserel-6a01d5aevs the candidatecand-tel045-opt-out-29af2d46, per distro:Lab evidence (min-regress batch
fixtel045on anvil, debian-12, ubuntu-24.04)Head:
4da824fa7ce06a8c6c968799639c613aa45abdf6at 4da824f (lab namecand-tel045-opt-out-29af2d46). Base:rel-6a01d5ae(rel-6a01d5ae).minrun inside a session joins the caller's trace (Option A)minrun inside a session joins the caller's trace (Option A)Fixed since base: 4. Still reproducing on both: 0. Regressed: 0. Passing on both: 4. A verdict links to the run's trace in Cloud Trace (project access needed); the scenario link is its source with the
# issue:it reproduces.Why each failed run failed: what the scenario measured, then what its trace recorded (error spans and warnings, most frequent first):
control_tp=TP-NONE control_otel=OTEL-UNSET optout_tp=TP-NONE optout_otel=OTEL-UNSET channel_spans=0:1:2 task_run_spans=0:1:1; trace:1x minimald WARN Copying instead of hardlinking: cache and destination are on different filesystems; further cross-device copies this run are silent (first:control_files=2 control_spans=55 control_threads=2 control_conns=2 control_tp=TP-NONE sdk_true_files=0 sdk_true_threads=0 sdk_true_conns=0 sdk_true_spans=0 sdk_true_logs=0 sdk_true_tp=TP-NONE sdk1_spans=55 logs_only_logs=74 logs_only_spans=0 logs_only_tp=TP-NONE; trace:2x minimald WARN Copying instead of hardlinking: cache and destination are on different filesystems; further cross-device copies this run are silent (first:control_tp=TP-NONE control_otel=OTEL-UNSET optout_tp=TP-NONE optout_otel=OTEL-UNSET channel_spans=0:1:2 task_run_spans=0:1:1; trace:1x minimald WARN Copying instead of hardlinking: cache and destination are on different filesystems; further cross-device copies this run are silent (first:control_files=2 control_spans=55 control_threads=2 control_conns=2 control_tp=TP-NONE sdk_true_files=0 sdk_true_threads=0 sdk_true_conns=0 sdk_true_spans=0 sdk_true_logs=0 sdk_true_tp=TP-NONE sdk1_spans=55 logs_only_logs=74 logs_only_spans=0 logs_only_tp=TP-NONE; trace:2x minimald WARN Copying instead of hardlinking: cache and destination are on different filesystems; further cross-device copies this run are silent (first:Each cited run's verdict and evidence
Each cited run's last VERDICT or ENV line and its EVIDENCE line (the results.jsonl rows of batch
fixtel045):VERDICT: control: the task got no TRACEPARENT although the daemon exported spans (TEL-023: files=2 spans=55 logs=74 threads=2 conns=2 tp=TP-NONE ran=1);EVIDENCE: control_files=2 control_spans=55 control_threads=2 control_conns=2 control_tp=TP-NONE sdk_true_files=0 sdk_true_threads=0 sdk_true_conns=0 sdk_true_spans=0 sdk_true_logs=0 sdk_true_tp=TP-NONE sdk1_spans=55 logs_only_logs=74 logs_only_spans=0 logs_only_tp=TP-NONEVERDICT: OTEL_SDK_DISABLED=true is inert (spool too), =1 is not, logs-only spools logs and adopts no trace;EVIDENCE: control_files=2 control_spans=55 control_threads=2 control_conns=2 control_tp=00-113779714071e7093842253f7fa24126-dec5b9b256d8e1e3-01 sdk_true_files=0 sdk_true_threads=0 sdk_true_conns=0 sdk_true_spans=0 sdk_true_logs=0 sdk_true_tp=TP-NONE sdk1_spans=55 logs_only_logs=74 logs_only_spans=0 logs_only_tp=TP-NONEVERDICT: control: the task got no TRACEPARENT although the daemon exported spans (TEL-023: files=2 spans=55 logs=74 threads=2 conns=2 tp=TP-NONE ran=1);EVIDENCE: control_files=2 control_spans=55 control_threads=2 control_conns=2 control_tp=TP-NONE sdk_true_files=0 sdk_true_threads=0 sdk_true_conns=0 sdk_true_spans=0 sdk_true_logs=0 sdk_true_tp=TP-NONE sdk1_spans=55 logs_only_logs=74 logs_only_spans=0 logs_only_tp=TP-NONEVERDICT: OTEL_SDK_DISABLED=true is inert (spool too), =1 is not, logs-only spools logs and adopts no trace;EVIDENCE: control_files=2 control_spans=55 control_threads=2 control_conns=2 control_tp=00-8d0c1a8879a4ff7a8697aeb3054866c8-3ce4267b9c16043e-01 sdk_true_files=0 sdk_true_threads=0 sdk_true_conns=0 sdk_true_spans=0 sdk_true_logs=0 sdk_true_tp=TP-NONE sdk1_spans=55 logs_only_logs=74 logs_only_spans=0 logs_only_tp=TP-NONEVERDICT: an opted-out task run is not opted out: opt-out:task-has-no-MINIMAL_OTEL=off(OTEL-UNSET) opt-out:daemon-recorded-1-session.channel-span(s);EVIDENCE: control_tp=TP-NONE control_otel=OTEL-UNSET optout_tp=TP-NONE optout_otel=OTEL-UNSET channel_spans=0:1:2 task_run_spans=0:1:1VERDICT: an opted-out task run leaves no daemon span and its task gets MINIMAL_OTEL=off, no TRACEPARENT;EVIDENCE: control_tp=00-97fcee2e1ec9b8606df55a3f20c9380d-f536f02e664ee061-01 control_otel=OTEL-UNSET optout_tp=TP-NONE optout_otel=off channel_spans=0:1:1 task_run_spans=0:1:1VERDICT: an opted-out task run is not opted out: opt-out:task-has-no-MINIMAL_OTEL=off(OTEL-UNSET) opt-out:daemon-recorded-1-session.channel-span(s);EVIDENCE: control_tp=TP-NONE control_otel=OTEL-UNSET optout_tp=TP-NONE optout_otel=OTEL-UNSET channel_spans=0:1:2 task_run_spans=0:1:1VERDICT: an opted-out task run leaves no daemon span and its task gets MINIMAL_OTEL=off, no TRACEPARENT;EVIDENCE: control_tp=00-54b394b669fd0f2262fb47e8a428d07f-7f7eb93e563d73d7-01 control_otel=OTEL-UNSET optout_tp=TP-NONE optout_otel=off channel_spans=0:1:1 task_run_spans=0:1:1VERDICT: nested min gets the caller's trace on opt-in, nothing when off;EVIDENCE: outer_trace=86299ac4d2e279e95c3e07ea8f6704f3 on_tp=00-86299ac4d2e279e95c3e07ea8f6704f3-724be1092a7d19da-01 on_leaked_headers=0 off_tp_injected=0VERDICT: nested min gets the caller's trace on opt-in, nothing when off;EVIDENCE: outer_trace=596d35c8b68aa6ba5aa68633eda3fe51 on_tp=00-596d35c8b68aa6ba5aa68633eda3fe51-c27f7a300720c506-01 on_leaked_headers=0 off_tp_injected=0VERDICT: nested min gets the caller's trace on opt-in, nothing when off;EVIDENCE: outer_trace=ac7b0e9baeea710f00d2b888d2869752 on_tp=00-ac7b0e9baeea710f00d2b888d2869752-362af1a4d874a020-01 on_leaked_headers=0 off_tp_injected=0VERDICT: nested min gets the caller's trace on opt-in, nothing when off;EVIDENCE: outer_trace=d77fa91b2f609146a6d38814acc40492 on_tp=00-d77fa91b2f609146a6d38814acc40492-b770689352e4b5db-01 on_leaked_headers=0 off_tp_injected=0VERDICT: min runs; gcc (MPFR folding) and a cached binary work;EVIDENCE: task_rc=0 tries=1 hello_c=1 zstd=1VERDICT: min runs; gcc (MPFR folding) and a cached binary work;EVIDENCE: task_rc=0 tries=1 hello_c=1 zstd=1VERDICT: min runs; gcc (MPFR folding) and a cached binary work;EVIDENCE: task_rc=0 tries=1 hello_c=1 zstd=1VERDICT: min runs; gcc (MPFR folding) and a cached binary work;EVIDENCE: task_rc=0 tries=1 hello_c=1 zstd=1🤖 Generated with Claude Code
Summary by CodeRabbit
MINIMAL_OTEL=offnow opt out of trace recording and do not forward trace context. Tasks started by those requests also honor the opt-out.