[apucacao]

Summary

A judge runs after the graded handler has ended its span, so each judge starts a trace of its own with its own launchdarkly.run.id. Nothing on that trace points to the run it graded. Today the only link is the judge's metric event, which carries the graded run id in its track data.

This puts the same link on the trace. Every SDK span started during a judge run now carries the graded run's identity, so a judge trace can be joined to the run it scored exactly instead of by timestamp.

This matches a JS turnkey SDK change in launchdarkly/js-ai-sdk (launchdarkly/js-ai-sdk#200). The cross-SDK spec rule is in launchdarkly/ai-sdks-monorepo#122.

What changed

• run_judges (inline) and run_judge (deferred) in launchdarkly_ai_server/judges.py wrap each judge run in stamp_span_attributes. That is the existing write-if-absent mechanism offline evaluations use to put launchdarkly.evaluation_run.id on every span of a dataset row, applied by ConversationIdSpanProcessor to spans from this SDK's tracers only.
• Three new span attributes, with constants in judges.py:
  • launchdarkly.judge.target.run.id (LD_JUDGE_TARGET_RUN_ID)
  • launchdarkly.judge.target.config.key (LD_JUDGE_TARGET_CONFIG_KEY)
  • launchdarkly.judge.target.variation.key (LD_JUDGE_TARGET_VARIATION_KEY)
• Inline judges read base_track_data. Deferred judges read JudgeTask.parent_track_data. Each key is written only when its value is a non-empty string, and never overwrites a value already on the span.
• They are identifiers, not content, so capture settings do not gate them.
• Spans outside a judge run are untouched.
• TELEMETRY-CONTRACT.md section 4a documents the keys.

The constants are not added to the package's __all__, same as JS, so the public API report is unchanged. The cross-handler vocabulary lock in tests/test_cross_handler_parity.py scans for keys passed to set_attribute and gen_ai.* literals; it does not list launchdarkly.evaluation_run.id either, so these keys are outside its scope and it is unchanged.

Testing

packages/client/tests/test_judge_target_run.py uses a real TracerProvider with ConversationIdSpanProcessor and an in-memory exporter. A stand-in judge handler opens invoke_agent and a chat span under it on an SDK tracer. It covers each test the spec rule requires:

• Inline path: both spans carry the graded run id, config key and variation key.
• Deferred path: the same three values, read from the task's parent_track_data.
• Outside a judge run: an SDK span carries none of the three.
• Other tracers: a span from a non-SDK tracer inside the judge run carries none of the three.
• Capture off: with the judge handler's capture_content=False, both spans still carry the three values.
• Empty values: with variationKey: "", that key is absent and the other two are present.
• Write-if-absent: a span started with launchdarkly.judge.target.run.id set to "own" keeps "own".
• Graphs: in graph().invoke() with a per-node judge and a graph_judge, the per-node judge's spans carry the node's run id (equal to that judge's metric event), the node's config key and its variation key. The graph judge's spans carry the graph run id (equal to the launchdarkly.graph span's launchdarkly.run.id and to its metric event), the graph key and the graph variation key. The graded node's own spans carry none of the three.

Checked that the tests catch regressions:

• With the stamping disabled, the inline, deferred, capture-off, empty-value, write-if-absent and graph tests fail. The outside-a-run and other-tracer tests pass, as they should.
• With write-if-absent removed from the span processor, the write-if-absent test fails.
• With the SDK tracer gate removed, the other-tracer test fails.

Results:

• uv run pytest: 3775 passed, 18 skipped.
• ruff check ., ruff format --check ., mypy packages/*/src and the API report check are clean.

Spec: launchdarkly/ai-sdks-monorepo#122

Original branch: ag/judge-target-run

---

Note

Overview
Judge runs start their own trace after the graded handler finishes, so traces were only linkable via metric events. This PR stamps every SDK span in a judge run with the graded run’s runId, configKey, and variationKey using the existing write-if-absent stamp_span_attributes mechanism (same pattern as offline evaluation rows).

Inline run_judges wraps each judge in _judge_target(base_track_data); deferred run_judge uses _judge_target(task.parent_track_data). Three new attributes (launchdarkly.judge.target.run.id, .config.key, .variation.key) are set only for non-empty strings and are not gated by content capture.

TELEMETRY-CONTRACT.md section 4a documents the keys. test_judge_target_run.py asserts inline and deferred paths stamp invoke_agent and chat spans, and that spans outside a judge run stay untouched.

^{Reviewed by Cursor Bugbot for commit bb98172. Bugbot is set up for automated code reviews on this repo. Configure here.}