Skip to content

Send telemetry with the CLI's own token only - #104

Open
zmofei wants to merge 9 commits into
webpki-advisoriesfrom
telemetry-cli-token-first
Open

zmofei wants to merge 9 commits into
webpki-advisoriesfrom
telemetry-cli-token-first

Conversation

@zmofei

@zmofei zmofei commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Telemetry events are now sent only with the CLI's own public token: MAPBOX_CLI_TOKEN when set, otherwise the token fetched from https://cli.mapbox.com/config/token.json. A user's token (--token, MAPBOX_ACCESS_TOKEN, a login) is never used for telemetry.

  • With no usable CLI token the event is dropped. The delivery log's no_token line says why (quiet, override_not_public).
  • One token, one attempt: a refused event is never resent. On a 401/403 the token is checked with the Tokens API and the endpoint is asked once right away; a different token served there, or a replacement for a revoked one, is used by the next run.
  • Backoff is per caller, replacing the shared fetch pause and the per-caller rest. A caller counts its failures in a row: a fetch with no usable token, or a refusal the endpoint doesn't answer with a different token. Every third quiets that caller for an hour, in which it sends and fetches nothing. An accepted request, or a different token on disk, whichever caller saved it, starts every caller's count over. One caller's quiet period never holds another, and waits an earlier build wrote are ignored. While nothing changes, that is at most 3 endpoint fetches per caller per hour, plus one per new token.
  • Each caller's use of the CLI token is one policy (paced, resends, user_fallback): telemetry is paced, never resends and never falls back to the user's token; report is not paced, resends once with a fresh token, and may fall back. Adding a caller is one entry.
  • MAPBOX_CLI_TOKEN must be a pk. token. It is sent as is, never checked or replaced, even while telemetry is quiet.
  • The detached sender no longer receives a user token: its stdin carries only the event.
  • mapbox report is unchanged: it still prefers the CLI's token, falls back to the user's, and resends with a fresh token after a refusal.

README, CHANGELOG and docs/architecture.md are updated.

Testing: fmt, clippy and the full suite (916 tests) pass locally on macOS, and CI passes on all three runners at bbe0165. New tests cover a user's token never appearing in a telemetry request on the override and fetched-token paths; nothing sent when the CLI token is unusable; no resend after a 401/403; a different token after a refusal used by the next run; three failures quieting telemetry for an hour and nothing sent or fetched meanwhile; another caller's state and an earlier build's waits not holding telemetry; a non-pk. override dropped; and an old-format state file. Each new test was seen failing under a mutation of its behavior. Against production, with this branch's build: 10 commands sent 10 events, all accepted (204), while the user's own login was invalid; a run with no login and an empty config directory fetched the CLI token from the real endpoint and its event was accepted too. Not verified: the Tokens API on a real refusal, since none occurred. report's resend and fallback after a refusal have no end-to-end test, since nothing points a command's API request at loopback; the outcome mapping is unit-tested.

Known follow-ups, not in this PR: right after a rotation, a revoked token can cost two Tokens API calls in the background sender; Policy's three flags allow combinations no caller uses; refused reads the state file several times per refusal.

Downstream: events from users who have their own token now carry the CLI's token, so the token-derived owner on those rows becomes the CLI token's account. The user's account is still in the event's auth.account. Operational: none, no deploy.

@zmofei zmofei added the locationai-team-skills Opened via the location-ai PR creation skill label Oct 20, 2026
@zmofei
zmofei marked this pull request as ready for review October 20, 2026 15:39
@zmofei
zmofei requested a review from a team as a code owner October 20, 2026 15:39
@zmofei zmofei self-assigned this Oct 20, 2026

@mattpodwysocki mattpodwysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me @zmofei. I traced every telemetry path and couldn't find any way for the user's token to reach a send: --token, MAPBOX_ACCESS_TOKEN, the config file and the stored login are never read on that path, the child's env has them stripped, and opt-out is checked before any network work. The deleted fallback chain didn't take anything still needed with it. Build, tests, clippy and fmt are clean.

Just nits, none blocking:

  • README around line 494: "never yours" isn't quite true if someone puts their own pk. token in MAPBOX_CLI_TOKEN, which the README documents and a release build honors. A few words saying it's a developer override for a different CLI token would cover it. That paragraph could use a reflow too (long line 495, "The token is" left on its own at 505).
  • SEND_TIMEOUT in src/telemetry_sink.rs is described as the child's time budget, but it only bounds the POST. With the fetch, the Tokens API check and a refetch, the child can run about 25s. Harmless since it's detached, the comment just misleads.
  • The never-the-user's-token tests use sk. and tk. values. Adding a pk. user token to a_logged_in_run_sends_with_the_clis_token would catch a regression that brought back a plausible-user-token fallback, which the pk. filter would otherwise let through.

Comment thread src/main.rs Outdated
);
match attempt {
CliAttempt::Done(result) => return result,
CliAttempt::Unusable { error, why, .. } if !caller.user_fallback() => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This arm can't be reached today, since report is the only command caller and it has user_fallback: true. On a dry run it would also execute with no token at all. Could we drop it until a caller needs it, or add a test that covers it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped it in 470bca1. No command uses a policy without user_fallback, so instead a test (every_cli_token_command_has_the_command_policy) now fails if one is added.

@zmofei
zmofei force-pushed the telemetry-cli-token-first branch from bbe0165 to 71f5234 Compare October 21, 2026 07:52
Mapbox Events keeps the token an event was sent with, and the request logs
record it as access_token, so sending with a user's token stored a login's
tk. token or a secret sk. one there. Telemetry now sends only with
MAPBOX_CLI_TOKEN or the fetched CLI token, and drops the event when neither
can be used. The user-token plumbing to the send child is removed.

An event is sent at most once. A refused CLI token is still checked, and a
dead one replaced at once, so the next run's event uses the new token.

While telemetry rests from a token an API refused, the endpoint is asked
for another on the existing fetch schedule: the rested token served again
counts as a miss, and a different token ends the rest.

report is unchanged: it keeps its resend and its fallback to the user's
token.
Keep a rest's misses on its cooldown so they no longer shorten a dead
token's wait, and keep the rest when the same token is taken back.
Require MAPBOX_CLI_TOKEN to be a pk. token for telemetry, and say why an
event was dropped in the delivery log. Remove the unused retry policy
parameter, and bring the docs and changelog in line with the behavior.
Test that a refused MAPBOX_CLI_TOKEN is never checked or replaced, that
it is sent through a rest or fetch pause, that no user token reaches the
fetched-token send, and how a rest that outlasts the fetch pause behaves.
Pull report's refusal handling into cli_attempt so it can be unit-tested.
Only the token on disk is shared now. Each background caller (today only
telemetry) keeps its own count of failures in a row: a fetch with no usable
token, or a refusal after which the endpoint, asked at once, serves no
different token. The third failure quiets the caller for an hour, in which it
sends and fetches nothing. An accepted request or a different token starts
the count over.

This replaces the global fetch pause and the per-caller rest, with its misses
and rotation-during-rest rules. Waits an earlier build wrote to
cli-token.json are not read. `mapbox report` and MAPBOX_CLI_TOKEN are never
held by telemetry's count, and report never writes it.
…y once

A new token on disk now ends every caller's failure count and quiet period,
whichever caller saved it. Each count records the token it was made with, so
a count merged back by a run that still had the old token holds nobody back.

Telemetry and commands now differ only by a Policy (paced, resends,
user_fallback) named next to each caller, instead of separate entry points
and ledger variants.
@zmofei
zmofei force-pushed the telemetry-cli-token-first branch from 71f5234 to e1786c3 Compare October 21, 2026 07:52
…a pk. user token

- Remove the command path for a caller without user_fallback. No command
  uses one, so a test now holds every CLI-token command to the COMMAND
  policy instead.
- README: say MAPBOX_CLI_TOKEN is a developer override for another CLI
  token, and reflow the paragraph.
- SEND_TIMEOUT bounds only the POST, not the child.
- Use a pk. user token in a_logged_in_run_sends_with_the_clis_token, so a
  fallback gated on is_plausible would be caught.
@zmofei

zmofei commented Oct 21, 2026

Copy link
Copy Markdown
Member Author

@mattpodwysocki Thanks! All four addressed in 470bca1:

  • README: MAPBOX_CLI_TOKEN is now described as a developer override for another CLI token, and the paragraph is reflowed.
  • SEND_TIMEOUT: the comment now says it bounds only the POST.
  • a_logged_in_run_sends_with_the_clis_token uses a pk. user token. I checked it with a mutation that falls back to a pk. user token: the old sk. value passes, the new one fails.
  • The unreachable arm is removed (replied inline).

fmt, clippy and the full suite pass locally. Could you take another look?

@zmofei
zmofei requested a review from mattpodwysocki October 21, 2026 08:13
@zmofei
zmofei changed the base branch from main to webpki-advisories October 21, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

locationai-team-skills Opened via the location-ai PR creation skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants