Skip to content

Add mapbox workflow: install and run multi-step workflows (beta) - #67

Open
zmofei wants to merge 34 commits into
mainfrom
feat/workflows
Open

zmofei wants to merge 34 commits into
mainfrom
feat/workflows

Conversation

@zmofei

@zmofei zmofei commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Adds mapbox workflow: multi-step recipes of mapbox commands and scripts, defined in a workflow.yaml. Beta and not recommended for use yet; every subcommand says so.

mapbox workflow list | show <name> | uninstall <name>
mapbox workflow install <name | ./path> [--repo OWNER/REPO] [--ref REF] [--force] [--dry-run]
mapbox workflow run <name> --<input> <value> ... [--dry-run]
  • Nothing ships in the binary. install copies a workflow from a local directory or a GitHub repo's workflow/<name>/ (default mapbox/mapbox-cli@main), after checking it against this build's commands. It asks first when the repo isn't ours, since steps run as you with your credentials.
  • A workflow has its own optional version, and requires for the oldest CLI it needs. Per-workflow tags and installing the latest by default come in a follow-up.
  • Command steps run this binary again, so auth, timeouts and history work as usual. --dry-run runs only steps marked safe to run.
  • The first workflow is copy-style, which copies a style with its fonts and icons between two accounts.

Format and rules: workflow/README.md. Commands: docs/commands.md#workflows.

Verified

  • Install from GitHub against the real API, with the commit recorded.
  • copy-style end to end between a staff account and a test account (on 9f8a3d5); the copy matched the source and has been deleted.
  • tests/workflow.rs runs real processes for runs, failures, dry runs, interrupts and installs. I checked the key tests fail without their fixes.

Not checked

  • copy-style on the latest commits; will rerun after merge.
  • The install prompt against a real third-party repo, and Windows.
  • A style with custom fonts.

@zmofei zmofei self-assigned this Sep 29, 2026
@zmofei zmofei changed the title Add mapbox workflow: install and run multi-step workflows (beta) [WIP] Add mapbox workflow: install and run multi-step workflows (beta) Sep 29, 2026
@zmofei
zmofei changed the base branch from feat/terminal-styling to feat/styles-download September 30, 2026 10:17
zmofei added 26 commits October 20, 2026 12:19
A workflow is a workflow.yaml of command and script steps, installed from a
local directory or a GitHub repository's workflow/<stage>/<name>/ into
~/.mapbox/workflows/, then run by name. Command steps run this binary again
with --output json, so they resolve credentials, timeouts and path encoding
exactly as a typed command does.

The first published workflow is workflow/beta/copy-style, which copies a
style between accounts using two credential profiles.
Every workflow subcommand now says on stderr that the feature is beta, in
development and not recommended for use, as do --help, the docs and the
changelog.

Workflows live at a fixed workflow/beta/<name>/ rather than under a
variable stage directory, so the stage list, the stage recorded at install
and the Stage shown by list and show are gone.
The beta/ level no longer means anything now that the stage concept is
gone, so copy-style moves up to workflow/copy-style and install looks for
workflow/<name>/ in a repository.
mapbox/cli redirects to a different repository on GitHub, so the default
source never held this repository's workflows.
The notice is one short line with a bold label, paths under the home
directory read as ~/..., list and show lay their fields out in aligned
columns, and the next command to run is a tip on stderr rather than part
of the result. Step progress is colored at a terminal. JSON is unchanged.
install takes a path, so uninstall ./workflow/copy-style is the natural
undo; it now names the workflow by that directory's name. Only an
installed workflow under ~/.mapbox/workflows/ is ever removed, as before.
The already-installed error now gives the install line to retry with
--force.
show lays every workflow out the same way: name and summary, the
description, inputs, steps, then where the installed copy came from. The
description is read as a small Markdown subset: paragraphs are reflowed,
indented lines are highlighted commands, '- ' lines are a list, and code
spans are highlighted. copy-style's description uses it.
mapbox workflow run copy-style --style-id <id> --from-profile source
--to-profile target, in place of repeated --input KEY=VALUE. Before the
parse, the named installed workflow is attached under `run` as a real
subcommand whose inputs are typed, required flags, so --help, a missing
input and a misspelled flag behave as on every other command. Every other
command line sees the tree unchanged.

An input may not take a global option's name. The run record keeps the path
as `workflow run`, without the workflow's name, and the generated agent
skill leaves `workflow` out, since it is not recommended for use.
The test matched the install line in stderr's raw text, but the error is
JSON there and JSON escapes a Windows path's backslashes, so it failed on
windows-2022 only. It now parses the error and compares next_actions.
Every top-level command needs an example line in the README, which the
docs contract now checks.
A step marked dry_run runs during a dry run with MAPBOX_WORKFLOW_DRY_RUN=1
and the promise to write nothing, so the plan can show real data. Only a
script can be marked, and a marked step reads only marked steps.
Download the style's ZIP as the source account, then upload new fonts,
create the style, upload its icons in batches of 25 and point its sprite
and glyphs at the target account. Both steps support --dry-run. A partial
failure lists what was created and the commands that remove it.
--use-login keeps a MAPBOX_ACCESS_TOKEN from outranking the profile, but a
MAPBOX_USERNAME still named the account, so a request could carry one
account's token to another's path and be refused. Pass --username with
the login's own account. Also check the target before downloading, and
show a failed command's message and fix rather than its JSON.
The Styles API refuses a new style whose sprite the target account cannot
read, which a private sprite of another account is. Create the copy with
no sprite and its glyphs already on the target, then set its own sprite
after the icons are uploaded.
At a terminal a step is its title, the details its script reports on
stderr, and a status line: a spinner beside the latest ::progress line
while it runs, then a check or a cross and the time it took. Off a
terminal it stays one [n/total] line, details follow, and progress lines
are dropped. Any workflow's script steps get this; copy-style now reports
each font, the created style and the icon batches.

A workflow's result also stops claiming its JSON is what an API sent.
A script's ::warn line is shown under its step and listed again after the
last one. --quiet keeps each step's title, outcome and warnings and drops
its details. A workflow's optional result template is what text mode
prints instead of the outputs as fields; -o json still prints the
outputs. copy-style uses all three, and every workflow can.
A command step can now keep its stdout as a file with save, in a working
directory removed when the run ends, which is how a style's ZIP reaches a
later step; and it can be marked dry_run when its command changes nothing.
copy-style checks both logins, downloads the style and lists the target's
fonts as command steps, and keeps a script only for the copy's loops.
It is public, so installing from it needs no token. Show the token with
a private --repo instead.
On Windows the working directory's path is full of backslashes, which the
script wrote into its own JSON unescaped, so its output parsed as text.
Take the path through outputs instead, which the runner encodes.
Use the theme's accent, muted and error colors in place of the ANSI
constants main removed, place workflow under CLI in top-level help, and
keep tests/workflow.rs offline for telemetry and the CLI's token as the
source guards now require.
A dry_run command step was allowed whenever its command had no --dry-run
of its own. tilesets-cli has none either, and hands its arguments to a
program that may write. The allowed set is now explicit: every API
operation that sends a GET, plus a short list of local reads.
The directory was created with the default umask, so a saved style was
readable by other users of a shared /tmp, and a Ctrl-C killed the run
before Drop could remove it. It is now 0700 on Unix, and a run catches
Ctrl-C, lets the step's child stop, and fails with `cancelled` so the
directory is removed. Adds ctrlc, since a signal handler takes unsafe.
.install.json kept only the requested ref, so an install of a branch
could not say what it holds once the branch moved. The ref is now
resolved to a commit first, that commit's tarball is downloaded, and the
commit is recorded and reported by install, list and show.
install --force removed the installed copy before renaming the new one
in, so a failed rename left no workflow at all. The old copy is now moved
aside, moved back if the rename fails, and removed only once the new one
is in place. An install interrupted in between restores it next time.
CI runs command steps and sh script steps on Windows, but a .py step
needs python3 on PATH, which the python.org installer does not add, and
copy-style has not been run there. Documents that, and tests that a .py
step runs under python3 by default on Unix.
Only `mapbox workflow --help` said so; `list`, `show`, `install`,
`uninstall`, `run` and an installed workflow's own `run <name>` help did
not, though each prints the notice when it runs.
ctrlc sets the flag from its own thread, so the child stopped by the same
Ctrl-C could be reaped first and the run reported an ordinary step
failure. A failed step now waits up to 250 ms for the flag.
- Put back a copy an interrupted install moved aside wherever a workflow
  is looked for, and have uninstall remove such leftovers, which could
  otherwise come back as a stale install.
- Stop before a step when Ctrl-C arrived between steps, since no child
  received it.
- Handle SIGTERM and SIGHUP as Ctrl-C is handled, through ctrlc's
  termination feature.
- List what copy-style created when it is interrupted mid-copy.
- Correct the docs on what an interrupt leaves behind and where the
  commit is reported.
@zmofei
zmofei changed the base branch from feat/styles-download to main October 20, 2026 10:37
@zmofei zmofei added the locationai-team-skills Opened via the location-ai PR creation skill label Oct 20, 2026
@zmofei zmofei changed the title [WIP] Add mapbox workflow: install and run multi-step workflows (beta) Add mapbox workflow: install and run multi-step workflows (beta) Oct 20, 2026
@zmofei
zmofei marked this pull request as ready for review October 20, 2026 10:56
@zmofei
zmofei requested a review from a team as a code owner October 20, 2026 10:56
@zmofei zmofei mentioned this pull request Oct 20, 2026

@mattpodwysocki mattpodwysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zmofei, this is a big one and the install side is really careful: archive paths, symlinks, name and ref checks, and the GitHub token only ever going to api.github.com all look right to me.

The two things I'd like sorted before it lands are both about telling people before workflow code runs (inline below). A few smaller things too:

  • A step can set yes: true and skip a delete confirmation, see the note on RESERVED.
  • When a command step fails in text mode, the terminal shows the child's raw JSON error, and the workflow_failed error drops the child's code, so a JSON caller can't branch on something like http_401. Pulling the last stderr JSON line in (like copy_style.py's readable() does) would fix both.
  • DEFAULT_REF is main, so a released binary installs whatever is on main that day. Defaulting to v{CARGO_PKG_VERSION} (with main for dev builds) would keep a release and its workflows in step.
  • SIGTERM only sets a flag, so a step that hangs keeps running until SIGKILL. Forwarding it to the child would help with CI timeouts.
  • The new error codes (invalid_name, invalid_repo, invalid_ref, unsafe_archive, workflow_not_found, workflow_not_installed, invalid_input, invalid_arguments) aren't in the codes table in docs/commands.md. invalid_arguments for --repo with a local path also exits 1, where everywhere else a flag conflict is usage with exit 2.
  • The tar ../ and absolute path rejection, skipped symlink entries, the local symlink refusal and the 64 MiB cap have no tests. The logic reads right, but those are the branches I'd most want pinned.
  • Small one: take(MAX_BYTES) truncates quietly, so an oversized download shows up as a confusing "ended unexpectedly". Reading MAX_BYTES + 1 and erroring would be clearer.

Build, tests, clippy and fmt are all clean on my end.

Comment thread src/workflow/mod.rs Outdated
`mapbox workflow run copy-style --style-id <id> --from-profile source \
--to-profile target`.\n\n\
`mapbox workflow run <name> --help` lists that workflow's flags, and \
`--dry-run` prints its plan without running a step.\n\n{BETA_NOTE}",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The help here says --dry-run runs without running a step, but steps marked dry_run: true really run, and for a script step that's arbitrary code under whatever interpreter the yaml names. I tried it with a local workflow whose dry_run script writes a file: workflow run probe --dry-run wrote it, and the script also saw my --token value and GITHUB_TOKEN. docs/commands.md describes this correctly, so it's the help text that's off (here and in the per-workflow flag help in run_command).

Since --dry-run is exactly what a careful person reaches for before trusting a workflow, could we fix the wording, and consider not running script steps in a dry run unless asked? Or at least list them in the plan as "will execute ...".

@zmofei zmofei Oct 21, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d03c387. The help now says marked steps really run, scripts included, as you, and the plan marks script steps as "ran its script in this dry run". I kept running marked scripts by default, since copy-style's plan step is a script and its dry run is useless without it; the trust question moved to install instead.

Comment thread src/workflow/mod.rs
Comment thread src/workflow/runner.rs

/// Global options a step may not set, because the runner owns them or
/// because they would put a credential in a file.
const RESERVED: &[(&str, &str)] = &[

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could yes go in here too? Right now a step like command: styles delete with args: { yes: true } installs fine and deletes without the prompt the person would otherwise get. Their own --yes on workflow run is still inherited, so they can opt in themselves.

@zmofei zmofei Oct 21, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in d03c387. A step setting yes is refused at install; --yes on workflow run is still inherited.

@zmofei

zmofei commented Oct 21, 2026 •

Copy link
Copy Markdown
Member Author

Thanks for the thorough review. All addressed in d03c387. The three inline ones are answered in their threads; the rest:

  • A failed command step now hands back its own error: workflow_failed carries it as cause (so cause.code can be http_401), and text mode shows its message instead of raw JSON.
  • Default ref: I kept main. Workflows will be versioned on their own rather than with the CLI, since they may move to a separate repo. workflow.yaml now has an optional version and requires: ">=X.Y.Z", and install refuses a workflow that needs a newer CLI. That also covers what scripts call through $MAPBOX_CLI, which the command-step check can't see. Per-workflow tags (workflow-<name>-<version>) and defaulting to the latest one come in a follow-up.
  • SIGTERM is passed on to the running step as SIGINT after 2s, so a hung step doesn't outlive a CI timeout.
  • Workflow error codes are in the table; --repo/--ref with a local path is now usage, exit 2.
  • Oversized archives/directories fail as too large instead of truncating. Tests added for ../ and absolute tar paths, skipped links, the local symlink refusal and the size cap.

Not verified: the install confirmation prompt against a real third-party repo (decision logic is unit-tested), cause with a real API 401 (tested with an offline error), and Windows.

@zmofei
zmofei force-pushed the feat/workflows branch 2 times, most recently from e064955 to b7f33ca Compare October 21, 2026 07:52
@zmofei
zmofei changed the base branch from main to webpki-advisories October 21, 2026 08:13
- `--dry-run` help no longer says no step runs: marked steps run, scripts
  included, as the person running it.
- `install` lists a workflow's scripts, and from a repository other than
  mapbox/mapbox-cli asks first; with no terminal it refuses with
  `confirmation_required` unless --yes. `show` marks script steps.
- A step may not set `yes`.
- A failed command step hands its error document back through a file, so
  the run's `workflow_failed` carries it as `cause` and text mode shows its
  message rather than raw JSON.
- A SIGTERM to the run is passed on to the running step as a SIGINT.
- `--repo`/`--ref` with a local directory is `usage`, exit 2; the workflow
  error codes are in the codes table.
- An oversized archive or directory is refused as too large rather than
  truncated, and the archive and symlink checks have tests.
- workflow.yaml names its format with `schema: 1` and may name the
  workflow's own `version` and the oldest CLI it `requires`. A workflow
  needing a newer CLI is refused at install, and show and run say why;
  this is the only check that covers what a script asks of $MAPBOX_CLI.
  Both are optional, and the old `version: 1` spelling still reads as an
  unversioned workflow, so earlier commits still install.

@mattpodwysocki mattpodwysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zmofei, this round looks really good. I went through each point against the binary: the dry-run wording, the script listing and trust note, yes being refused, cause (checked with a real 401, and text mode reads well now), requires, the error codes table, the usage exit 2, the archive tests and the size cap all check out. Keeping main with requires makes sense to me.

Two things before it lands: the --ref hole in the install thread, and a panic with --timeout (inline). A few smaller ones:

  • SIGTERM forwarding works for me on macOS. At a real terminal, though, the child already got the Ctrl-C, so if it takes more than 2s to wind down it gets a second one, which can cut off copy-style's "here's what I created" report. Checking whether we're the terminal's foreground group (tcgetpgrp) before forwarding would avoid that. Also only the step's own pid is signaled, so a shell step running a long foreground command keeps the run alive until it finishes. A line in the docs is fine for beta, and it'd be worth saying this is Unix only.
  • MAPBOX_YES=1 in the environment also answers the trust question. That's documented, but a CI job that sets it for deletes would now trust any --repo silently. Maybe require the --yes flag itself for that one?
  • Tiny: when a step fails with a 401, the fix copied up says the token came from MAPBOX_ACCESS_TOKEN even when I typed --token, since that's how the runner hands it to the child.

Comment thread src/workflow/runner.rs Outdated
for option in ["profile", "username", crate::http::TIMEOUT_ARG] {
let typed =
matches.value_source(option) == Some(clap::parser::ValueSource::CommandLine);
if let (true, Some(value)) = (typed, matches.get_one::<String>(option)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mapbox workflow run copy-style --timeout 30 ... panics here: --timeout is parsed as a Duration, so get_one::<String> hits clap's downcast mismatch. The test builds Inherited by hand, so it doesn't catch it. Reading the raw value (matches.get_raw) or formatting the Duration back would fix it, plus a test that goes through from_matches.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 21b0a0e. It reads the raw value now. The new test parses through the real command tree, and it reproduces the downcast panic when the fix is reverted.

Ask before installing a --ref that is not on main, take only a typed --yes
for the trust question, fix the --timeout panic in workflow run, skip
forwarding a signal at the terminal's foreground, and name --token in a
step's auth advice when that is where the token came from.
@zmofei

zmofei commented Oct 21, 2026

Copy link
Copy Markdown
Member Author

Thanks! All addressed in 21b0a0e:

  • SIGTERM: nothing is forwarded while we're the terminal's foreground group (tcgetpgrp). Checked with a pty: a step trapping SIGINT got 2 before and 1 after. The docs now say it's Unix only and that it only reaches the step's own process.
  • Only a typed --yes answers the trust question. MAPBOX_YES doesn't.
  • With --token, a step's 401/403 advice now names the flag.

Not verified: the on-main path end to end (no workflow on main yet, so unit tests only), a fork-only SHA, and Windows.

@zmofei
zmofei requested a review from mattpodwysocki October 21, 2026 17:12
mattpodwysocki
mattpodwysocki previously approved these changes Oct 21, 2026

@mattpodwysocki mattpodwysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zmofei, this is good to go from me. The on-main check closes the --ref hole nicely: I tried the compare call against the live API and an old main commit comes back behind while this PR's head comes back diverged, so a PR ref or a fork's commit gets the question, and anything that fails to check asks too. The --timeout fix, the typed---yes-only trust question, the foreground check before forwarding a signal, and the --token advice all look right. Build, tests, clippy and fmt pass on my end.

@zmofei
zmofei changed the base branch from webpki-advisories to main October 21, 2026 17:51
@zmofei
zmofei dismissed mattpodwysocki’s stale review October 21, 2026 17:51

The base branch was changed.

@zmofei
zmofei added this pull request to stack #111 October 21, 2026 18:05

@mattpodwysocki mattpodwysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zmofei, the beta wording reads much better and the CONTRIBUTING example is back. I checked that the merge with main kept everything from the version I approved, and fmt, clippy and the full suite pass on my end. One tiny thing, not blocking: the rewording left uneven line breaks in the CHANGELOG entry ("multi-step recipes of" on its own line) and in the Beta paragraph in docs/commands.md. Approving.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

locationai-team-skills Opened via the location-ai PR creation skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants