diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 70f83ee43..705cc0aa1 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -175,6 +175,8 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **Takeover reconciliation (every hosted ecosystem, v5.0)**: vendoring over a hosted pin (`vendor`, `scan --mode vendored`, `get --mode vendored`) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). **A takeover the vendored backend does not carry through keeps the hosted pin (#853, #944)**: the wet run holds the restore in its group commit, and when the backend then refuses the purl — whatever the code: a pnpm `catalog:` dependency (`vendor_lock_entry_unsupported`), a CRLF `pnpm-lock.yaml` (`vendor_lockfile_crlf_unsupported`), a conflicting `pnpm-workspace.yaml` override — a range, another version or a `>` selector (`vendor_override_conflict`), a uv inline `[tool.uv] sources` table, a prebuilt download that fails, … — or its apply fails with nothing recorded, the restore is rolled back before anything reaches disk: the purl is reported `failed ` with the backend's own code and detail, neither `vendor_takeover_reverted_redirect` nor the restore's advisories are recorded for it, and the hosted wiring stays byte-for-byte (exit 1 / `partialFailure`), so the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed ` by running the backend's dry run over the restored project staged in memory (nothing written). A restore that writes a file outside the group commit's captured set (`.socket/gradle/hosted-index.tsv`) is not rolled back. The `scan` / `get --mode vendored --dry-run` preview, which stages no restore (it never touches the registry), predicts the pnpm (lockfileVersion 9) part of this (#853): a hosted pin the pnpm vendored backend refuses on lock or manifest text (a `catalog:` dependency, a CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml`, a conflicting override, …) is listed `would_refuse` with the wet run's `errorCode` and the backend's own `error` detail, evaluated on the still-hosted project (the pnpm restore only rewrites the entry's `resolution:`, which none of those gates reads); a hosted pin of any other lock flavor or ecosystem — a legacy pnpm 7/8 lock (`lockfileVersion` 5.4 / 6.0) included, whose CRLF, override-conflict and entry refusals are not lock-text gated — still previews `would_vendor` even when the wet takeover refuses it (the wet refusal keeps the hosted pin), except for the gem preflight below. **Gem preflight before the takeover**: `scan` / `get --mode vendored` ask the gem vendored backend's own refusals BEFORE the upstream restore — the manifest gate (`gemfile_not_loaded`: a `gems.rb` twin or a `BUNDLE_GEMFILE`-configured manifest) and the Gemfile declaration gate evaluated on the Gemfile and Gemfile.lock text the restore would leave (`gemfile_declaration_not_editable`: a declaration inside a `group` / `platforms` / conditional block, a parenthesized or duplicate declaration, …) — so a hosted gem vendored mode cannot wire is reported `failed ` with the hosted `Gemfile` / `Gemfile.lock` byte-untouched (exit 1 / `partialFailure`), and their `--dry-run` preview reports that gem as `would_refuse` with the same `errorCode` (exit 0, like the Bun / vlt `would_refuse` rows), never `would_vendor`. Pinned against real Bundler by `tests/e2e_redirect_gem_build.rs`. `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed `, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (for `bun.lock` / `bun.lockb` the detail also adds `, then run \`bun install --force\` (a plain \`bun install\` keeps the patched copy)`) (exit 1 / `partialFailure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed ` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partialFailure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. +**Withdrawn pins (`scan --mode hosted`, additive/MINOR)**: before the rewrite, a hosted scan checks every patch uuid the lockfiles pin, for a package this run covered, against the reference endpoint. A pin the server reports `withdrawn` (unpublished) or `not_found` (no such patch) no longer installs (the patch server answers 410), so it is restored to its default upstream registry entry: the same restore `rollback` runs (see "Hosted unwind coverage"), under the apply lock, `--dry-run` previewing it without writing. If the package has a replacement patch on offer, the rewrite that follows pins the replacement. Each restored pin is one `rolledBack` event (`verified` on a dry run) with `uuid`, `errorCode: "patch_withdrawn"` and `details.mode: "hosted"`; the restore's advisories (`redirect_bun_reinstall_required`, `redirect_pypi_reinstall_required`, `redirect_vlt_reinstall_required`) join `warnings[]`. Only those two statuses unpin: a pin merely absent from the search results (a failed batch, the free-only proxy fallback, a lapsed paid entitlement), or one reported `forbidden`, `pending_build` or `build_failed`, is kept. If the reference lookup fails, every pin is kept with a `redirect_withdrawn_check_failed` warning; a pin the restore refuses (or the apply lock being held) is kept with a `redirect_withdrawn_restore_failed` warning naming the remedy. Neither changes the status or the exit code. A run whose every search batch failed exits before the check. + ### Scan modes (v5.0) **Mode resolution (`resolve_mode_flags`, MAJOR in v5.0).** `--mode`, or `--sync` (agent mode), picks the mode. With neither, `scan` and `get` keep the mode the project's patch state already records (`mode_from_project_state`, #1088; both stores are read from the project the resolved manifest belongs to, so a `--manifest-path` into another project consults that project's ledger): a non-empty vendor ledger (`.socket/vendor/state.json`; also an unreadable or malformed one, so the vendored flow reports it) means **vendored**, a manifest holding patches the ledger does not already cover (by key or base purl), or an unreadable or malformed manifest (so the agent flow reports it), means **agent** — a record the ledger covers, as `get --save-only` then `vendor` leaves, is vendored state — and a project with neither runs **hosted** mode — JSON and human alike (see the hosted paragraph below and [`scan` and `get` JSON](#scan-and-get-json-v50-major)). A project holding both agent and vendored state is a usage error (exit 2, code `mode_ambiguous`, naming the three `--mode` values) and writes nothing. So is a vendor ledger beside hosted pins in the project's lockfiles (#1437), the mixed state a partial `--mode hosted` takeover or per-package `get --mode` leaves: hosted mode keeps no ledger, so its pins are its state, and keeping vendored mode would take every pin over. A human-mode run that kept vendored or agent mode says so on stderr (`Note: using --mode vendored because … already holds vendored patches; pass --mode explicitly to switch modes`); `--json` and `--silent` print no note. **Switching modes is always explicit**: the in-place vendored → hosted takeover (see **Staged takeover**) runs only under `--mode hosted`, the hosted → vendored takeover only under `--mode vendored`, and a bare `scan`/`get` never converts a project. With hosted/vendored PATH arguments and no `--mode`, each project directory takes its mode from its own state. The one exception to the state-derived default: a `--prune` or `--global`/`--global-prefix` scan with no mode has no project lockfile to rewire, so it is **report-only** — discovery, the table, the `updates` array and the `redirectState` block below, plus the `--prune` GC — and, in human mode, ends with the hint `To apply these patches in place, run:` / ` socket-patch scan --mode agent [PATHS]` / ` socket-patch get `. A global scan's hint carries the run's scope, so it can be run verbatim: `-g` (`scan --mode agent -g` / `get -g <…>`), or `--global-prefix ` when a prefix was given (the directory shell-quoted when it needs it). An explicit `--mode hosted` or `--mode vendored` with `--global`/`--global-prefix` is a usage error (exit 2: global installs have no project lockfile to redirect, or to wire vendored artifacts into); `get` enforces the same rule with the same wording. @@ -1384,7 +1386,7 @@ command's integration tests check its output. | `removed` | `remove`, `repair`, `vendor`, `rollback`, `scan --prune` (manifest entries carry `details.manifest: true`) | A manifest entry or vendored state was removed, or (artifact-level, no `purl`) `.socket/` artifacts were swept — that GC carrier sets `bytes` and is not counted in `summary.removed`; the sweep's totals are the envelope's `gc`. | | `verified` | `apply`, `remove`, `repair`, `vendor`, `rollback`, `get`, `scan` (dry run); `vex`; `--update --dry-run` | The action *would* succeed cleanly (`files` lists previewed changes); `vex`: the patch verified and was attested. | | `rebuilt` | `repair`, `vendor` | A missing/corrupt vendored artifact was restored from its exact server download (v5.0: never a lost ledger entry — see `vendor_ledger_missing`). `summary.rebuilt` counts these. | -| `rolledBack` | `rollback`, `remove` | A patched package was restored to its original files (`files` lists them), or (`rollback`) a vendored / hosted wiring was unwound (`details.mode`). `remove`: one per restored installed copy, `details.path` naming it — v5.0, MAJOR: was the GC carrier's `details.rolledBack`. | +| `rolledBack` | `rollback`, `remove`, `scan --mode hosted` | A patched package was restored to its original files (`files` lists them), or (`rollback`) a vendored / hosted wiring was unwound (`details.mode`). `remove`: one per restored installed copy, `details.path` naming it — v5.0, MAJOR: was the GC carrier's `details.rolledBack`. `scan --mode hosted`: a pin whose patch was withdrawn was restored upstream (`errorCode: "patch_withdrawn"`). | ### Stable `errorCode` tags @@ -1574,6 +1576,9 @@ command's integration tests check its output. | `redirect_bun_reinstall_required` | rollback/remove `warnings[]` (+ human stderr), `vendor --revert` `warnings[]` | a wet hosted unwind that restored `bun.lock` / `bun.lockb` pins to the registry record while their installed copy may be kept (the `vendor_bun_reinstall_required` rule: Bun's hoisted linker keeps it through a plain `bun install`). One run-level warning naming every such `name@version` and `bun install --force` (or deleting `node_modules`). `vendor --revert` leaves it out when the vendored revert already advised `vendor_bun_reinstall_required` for every package it re-hosted. | | `vendor_pypi_reinstall_required` | `skipped` (advisory; human: `Warning: …`); rollback/remove `warnings[]`; `scan --prune` top-level `warnings[]` (human: `GC: …`) | a PyPI vendored revert (`vendor --revert`, rollback / remove of a vendored entry, the manifest reconcile, `--preserve-state` and `--dry-run` included) of a PDM, uv or Pipenv entry (#477). These tools reinstall a same-version package only when the LOCKED candidate is a URL or file that differs from the installed one, so once the lock is back on the registry release a plain `pdm sync` / `pdm install`, `uv sync` or `pipenv sync` reports nothing to do and keeps the vendored build. The detail names the purl and the reinstall that restores the upstream bytes: `pdm sync --reinstall` (or recreating the virtualenv / `__pypackages__`), `uv sync --reinstall-package `, or Pipenv's `pipenv run pip uninstall -y && pipenv sync` (the categories from `Pipfile.lock`; `pipenv --rm && pipenv sync` for a clean virtualenv). Poetry, requirements.txt, Hatch and PEP 751 entries do not get it. | | `redirect_pypi_reinstall_required` | rollback/remove `warnings[]` (+ human stderr), `vendor --revert` `warnings[]` | a hosted unwind (wet or `--dry-run`) that restored PyPI pins in `pdm.lock`, `uv.lock` or `Pipfile.lock` to their registry entry: the `vendor_pypi_reinstall_required` rule for hosted pins (the patched build's `direct_url.json` still names the patch server). One run-level warning naming each purl and its reinstall. | +| `redirect_withdrawn_check_failed` | `warnings[]` (warning; human: `Warning: …`) | scan `--mode hosted`: the reference lookup for the uuids the lockfiles pin failed, so the run could not tell whether any was withdrawn. Every pin is kept; status and exit unchanged. See "Withdrawn pins". | +| `redirect_withdrawn_restore_failed` | `warnings[]` (warning; human: `Warning: …` / `Error: …`) | scan `--mode hosted`: a pin whose patch is `withdrawn` / `not_found` could not be restored to its upstream registry entry (the restore refused it, writing the restored files failed, or the apply lock was held). The pin is kept; the detail names the remedy (`git checkout -- `, or re-run / `socket-patch rollback `). Status and exit unchanged. | +| `patch_withdrawn` | `rolledBack` / `verified` (hosted event) | scan `--mode hosted`: the lockfile pinned a patch the server reports `withdrawn` or `not_found`; the pin was restored (dry run: would be) to its upstream registry entry. `reason` names the uuid and status. See "Withdrawn pins". | | `vendor_prebuilt_stub_invalid` | `failed` | RubyGems: the server stub lacks required attributes or is otherwise invalid; no local stub fallback is permitted. | | `vendor_*` / `pypi_*` / `gemfile_*` / `lock_*` / `locked_version_mismatch` / `user_authored_*` / `native_extensions_unsupported` / `platform_gem_unsupported` | `failed`/`skipped` | vendor: per-ecosystem refusal + drift vocabulary; see the Vendor command contract section. New tags are additive (MINOR). | diff --git a/crates/socket-patch-cli/src/commands/scan/mod.rs b/crates/socket-patch-cli/src/commands/scan/mod.rs index 09b0a359c..a5eadaa04 100644 --- a/crates/socket-patch-cli/src/commands/scan/mod.rs +++ b/crates/socket-patch-cli/src/commands/scan/mod.rs @@ -58,6 +58,7 @@ pub(crate) mod rollout; pub mod rollout_args; mod socket_yml_args; pub(crate) mod vendor_flow; +mod withdrawn; use self::discovery::{ collect_vuln_ids, detect_updates, lockfile_only_contains, lockfile_supplement, @@ -2571,6 +2572,17 @@ async fn run_scan( status.finish(); } + // Hosted mode: restore the lockfile pins whose patch the server no + // longer serves, before the rewrite below re-pins any replacement. + // Runs ahead of every "nothing found" exit: a package whose only patch + // was withdrawn has no search results at all. + let withdrawn_sweep = if hosted && !args.common.is_global() && policy.writes_allowed() { + let pins = withdrawn::pins_in_scope(&hosted_pin_list, &all_purls); + Some(withdrawn::sweep(&args.common, &api_client, &pins).await) + } else { + None + }; + // Calculate patch counts let mut free_patches = 0usize; let mut paid_patches = 0usize; @@ -2683,6 +2695,9 @@ async fn run_scan( // Hosted mode: the redirect engine records its events into this // envelope and prints it. if hosted { + if let Some(sweep) = withdrawn_sweep { + sweep.record_into(&mut env); + } return run_redirect( &args, &api_client, diff --git a/crates/socket-patch-cli/src/commands/scan/withdrawn.rs b/crates/socket-patch-cli/src/commands/scan/withdrawn.rs new file mode 100644 index 000000000..c5ccc120b --- /dev/null +++ b/crates/socket-patch-cli/src/commands/scan/withdrawn.rs @@ -0,0 +1,274 @@ +//! Hosted mode's withdrawn-pin sweep: a lockfile pin whose patch the +//! server no longer serves is restored to its upstream registry entry. +//! +//! A hosted pin points the lockfile at the patch server's copy of one +//! patch uuid. Once that patch is unpublished the server stops serving it +//! (the download answers 410), so every fresh install of the pinned +//! lockfile fails. The rewrite never notices on its own: a withdrawn patch +//! drops out of the search results, so nothing selects it and its pin is +//! left as is. +//! +//! Before the rewrite, `scan --mode hosted` asks the reference endpoint +//! about every uuid the lockfiles pin for a package this run covered. A pin +//! whose uuid comes back `withdrawn` (unpublished) or `not_found` (no such +//! patch) is restored upstream, the way `rollback` restores it +//! ([`crate::commands::hosted_unwind::run_hosted_leg`]). If the package has +//! a replacement patch on offer, the rewrite that follows pins the +//! replacement, so the net effect is an update. Otherwise the package goes +//! back to the registry's unpatched copy. +//! +//! Only those two explicit statuses unpin. A pin that is merely missing +//! from the search results is kept: that is also what a failed search +//! batch, a fallback to the free-only public proxy, or a lapsed paid +//! entitlement looks like. A `forbidden`, `pending_build` or `build_failed` +//! answer keeps the pin too. If the reference lookup fails (the patch API +//! is down, say), every pin is kept and the run warns. + +use std::collections::BTreeSet; +use std::time::Duration; + +use socket_patch_core::api::client::ApiClient; +use socket_patch_core::patch::redirect::upstream::HostedPin; +use socket_patch_core::utils::purl_key::PurlKey; + +use crate::args::GlobalArgs; +use crate::commands::agent_download::tag_mode; +use crate::json_envelope::{Envelope, PatchAction, PatchEvent}; + +/// `reason` code on the `rolledBack` event of a pin the sweep restored. +pub(super) const PATCH_WITHDRAWN: &str = "patch_withdrawn"; + +/// Warning: a withdrawn pin could not be restored upstream (the restore +/// refused it, the restored files could not be written, or the apply lock +/// was held). The pin is unchanged; the detail names the remedy. +pub(super) const REDIRECT_WITHDRAWN_RESTORE_FAILED: &str = "redirect_withdrawn_restore_failed"; + +/// Warning: the reference lookup for the pinned uuids failed, so the run +/// could not tell whether any pin was withdrawn. Every pin is kept. +pub(super) const REDIRECT_WITHDRAWN_CHECK_FAILED: &str = "redirect_withdrawn_check_failed"; + +/// Whether a reference `status` means the pinned patch is gone for good. +fn is_gone(status: &str) -> bool { + matches!(status, "withdrawn" | "not_found") +} + +/// What the sweep did, for the `--json` envelope. Human output is printed +/// as the sweep runs. +#[derive(Debug, Default)] +pub(super) struct WithdrawnSweep { + /// `rolledBack` (`verified` on a dry run) per restored pin. + events: Vec, + /// `(code, detail)` run-level warnings. + warnings: Vec<(String, String)>, +} + +impl WithdrawnSweep { + /// Fold the outcome into the scan envelope. + pub(super) fn record_into(self, env: &mut Envelope) { + for event in self.events { + env.record(event); + } + for (code, detail) in self.warnings { + env.warn(code, detail); + } + } + + fn warn(&mut self, common: &GlobalArgs, code: &str, detail: String) { + // Warnings print even under --silent: a kept withdrawn pin breaks + // the next fresh install. + if !common.json { + eprintln!("Warning: {detail}"); + } + self.warnings.push((code.to_string(), detail)); + } +} + +/// The pins in `pins` whose package this run covered (`covered`: the purls +/// it queried), matched by release identity, so a qualified or +/// percent-encoded spelling still names its pin. +pub(super) fn pins_in_scope( + pins: &[HostedPin], + covered: impl IntoIterator>, +) -> Vec { + let covered: BTreeSet = covered + .into_iter() + .map(|p| PurlKey::new(p.as_ref())) + .collect(); + pins.iter() + .filter(|pin| covered.contains(&PurlKey::new(&pin.purl))) + .cloned() + .collect() +} + +/// Check `pins` against the reference endpoint and restore the withdrawn +/// ones upstream (see the module docs). A wet restore runs under the apply +/// lock, released before the hosted rewrite takes it again. +pub(super) async fn sweep( + common: &GlobalArgs, + api_client: &ApiClient, + pins: &[HostedPin], +) -> WithdrawnSweep { + let mut out = WithdrawnSweep::default(); + if pins.is_empty() { + return out; + } + let uuids: Vec = pins + .iter() + .map(|pin| pin.uuid.clone()) + .collect::>() + .into_iter() + .collect(); + let references = match api_client.fetch_registry_references(&uuids).await { + Ok(references) => references, + Err(e) => { + out.warn( + common, + REDIRECT_WITHDRAWN_CHECK_FAILED, + format!( + "could not check whether the hosted patches pinned in the lockfiles are \ + still published ({e}); every pin was kept, re-run to check again" + ), + ); + return out; + } + }; + // A uuid the response leaves out is not known to be gone: kept. + let gone: Vec<(HostedPin, String)> = pins + .iter() + .filter_map(|pin| { + let status = &references.get(&pin.uuid)?.status; + is_gone(status).then(|| (pin.clone(), status.clone())) + }) + .collect(); + if gone.is_empty() { + return out; + } + + if !common.json && !common.silent { + for (pin, status) in &gone { + eprintln!( + "Note: patch {} for {} is no longer published ({status}); restoring its \ + upstream registry entry", + pin.uuid, pin.purl + ); + } + } + + let _lock = if common.dry_run { + None + } else { + let socket_dir = common.socket_dir(); + let timeout = Duration::from_secs(common.lock_timeout.unwrap_or(0)); + match crate::commands::lock_cli::acquire_with_status(&socket_dir, timeout) { + Ok(guard) => Some(guard), + Err(err) => { + let (_, message) = crate::commands::lock_cli::lock_failure(&err, timeout); + for (pin, status) in &gone { + out.warn( + common, + REDIRECT_WITHDRAWN_RESTORE_FAILED, + format!( + "{} is pinned to patch {}, which is no longer published \ + ({status}), but it was not restored: {message}; re-run, or run \ + `socket-patch rollback {}`", + pin.purl, pin.uuid, pin.purl + ), + ); + } + return out; + } + } + }; + + let to_restore: Vec = gone.iter().map(|(pin, _)| pin.clone()).collect(); + let leg = crate::commands::hosted_unwind::run_hosted_leg(common, &to_restore).await; + + let action = if common.dry_run { + PatchAction::Verified + } else { + PatchAction::RolledBack + }; + for (pin, status) in &gone { + if !leg.reverted.iter().any(|p| p == &pin.purl) { + continue; + } + let verb = if common.dry_run { + "would be restored" + } else { + "was restored" + }; + out.events.push(tag_mode( + PatchEvent::new(action, pin.purl.as_str()) + .with_uuid(pin.uuid.as_str()) + .with_reason( + PATCH_WITHDRAWN, + format!( + "patch {} is no longer published ({status}); the pin {verb} to the \ + upstream registry entry", + pin.uuid + ), + ), + Some("hosted"), + )); + } + // `run_hosted_leg` already printed each failure to stderr. + for (purl, why) in leg.failed { + out.warnings.push(( + REDIRECT_WITHDRAWN_RESTORE_FAILED.to_string(), + format!("{purl}: {why}"), + )); + } + for (code, detail) in leg.warnings { + if !common.json && !common.silent { + eprintln!("Warning: {detail}"); + } + out.warnings.push((code, detail)); + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + + fn pin(purl: &str, uuid: &str) -> HostedPin { + HostedPin { + purl: purl.to_string(), + uuid: uuid.to_string(), + files: vec!["package-lock.json".to_string()], + } + } + + #[test] + fn only_withdrawn_and_not_found_are_gone() { + assert!(is_gone("withdrawn")); + assert!(is_gone("not_found")); + for kept in [ + "granted", + "reused", + "pending_build", + "build_failed", + "forbidden", + "", + ] { + assert!(!is_gone(kept), "{kept} must keep the pin"); + } + } + + #[test] + fn scope_matches_by_release_identity() { + let pins = vec![ + pin("pkg:npm/@scope/a@1.0.0", "u1"), + pin("pkg:npm/b@2.0.0", "u2"), + ]; + let in_scope = pins_in_scope(&pins, ["pkg:npm/%40scope/a@1.0.0"]); + assert_eq!(in_scope, vec![pins[0].clone()]); + } + + #[test] + fn uncovered_pins_are_out_of_scope() { + let pins = vec![pin("pkg:npm/b@2.0.0", "u2")]; + assert!(pins_in_scope(&pins, ["pkg:npm/b@2.0.1"]).is_empty()); + assert!(pins_in_scope(&pins, Vec::::new()).is_empty()); + } +} diff --git a/crates/socket-patch-cli/tests/in_process_scan_withdrawn.rs b/crates/socket-patch-cli/tests/in_process_scan_withdrawn.rs new file mode 100644 index 000000000..a4629eab5 --- /dev/null +++ b/crates/socket-patch-cli/tests/in_process_scan_withdrawn.rs @@ -0,0 +1,527 @@ +//! `scan --mode hosted` over a lockfile pin whose patch was withdrawn. +//! +//! Each test wires an npm package-lock project through the REAL hosted flow +//! (in-process `scan --mode hosted` against a wiremock API, the +//! `in_process_rollback_hosted.rs` fixture), then re-scans against a second +//! mock whose reference endpoint reports the pinned uuid's new state: +//! +//! - `withdrawn` / `not_found`: the pin is restored to its upstream registry +//! entry (byte-identical to the pristine lock), or, when a replacement +//! patch is offered, re-pinned to the replacement. +//! - `granted`, or a failed reference lookup: the pin is kept, even when the +//! search results no longer list the patch. +//! +//! `#[serial]`: every command's `run` mirrors env toggles into +//! process-global env vars (`apply_env_toggles`), and the upstream restore +//! reads `SOCKET_NPM_REGISTRY`. + +#[path = "common/hermetic.rs"] +mod hermetic; + +use std::path::Path; + +use serde_json::Value; +use serial_test::serial; +use socket_patch_cli::commands::scan::{run as scan_run, ScanArgs, ScanMode}; +use wiremock::matchers::{method, path, path_regex}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +const ORG: &str = "test-org"; +const NAME: &str = "in-proc-redirect"; +const VERSION: &str = "1.0.0"; +const PURL: &str = "pkg:npm/in-proc-redirect@1.0.0"; +const UUID: &str = "11111111-1111-4111-8111-111111111111"; +const HOSTED_URL: &str = "http://patch.test/patch/npm/in-proc-redirect/1.0.0/22222222-2222-4222-8222-222222222222/11111111-1111-4111-8111-111111111111/in-proc-redirect-1.0.0.tgz"; +const PATCHED_SHA512: &str = "sha512-PATCHEDpatchedPATCHEDpatched0123456789=="; +/// The replacement patch a superseding offer pins. +const UUID2: &str = "33333333-3333-4333-8333-333333333333"; +const HOSTED_URL2: &str = "http://patch.test/patch/npm/in-proc-redirect/1.0.0/22222222-2222-4222-8222-222222222222/33333333-3333-4333-8333-333333333333/in-proc-redirect-1.0.0.tgz"; +const PATCHED_SHA512_2: &str = "sha512-REPLACEMENTreplacementREPLACEMENT0123456=="; + +fn scan_args(cwd: &Path, api_url: String, dry_run: bool) -> ScanArgs { + ScanArgs { + socket_yml: Default::default(), + paths: Vec::new(), + packages: Vec::new(), + common: socket_patch_cli::args::GlobalArgs { + cwd: cwd.to_path_buf(), + org: Some(ORG.to_string()), + api_token: Some("fake".to_string()), + api_url: Some(api_url), + // The mock patch host's pins count as hosted. + patch_server_url: Some("http://patch.test".to_string()), + json: true, + yes: true, + dry_run, + ..socket_patch_cli::args::GlobalArgs::default() + }, + batch_size: Some(100), + prune: false, + sync: false, + mode: Some(ScanMode::Hosted), + all_releases: false, + vex: Default::default(), + rollout: Default::default(), + } +} + +/// Run an in-process hosted scan with the npm registry pointed at +/// `server`, scrubbing the env toggles `run` mirrors into the process. +async fn scan(cwd: &Path, server: &MockServer, dry_run: bool) -> i32 { + std::env::set_var( + "SOCKET_NPM_REGISTRY", + format!("{}/npm-registry", server.uri()), + ); + let code = scan_run(scan_args(cwd, server.uri(), dry_run)).await; + for var in ["SOCKET_NPM_REGISTRY", "SOCKET_DRY_RUN"] { + std::env::remove_var(var); + } + code +} + +fn offer(uuid: &str) -> Value { + serde_json::json!({ + "uuid": uuid, "purl": PURL, "tier": "free", + "cveIds": [], "ghsaIds": [], "severity": "high", + "title": "withdrawn fixture" + }) +} + +/// The batch search (offering `offers`) and by-package endpoints. +async fn mock_search(server: &MockServer, offers: &[&str]) { + let packages = if offers.is_empty() { + serde_json::json!([]) + } else { + serde_json::json!([{ + "purl": PURL, + "patches": offers.iter().map(|u| offer(u)).collect::>(), + }]) + }; + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/batch"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "packages": packages, + "canAccessPaidPatches": false, + }))) + .mount(server) + .await; + let records: Vec = offers + .iter() + .map(|u| { + serde_json::json!({ + "uuid": u, "purl": PURL, + "publishedAt": "2024-01-01T00:00:00Z", + "description": "x", "license": "MIT", "tier": "free", + "vulnerabilities": {} + }) + }) + .collect(); + Mock::given(method("GET")) + .and(path_regex(format!( + "^/v0/orgs/{ORG}/patches/by-package/.+$" + ))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "patches": records, + "canAccessPaidPatches": false, + }))) + .mount(server) + .await; +} + +fn granted(url: &str, sha512: &str) -> Value { + serde_json::json!({ + "status": "granted", + "url": url, + "purl": PURL, + "artifacts": [{ + "kind": "tarball", + "url": url, + "integrity": { "sha512": sha512 } + }], + "registryOverride": null + }) +} + +/// The reference endpoint answering `results`. +async fn mock_references(server: &MockServer, results: Value) { + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/package"))) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!({ "results": results })), + ) + .mount(server) + .await; +} + +async fn mock_view(server: &MockServer, uuid: &str) { + Mock::given(method("GET")) + .and(path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "uuid": uuid, + "purl": PURL, + "publishedAt": "2024-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": "a".repeat(64), + "afterHash": "b".repeat(64), + } + }, + "vulnerabilities": {}, + "description": "x", "license": "MIT", "tier": "free" + }))) + .mount(server) + .await; +} + +/// The npm registry's version document, for the upstream restore. +async fn mock_npm_registry(server: &MockServer) { + Mock::given(method("GET")) + .and(path(format!("/npm-registry/{NAME}/{VERSION}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "name": NAME, + "version": VERSION, + "dist": { + "tarball": format!("https://registry.npmjs.org/{NAME}/-/{NAME}-{VERSION}.tgz"), + "integrity": "sha512-UPSTREAMupstream==", + } + }))) + .mount(server) + .await; +} + +/// Write the npm project and return the pristine lock bytes (normalized +/// the way the redirect writer serializes, so a restore can be compared +/// byte for byte). +fn write_npm_project(root: &Path) -> String { + std::fs::write( + root.join("package.json"), + format!( + r#"{{ "name": "consumer", "version": "0.0.0", "dependencies": {{ "{NAME}": "{VERSION}" }} }}"# + ), + ) + .unwrap(); + let pkg = root.join("node_modules").join(NAME); + std::fs::create_dir_all(&pkg).unwrap(); + std::fs::write( + pkg.join("package.json"), + format!(r#"{{ "name": "{NAME}", "version": "{VERSION}" }}"#), + ) + .unwrap(); + let raw = format!( + r#"{{ + "name": "consumer", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": {{ + "": {{ "name": "consumer", "version": "0.0.0", "dependencies": {{ "{NAME}": "{VERSION}" }} }}, + "node_modules/{NAME}": {{ + "version": "{VERSION}", + "resolved": "https://registry.npmjs.org/{NAME}/-/{NAME}-{VERSION}.tgz", + "integrity": "sha512-UPSTREAMupstream==" + }} + }} +}} +"# + ); + let normalized = format!( + "{}\n", + serde_json::to_string_pretty(&serde_json::from_str::(&raw).unwrap()).unwrap() + ); + std::fs::write(root.join("package-lock.json"), &normalized).unwrap(); + normalized +} + +fn lock(root: &Path) -> String { + std::fs::read_to_string(root.join("package-lock.json")).unwrap() +} + +/// Wire the project to [`UUID`] through a real hosted scan; returns the +/// pristine and the wired lock. +async fn wired_project(root: &Path) -> (String, String) { + let pristine = write_npm_project(root); + let server = MockServer::start().await; + mock_search(&server, &[UUID]).await; + mock_references( + &server, + serde_json::json!({ UUID: granted(HOSTED_URL, PATCHED_SHA512) }), + ) + .await; + mock_view(&server, UUID).await; + assert_eq!(scan(root, &server, false).await, 0, "wiring scan"); + let wired = lock(root); + assert!( + wired.contains(HOSTED_URL) && wired.contains(PATCHED_SHA512), + "the lock must be wired before the re-scan means anything:\n{wired}" + ); + (pristine, wired) +} + +/// Requests `server` received on the reference endpoint. +async fn reference_posts(server: &MockServer) -> usize { + server + .received_requests() + .await + .unwrap_or_default() + .iter() + .filter(|r| r.url.path().ends_with("/patches/package")) + .count() +} + +#[tokio::test] +#[serial] +async fn a_withdrawn_pin_is_restored_upstream() { + let tmp = tempfile::tempdir().unwrap(); + let (pristine, _) = wired_project(tmp.path()).await; + + // The patch was unpublished: search no longer lists it. + let server = MockServer::start().await; + mock_search(&server, &[]).await; + mock_references( + &server, + serde_json::json!({ UUID: { "status": "withdrawn" } }), + ) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 0); + + assert_eq!( + lock(tmp.path()), + pristine, + "the withdrawn pin must be restored to the upstream registry entry" + ); + assert!( + !tmp.path().join(".socket").exists(), + "the sweep's apply lock leaves no .socket/ residue" + ); +} + +#[tokio::test] +#[serial] +async fn a_not_found_pin_is_restored_upstream() { + let tmp = tempfile::tempdir().unwrap(); + let (pristine, _) = wired_project(tmp.path()).await; + + let server = MockServer::start().await; + mock_search(&server, &[]).await; + mock_references( + &server, + serde_json::json!({ UUID: { "status": "not_found" } }), + ) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 0); + + assert_eq!(lock(tmp.path()), pristine); +} + +#[tokio::test] +#[serial] +async fn a_withdrawn_pin_with_a_replacement_is_repinned() { + let tmp = tempfile::tempdir().unwrap(); + wired_project(tmp.path()).await; + + // The old patch was unpublished and a new one published for the same + // package version. + let server = MockServer::start().await; + mock_search(&server, &[UUID2]).await; + mock_references( + &server, + serde_json::json!({ + UUID: { "status": "withdrawn" }, + UUID2: granted(HOSTED_URL2, PATCHED_SHA512_2), + }), + ) + .await; + mock_view(&server, UUID2).await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 0); + + let after = lock(tmp.path()); + assert!( + after.contains(HOSTED_URL2) && after.contains(PATCHED_SHA512_2), + "the replacement patch must be pinned:\n{after}" + ); + assert!( + !after.contains(UUID), + "nothing may still name the withdrawn patch:\n{after}" + ); +} + +#[tokio::test] +#[serial] +async fn a_pin_missing_from_search_but_still_granted_is_kept() { + let tmp = tempfile::tempdir().unwrap(); + let (_, wired) = wired_project(tmp.path()).await; + + // Absent from search (a failed batch, the free-only proxy and a lapsed + // entitlement all look like this), but the server still serves it. + let server = MockServer::start().await; + mock_search(&server, &[]).await; + mock_references( + &server, + serde_json::json!({ UUID: granted(HOSTED_URL, PATCHED_SHA512) }), + ) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 0); + + assert_eq!(lock(tmp.path()), wired, "a served pin is never unpinned"); + assert_eq!( + reference_posts(&server).await, + 1, + "the pin was checked against the reference endpoint" + ); +} + +#[tokio::test] +#[serial] +async fn a_forbidden_pin_is_kept() { + let tmp = tempfile::tempdir().unwrap(); + let (_, wired) = wired_project(tmp.path()).await; + + let server = MockServer::start().await; + mock_search(&server, &[]).await; + mock_references( + &server, + serde_json::json!({ UUID: { "status": "forbidden" } }), + ) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 0); + + assert_eq!(lock(tmp.path()), wired); +} + +#[tokio::test] +#[serial] +async fn a_failed_reference_lookup_keeps_every_pin() { + let tmp = tempfile::tempdir().unwrap(); + let (_, wired) = wired_project(tmp.path()).await; + + // The patch API answers search but its reference endpoint is down. + let server = MockServer::start().await; + mock_search(&server, &[]).await; + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/package"))) + .respond_with(ResponseTemplate::new(500)) + .mount(&server) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 0); + + assert_eq!(lock(tmp.path()), wired, "an outage must never unpin"); +} + +#[tokio::test] +#[serial] +async fn a_search_outage_never_reaches_the_sweep() { + let tmp = tempfile::tempdir().unwrap(); + let (_, wired) = wired_project(tmp.path()).await; + + // Every search batch fails: the scan fails before the sweep runs. + let server = MockServer::start().await; + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/batch"))) + .respond_with(ResponseTemplate::new(500)) + .mount(&server) + .await; + mock_references( + &server, + serde_json::json!({ UUID: { "status": "withdrawn" } }), + ) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, false).await, 1); + + assert_eq!(lock(tmp.path()), wired); + assert_eq!(reference_posts(&server).await, 0); +} + +#[tokio::test] +#[serial] +async fn a_dry_run_previews_the_restore_without_writing() { + let tmp = tempfile::tempdir().unwrap(); + let (_, wired) = wired_project(tmp.path()).await; + + let server = MockServer::start().await; + mock_search(&server, &[]).await; + mock_references( + &server, + serde_json::json!({ UUID: { "status": "withdrawn" } }), + ) + .await; + mock_npm_registry(&server).await; + assert_eq!(scan(tmp.path(), &server, true).await, 0); + + assert_eq!(lock(tmp.path()), wired, "a dry run writes nothing"); + assert!(!tmp.path().join(".socket").exists()); +} + +/// The `--json` envelope (through the binary: an in-process run prints to +/// the real stdout): one `rolledBack` event with `reason: patch_withdrawn`. +#[tokio::test] +#[serial] +async fn the_envelope_reports_the_restored_pin() { + let tmp = tempfile::tempdir().unwrap(); + let (pristine, _) = wired_project(tmp.path()).await; + + let server = MockServer::start().await; + mock_search(&server, &[]).await; + mock_references( + &server, + serde_json::json!({ UUID: { "status": "withdrawn" } }), + ) + .await; + mock_npm_registry(&server).await; + + let mut cmd = hermetic::binary_command(); + for key in ["NPM_CONFIG_REGISTRY", "npm_config_registry"] { + cmd.env_remove(key); + } + let out = cmd + .env( + "SOCKET_NPM_REGISTRY", + format!("{}/npm-registry", server.uri()), + ) + .args([ + "scan", + "--json", + "--yes", + "--mode", + "hosted", + "--org", + ORG, + "--api-token", + "fake", + "--api-url", + &server.uri(), + "--patch-server-url", + "http://patch.test", + "--cwd", + tmp.path().to_str().unwrap(), + ]) + .output() + .expect("run socket-patch"); + let envelope: Value = serde_json::from_slice(&out.stdout).unwrap_or_else(|e| { + panic!( + "scan --json stdout must be a JSON envelope: {e}\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) + }); + assert_eq!(out.status.code(), Some(0), "{envelope:#}"); + assert_eq!(envelope["status"], "success", "{envelope:#}"); + let restored: Vec<&Value> = envelope["events"] + .as_array() + .unwrap() + .iter() + .filter(|e| e["action"] == "rolledBack") + .collect(); + assert_eq!(restored.len(), 1, "{envelope:#}"); + assert_eq!(restored[0]["purl"], PURL); + assert_eq!(restored[0]["uuid"], UUID); + assert_eq!(restored[0]["errorCode"], "patch_withdrawn", "{envelope:#}"); + assert_eq!(restored[0]["details"]["mode"], "hosted"); + assert_eq!(lock(tmp.path()), pristine); +} diff --git a/crates/socket-patch-cli/tests/scan/scan_render.rs b/crates/socket-patch-cli/tests/scan/scan_render.rs index a996d2ec9..0a718b102 100644 --- a/crates/socket-patch-cli/tests/scan/scan_render.rs +++ b/crates/socket-patch-cli/tests/scan/scan_render.rs @@ -1128,10 +1128,27 @@ async fn scan_hosted_human_prints_table_updates_and_redirects() { !stderr.contains("Non-interactive mode detected"), "scan never prompts; got {stderr:?}" ); + // Two reference posts: the withdrawn-pin sweep checks the pinned + // OLD_UUID first (forbidden here, so the pin is kept), then the engine + // resolves the offered UUID. + let reqs = recorded(&mock).await; assert_eq!( - reference_posts(&recorded(&mock).await), - 1, - "the engine resolved the reference" + reference_posts(&reqs), + 2, + "the withdrawn sweep and the engine each resolved a reference" + ); + let reference_bodies: Vec = reqs + .iter() + .filter(|r| format!("{}", r.method) == "POST" && r.url.path().ends_with("/patches/package")) + .map(|r| String::from_utf8_lossy(&r.body).into_owned()) + .collect(); + assert!( + reference_bodies.iter().any(|b| b.contains(OLD_UUID)), + "the withdrawn sweep checked the pinned uuid; got {reference_bodies:?}" + ); + assert!( + reference_bodies.iter().any(|b| b.contains(UUID)), + "the engine resolved the offered uuid; got {reference_bodies:?}" ); // `--dry-run` previews without confirming (the engine honors it itself).