Skip to content

RTECO-2751 - Look up the newest allowed build-info extractor version - #1636

Open
bhanurp wants to merge 15 commits into
masterfrom
RTECO-2751-auto-extractor-version
Open

bhanurp wants to merge 15 commits into
masterfrom
RTECO-2751-auto-extractor-version

Conversation

@bhanurp

@bhanurp bhanurp commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

RTECO-2751

Depends on jfrog/build-info-go#440. This PR needs a go.mod bump to the build-info-go commit of that PR after it is merged. It stays a draft until then.

What this adds

If JFrog Curation blocks the fixed extractor version in a user's remote repository, jf mvn and jf gradle fail until a new CLI is released. This PR finds the newest allowed version at run time.

  • ListMavenExtractorVersions and ListGradleExtractorVersions read maven-metadata.xml from the same place as the extractor jar. That place is releases.jfrog.io, or the remote repository of JFROG_CLI_RELEASES_REPO or the deprecated JFROG_CLI_EXTRACTORS_REMOTE. Both variables keep their own URL layout. The Gradle lookup stays in the major line that the Gradle version needs.
  • DownloadDependency reports a Curation block as biutils.ErrBlockedByCuration. A 403 counts as a Curation block only if the body has "type": "Curation" or the Curation text. Any other 403 (for example a missing permission) is not retried. A download copies the file into place atomically.
  • A pinned version: JFROG_CLI_MAVEN_EXTRACTOR_VERSION and JFROG_CLI_GRADLE_EXTRACTOR_VERSION. A pin skips the lookup. The Gradle pin must be in the major line that the Gradle version needs.
  • The list of versions and the blocked versions are saved for one hour in <dependencies>/extractor-versions/. The files have owner-only permissions. Damaged files are ignored. A failed lookup uses an old saved list. A run with a saved list sends no request.
  • The lookup has a timeout of 10 seconds and does not announce a download. The deprecation warning appears once per command.

DownloadDependency is also used by jfrog-cli-security. For a normal download nothing changes. A Curation 403 now returns an error that wraps ErrBlockedByCuration, and the original text stays in the message.

Tests

  • New unit tests cover the Curation detection (JSON type, text, other 403, empty body, wrapped error), the whole chain through the build-info-go walk against a fake server, both remote variables and their URL layout, the pin rules, the saved list (fresh, old, other repository, damaged, server down), the remembered blocks, and the atomic copy.
  • go test -race ./utils/dependencies/ ./utils/coreutils/ passes. gofmt and go vet are clean.
  • Verified end to end with a real jf mvn (Maven 3.10) and jf gradle (Gradle 8.7). Both were built from this change together with the build-info-go and jfrog-cli-artifactory PRs. Cases: direct from releases.jfrog.io, through a remote repository with Curation, the deprecated variable, blocked versions, pins, a server that is down, a 502 on the newest version, and six parallel runs.

  • All tests passed. If this feature is not already covered by the tests, I added new tests.
  • All static analysis checks passed.
  • This pull request is on the master branch.
  • I used gofmt for formatting the code before submitting the pull request.

Read maven-metadata.xml from the place the extractor jars come from (releases.jfrog.io, or the
remote repository of JFROG_CLI_RELEASES_REPO / JFROG_CLI_EXTRACTORS_REMOTE) and give the list of
versions to build-info-go. Skip a version that JFrog Curation blocks.

- ListMavenExtractorVersions and ListGradleExtractorVersions, with the pin variables
  JFROG_CLI_MAVEN_EXTRACTOR_VERSION and JFROG_CLI_GRADLE_EXTRACTOR_VERSION.
- DownloadDependency reports a Curation block as biutils.ErrBlockedByCuration, and copies the
  downloaded file into place atomically.
- The list of versions and the blocked versions are saved for one hour.
…sion selection

Temporary, until the build-info-go change is merged: then bump to the merged commit.
C1-C11 Add tests: non-Curation errors are not remembered as blocked, second Curation check on the download, Range header, cache pruning and merging, case of the Curation type, blocked beats verified, file mode of the cache files, Gradle list path, Verify flag, malformed metadata, copyFileAtomically
C12 Rename serverReachable to listIsCurrent
C13 versionsCachePath uses urlCachePath
Use the build-info-go commit with the matching tests
K1 Test that a slow probe times out
K2 A lost rename of the jar to a jar that already exists is not an error (Windows)
K3 Test that the list functions are silent and the deprecated warning is printed once
K4 Test DownloadDependency with an archive
K5 checkRecentlyBlocked takes the URL that the caller resolved
K6 Note that a lost cache entry is accepted
Use the build-info-go commit with the matching fixes
… no new failure

The error of a lookup that is skipped because of the failure window wraps ErrLookupNotRepeated, so build-info-go logs it at Debug level.
Use the build-info-go commit with that change.

@itsmeleela itsmeleela left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of RTECO-2751. Blocking item: this pins an unmerged build-info-go commit and is not a draft.

Comment thread go.mod Resolved
Comment thread utils/dependencies/curation.go Resolved
Comment thread utils/coreutils/utils.go Resolved
Comment thread utils/dependencies/utils.go Resolved
After a crash, an empty or partial jar must not stay under the final name. Use the build-info-go commit with the same change.
build-info-go now keeps an existing file when the rename is refused (Windows), so the copy of this logic and its tests are removed.
…extractor version

When it is false, the CLI makes no lookup of versions and no request to check a downloaded jar, and uses the bundled version, as before the newest version was chosen at run time. A pinned version is still used. A value that is not true or false fails the command.
…wnload error message

The request for the reason of a 403 reads at most 64 KiB, and stops after the verify timeout, so a server that ignores the Range header cannot make the CLI read a whole jar or wait for ever. The message of a failed download had two verbs and one argument.
…tion

The CI lint rule forcetypeassert refused w.(http.Flusher). http.NewResponseController does the same flush.
@bhanurp
bhanurp deployed to build-gate November 19, 2026 05:25 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

This branch was successfully deployed

1 active deployment
build-gate — 51290f01 Deployed Nov 19, 2026 by bhanurp via gate #200
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new feature Automatically generated release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants