Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions commands/curation/curationaudit.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ const (

errorTemplateUnsupportedTech = "It looks like this project uses '%s' to download its dependencies. " +
"This package manager however isn't supported by this command."
errorNothingAudited = "no supported package manager was recognised in any working directory, so nothing was audited. " +
"Check the working directories, or pass --requirements-file for a custom pip file name."
Comment on lines +97 to +98

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When --requirements-file was passed but matched nothing, the "pass --requirements-file" hint isn't actionable. Consider naming the value only when it is set (the wording is neutral because the same error also covers unsupported-only and HF-empty runs):

Suggested change
errorNothingAudited = "no supported package manager was recognised in any working directory, so nothing was audited. " +
"Check the working directories, or pass --requirements-file for a custom pip file name."
errorNothingAudited = "nothing was audited: no supported project files were found in the working directories. " +
"For a custom-named pip requirements file, use --requirements-file."
errorNothingAuditedWithRequirementsFile = "nothing was audited: no supported project files were found in the working directories " +
"(--requirements-file %q matched no file there)."

plus a small helper that picks the message, used at the call site in Run():

func (ca *CurationAuditCommand) nothingAuditedError() error {
	if file := ca.PipRequirementsFile(); file != "" {
		return errorutils.CheckErrorf(errorNothingAuditedWithRequirementsFile, file)
	}
	return errorutils.CheckErrorf(errorNothingAudited)
}

Review generated by xray-pr-review


WaiverRequestForbidden = "One or more policies blocking this package do not allow waiver requests."
WaiverRequestApproved = "The waiver request was automatically granted; you can use this package.\nNOTE: The policy owner may review this waiver more thoroughly and contact you if issues are found."
Expand Down Expand Up @@ -311,7 +313,9 @@ type CurationAuditCommand struct {
dockerImageName string
huggingFaceModel string
// hfProjectNameHint is the collision-free HF root-node name for the current working dir.
hfProjectNameHint string
hfProjectNameHint string
// detectedTechCount counts technologies detected across all working dirs; zero fails the run.
detectedTechCount int
includeCachedPackages bool
mvnIncludePluginDeps bool
scriptPath string
Expand Down Expand Up @@ -495,6 +499,9 @@ func (ca *CurationAuditCommand) Run() (err error) {
if ca.Progress() != nil {
err = errors.Join(err, ca.Progress().Quit())
}
if err == nil && ca.detectedTechCount == 0 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runs that produce no audit report (an unsupported-only directory such as conanfile.txt, or Hugging Face auto-discovery that finds no model references) still exit 0 with nothing audited, because the counter is incremented on detection (techs), not on a produced report. Consider checking results instead. results only gets an entry once an audit produced a report, so it needs no extra state; detectedTechCount (field at 316-318 and the += in doCurateAudit) can then be removed.

Suggested change
if err == nil && ca.detectedTechCount == 0 {
if err == nil && len(results) == 0 {

One question: an HF run where every reference is unresolved or dynamic still adds a warnings-only placeholder to results (it already exits 0 today with a "NOT audited" warning). Is exit 0 intended there?


Review generated by xray-pr-review

err = errorutils.CheckErrorf(errorNothingAudited)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the exit-code change (exit 0 → error when nothing is audited) part of XRAY-170058's scope? It also applies to the curation-after-failure path (an extra [Error] line after the user's own failed install) and to HF-only runs under a globally exported HF_ENDPOINT. If it is intended, could the PR description, the release note and the help.go "Gotchas" mention it? Would a warning first be an option for existing CI usage?


Review generated by xray-pr-review

}
// Print after the spinner has stopped so messages are not overwritten by ANSI clear codes.
for _, w := range ca.pendingWarnings {
log.Warn(w)
Expand Down Expand Up @@ -780,7 +787,7 @@ func (ca *CurationAuditCommand) techsToAudit() []string {
log.Debug(fmt.Sprintf("Docker image name '%s' was provided, running Docker curation audit.", ca.DockerImageName()))
return []string{techutils.Docker.String()}
default:
techs := promotePnpmWorkspaceMember(techutils.DetectedTechnologiesListForCurationAudit())
techs := promotePnpmWorkspaceMember(techutils.DetectedTechnologiesListForCurationAuditWithRequirementsFile(ca.PipRequirementsFile()))
techs = promoteYarnWorkspaceMember(techs)
techs = promotePipToUvIn(techs, ca.OriginPath)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With a machine-global ~/.config/uv/uv.toml (a curation-only signal, PromotePipToUvForCurationAudit), --requirements-file=requirements-dev.txt — the example in the jf ca help — is audited as uv, so the flag is silently ignored. jf audit doesn't consider the global config. Consider using techutils.PromotePipToUv only when the named file exists in this directory, so other --working-dirs entries keep the default promotion, and replacing this line with techs = ca.promotePipToUv(techs):

func (ca *CurationAuditCommand) promotePipToUv(techs []string) []string {
	if ca.PipRequirementsFile() == "" || ca.OriginPath == "" {
		return promotePipToUvIn(techs, ca.OriginPath)
	}
	if _, statErr := os.Stat(filepath.Join(ca.OriginPath, ca.PipRequirementsFile())); statErr != nil {
		return promotePipToUvIn(techs, ca.OriginPath)
	}
	promoted := techutils.PromotePipToUv(techutils.ToTechnologies(techs), ca.OriginPath)
	result := make([]string, 0, len(promoted))
	for _, t := range promoted {
		result = append(result, t.String())
	}
	return result
}

A uv.lock next to the custom file still promotes to uv, as in jf audit. An absolute --requirements-file value isn't covered by the stat (it falls back to the current behavior).


Review generated by xray-pr-review

techs = dedupeDotnetFromNuget(techs)
Expand Down Expand Up @@ -830,6 +837,7 @@ func (ca *CurationAuditCommand) doCurateAudit(results map[string]*CurationReport
return err
}
techs := ca.techsToAudit()
ca.detectedTechCount += len(techs)
for _, tech := range techs {
supportedFunc, ok := supportedTech[techutils.Technology(tech)]
if !ok {
Expand Down
71 changes: 70 additions & 1 deletion commands/curation/curationaudit_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2266,7 +2266,8 @@ func TestRunGeneratesAuditIdForHttpDetails(t *testing.T) {
defer func() { require.NoError(t, os.Chdir(origWd)) }()

ca := NewCurationAuditCommand().SetWorkingDirs([]string{curWd})
require.NoError(t, ca.Run())
// An empty directory audits nothing, so Run fails; the audit ID is still generated.
require.ErrorContains(t, ca.Run(), "nothing was audited")

require.NotEmpty(t, ca.auditId)
_, parseErr := uuid.Parse(ca.auditId)
Expand Down Expand Up @@ -4759,6 +4760,74 @@ func TestPureUvProjectNotReportedAsPip(t *testing.T) {
assert.NotContains(t, techs, techutils.Pip.String(), "must not also report pip for a plain uv-only project")
}

// TestTechsToAuditRecognisesCustomRequirementsFile: jf ca must detect pip from the file
// named by --requirements-file, the same way jf audit does.
func TestTechsToAuditRecognisesCustomRequirementsFile(t *testing.T) {
// Isolate the home dir so a machine-global ~/.config/uv/uv.toml cannot promote pip to uv.
t.Setenv("HOME", t.TempDir())
t.Setenv("USERPROFILE", t.TempDir())
projectDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(projectDir, "constraints.txt"), []byte("requests==2.31.0\n"), 0644))
t.Chdir(projectDir)

ca := NewCurationAuditCommand()
ca.SetPipRequirementsFile("constraints.txt")
ca.OriginPath = projectDir

assert.Contains(t, ca.techsToAudit(), techutils.Pip.String(), "the file named by --requirements-file must be audited as pip")
}

// TestCurationAuditFailsWhenNothingIsAudited: a run that recognises no package manager
// must fail, so a CI gate cannot pass with nothing checked.
func TestCurationAuditFailsWhenNothingIsAudited(t *testing.T) {
t.Setenv("HOME", t.TempDir())
t.Setenv("USERPROFILE", t.TempDir())
projectDir := t.TempDir()
t.Chdir(projectDir)

ca := NewCurationAuditCommand()
ca.SetWorkingDirs([]string{projectDir})

err := ca.Run()

require.Error(t, err, "a run that audits nothing must not succeed")
assert.Contains(t, err.Error(), "nothing was audited")
}

// TestRunSucceedsWhenOneWorkingDirIsAudited: a run with one audited directory and one empty
// directory must still succeed; only a run that audits nothing fails.
func TestRunSucceedsWhenOneWorkingDirIsAudited(t *testing.T) {
var tt testCase
for _, candidate := range getTestCasesForDoCurationAudit() {
if candidate.name == "python tree - one blocked package" {
tt = candidate
}
}
require.NotEmpty(t, tt.name, "the pip curation case must exist")
basePathToTests, err := filepath.Abs(TestDataDir)
require.NoError(t, err)

cleanUpFlags := setCurationFlagsForTest(t)
defer cleanUpFlags()
mockServer, serverDetails := curationServer(t, tt.expectedBuildRequest, tt.expectedRequest, tt.requestToFail, tt.requestToError, tt.serveResources)
defer mockServer.Close()
cleanUp := createCurationTestEnv(t, basePathToTests, tt, serverDetails)
defer cleanUp()
require.NoError(t, os.Mkdir("empty-dir", 0755))

ca := NewCurationAuditCommand()
ca.SetIsCurationCmd(true)
ca.parallelRequests = 3
ca.SetInsecureTls(tt.allowInsecureTls)
ca.SetIgnoreConfigFile(tt.shouldIgnoreConfigFile)
ca.SetWorkingDirs([]string{".", "empty-dir"})
if defaultServer, confErr := config.GetDefaultServerConf(); confErr == nil {
ca.SetServerDetails(defaultServer)
}

assert.NoError(t, ca.Run(), "a run where one directory is audited must succeed")
}

// TestTechsToAuditQueuesPep723HintForDeferredLogging: the PEP 723 hint must be queued in pendingWarnings.
func TestTechsToAuditQueuesPep723HintForDeferredLogging(t *testing.T) {
projectDir := t.TempDir()
Expand Down
20 changes: 16 additions & 4 deletions utils/techutils/techutils.go
Original file line number Diff line number Diff line change
Expand Up @@ -476,29 +476,41 @@ func DetectedTechnologiesList() (technologies []string) {
return detectedTechnologiesListInPath(wd, false)
}

// DetectedTechnologiesListForCurationAudit is the curation-audit variant of
// DetectedTechnologiesListForCurationAudit is DetectedTechnologiesListForCurationAuditWithRequirementsFile
// with no requirements file.
func DetectedTechnologiesListForCurationAudit() (technologies []string) {
return DetectedTechnologiesListForCurationAuditWithRequirementsFile("")
}

// DetectedTechnologiesListForCurationAuditWithRequirementsFile is the curation-audit variant of
// DetectedTechnologiesList that additionally re-routes any working directory
// detected as Npm to Yarn whenever a parent directory is a yarn workspace
// root that claims it. The promotion is curation-only by design — the
// generic file-based detector intentionally does not walk upward, and other
// commands ('jf audit', 'jf scan', etc.) keep the legacy npm-fallback
// behaviour for yarn workspace members so this change cannot regress them.
//
// pipRequirementsFile is the --requirements-file value; it is matched as jf audit matches it.
//
// Unlike DetectedTechnologiesList, this does not log the result: the caller still has
// more promotions to apply (pnpm/yarn, pip→uv) before the list is final, and logging
// early could show a technology immediately superseded (e.g. "Detected: pip." right
// before "...treating as uv.").
//
// Used by doCurateAudit so 'jf ca --working-dirs=<workspace member>'
// Used by techsToAudit so 'jf ca --working-dirs=<workspace member>'
// resolves through yarn (matching how the project's lockfile was produced)
// instead of npm (which would synthesise a different dependency set and
// produce curation answers that don't match what yarn would have resolved).
func DetectedTechnologiesListForCurationAudit() (technologies []string) {
func DetectedTechnologiesListForCurationAuditWithRequirementsFile(pipRequirementsFile string) (technologies []string) {
wd, err := os.Getwd()
if errorutils.CheckError(err) != nil {
return
}
detected, err := DetectTechnologiesDescriptors(wd, false, []string{}, map[Technology][]string{}, "")
requestedDescriptors := map[Technology][]string{}
if pipRequirementsFile != "" {
requestedDescriptors[Pip] = []string{pipRequirementsFile}
}
detected, err := DetectTechnologiesDescriptors(wd, false, []string{}, requestedDescriptors, "")
if err != nil {
return
}
Expand Down
32 changes: 32 additions & 0 deletions utils/techutils/techutils_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1278,6 +1278,38 @@ func TestPep723Hint_UvSibling_HintFires(t *testing.T) {
assert.True(t, found, "the sibling script's inline deps are out of scope for the project's own audit")
}

// TestDetectedTechnologiesListForCurationAuditWithRequirementsFile: a custom-named file
// named by the requirements flag must make the directory pip, as jf audit does.
func TestDetectedTechnologiesListForCurationAuditWithRequirementsFile(t *testing.T) {
root := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(root, "constraints.txt"), []byte("requests==2.31.0\n"), 0644))

prevWd, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(root))
defer clientTests.ChangeDirAndAssert(t, prevWd)

techs := DetectedTechnologiesListForCurationAuditWithRequirementsFile("constraints.txt")

assert.Contains(t, techs, Pip.String(), "the file named by --requirements-file must make the directory pip")
}

// TestDetectedTechnologiesListForCurationAuditWithoutRequirementsFile: without the flag a
// custom-named file is not a pip indicator, so nothing is detected.
func TestDetectedTechnologiesListForCurationAuditWithoutRequirementsFile(t *testing.T) {
root := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(root, "constraints.txt"), []byte("requests==2.31.0\n"), 0644))

prevWd, err := os.Getwd()
require.NoError(t, err)
require.NoError(t, os.Chdir(root))
defer clientTests.ChangeDirAndAssert(t, prevWd)

techs := DetectedTechnologiesListForCurationAuditWithRequirementsFile("")

assert.Empty(t, techs, "a custom-named file must not be detected as pip when the flag is unset")
}

func TestHasPep723ScriptMetadata(t *testing.T) {
tests := []struct {
name string
Expand Down
Loading