Repository navigation
fix(cli): stop before any model call when a locale's persona dataset is missing - #60
Merged
Merged
Conversation
…is missing simulate, panel and --materialize-inputs check each locale's Nemotron-Personas dataset where Data Designer reads it, and name the data-designer download personas command that fetches a missing one, or the locales Data Designer provides. Dry runs report the same and still succeed; --inputs replays stored personas and needs no dataset. Signed-off-by: Yev Meyer <ymeyer@nvidia.com>
…s missing Signed-off-by: Yev Meyer <ymeyer@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
Persona datasets are a one-time download per locale. When one was missing,
simulateandpanelfailed inside Data Designer with a file-pattern traceback, and a livesimulategot there only after its model health checks. They now stop before any model call and say what to do.simulate,panelandsimulate --materialize-inputscheck each locale's dataset in the folder Data Designer's sampler reads, including whenDATA_DESIGNER_MANAGED_ASSETS_PATHorDATA_DESIGNER_HOMEmoves it. A missing dataset raises aConfigError(exit 2) that names the expected file and gives onedata-designer download personas --locale …command for all of them, with the NGC steps. India language variants map to theen_INdataset, and a locale Data Designer has no dataset for gets the list of locales it provides.simulate --dry-runandpanel --dry-runadd apersonas:line,readyor the same report, and still exit 0, since the offline check comes before the download.usersim smokeadds the download step to its next-step hint when theen_USdataset is missing.simulate --inputsreplays stored personas and skips the check.architecture/cli.mddescribes the check.How it was verified
make check-all: lint clean, 333 files already formatted.make test: 4083 passed. New:tests/test_cli_persona_datasets.py(12 tests) and one test intests/test_smoke.py, on the suite'sen_US-only persona fixture. Each fails without the change.make smoke,make check-license-headers,make check-doc-links,make check-notebooks-cleanandmake check-distpass.usersim simulate --locale en_US --locale ta_Taml_IN --locale en_GB --num-rows 1 --out oexits 2 before any model call. It namesen_USanden_INwithdata-designer download personas --locale en_US --locale en_IN, and lists the locales Data Designer provides foren_GB. The same command with--dry-runreports this and exits 0.