Repository navigation
FEAT: Rebuild saved registry entries in dependency order - #3908
Open
varunj-msft wants to merge 44 commits into
Open
varunj-msft wants to merge 44 commits into
varunj-msft wants to merge 44 commits into
Conversation
…API input External construction now rejects a parameter whose type includes Path unless the calling service passes the name in allowed_path_params after making the value safe. The converter service allows its upload parameters once they are stored, and the target service allows the server-set upload folder and the GitHubCopilotTarget working directory. The scorer-only check is removed, and the "Valid parameters" list leaves out path parameters callers cannot set.
The target catalog now applies the path rule to required parameters too, so a target that needs a disallowed path is left out instead of listed without the field, matching the scorer catalog. allowed_path_params is typed frozenset[str] so a bare string cannot turn the allow-list into a substring match.
The target catalog still lists a target whose required path parameter is its server-set upload folder when the server has one, and leaves it out otherwise.
# Conflicts: # tests/unit/backend/test_converter_service.py
…reports an unknown attack type
…s that can't be checked
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the dependency-ordered registry restoration changes.
No open issues 🎉
What changed in this PR
Adds dependency-ordered rebuilding of saved PyRIT registry entries, building on the storage work in #3891 and #3892. Startup and apply integration remain deferred to the next PR.
Changes:
- Restores targets, converters, scorers, and techniques after their saved dependencies, reporting missing entries, failures, and dependency loops.
- Applies registry input checks and credential restrictions, with sanitized error messages and a structured restore report.
- Includes the stacked recipe models, issue-aware storage listing, and tests for restoration, validation, and storage failures.
| File | Description |
|---|---|
tests/unit/registry/test_registry_recipe_storage.py |
Tests recipe parsing, versions, storage sources, and report serialization. |
tests/unit/registry/test_registry_recipe_restore.py |
Tests dependency ordering, rebuild checks, failures, and message sanitization. |
tests/unit/registry/test_file_document_storage.py |
Tests local and Blob listing issues, file replacement races, and safe errors. |
pyrit/registry/registry_recipe_storage.py |
Reads and validates versioned recipes from per-registry folders. |
pyrit/registry/registry_recipe_restore.py |
Plans dependencies, rebuilds entries, and reports restore outcomes. |
pyrit/registry/file_document_storage.py |
Adds issue-aware listings without changing legacy listing behavior. |
pyrit/models/request_limits.py |
Adds the shared text-length limit for technique descriptions. |
pyrit/models/catalog/registry_recipe.py |
Defines recipe, reference, failure, and restore-report models. |
pyrit/models/catalog/__init__.py |
Exposes the new models through lazy exports. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is one of the changes for saving Registry entries across restarts. It's the third of four stacked PRs and builds on #3892. Until #3891 and #3892 merge, the diff here also shows their changes; this PR's own code is
pyrit/registry/registry_recipe_restore.pyandRegistryRestoreReport, with their tests.Problem. Nothing turns saved recipes back into registry entries.
Change.
RegistryRecipeRestorer:read()reads every folder without building anything, so a caller on an event loop can run it in a worker thread;restore()then builds on the calling thread, as the target API does, because a constructor can need the running loop.restore():__module__, and looks a technique's attack type up in the attack registry. It never imports a module a file names.Parameter.referenceand the technique fields. A name resolves to a saved entry first (even a broken one), then to a startup entry. Missing and failed dependencies and loops are reported with the entry each one waits on.create_named_instance(..., external_input=True), so a server path is refused exactly as for API input (FIX: Reject path parameters by default when building components from API input #3594): a target may set only the path parameters its class lets callers set (caller_path_parameters, such asGitHubCopilotTarget'sworking_directory), and converters and scorers none. Techniques go throughcreate_factory(..., strict_json=True)andregister_runtime, as the technique API does. A test checks that technique recipes and the create request accept and reject the same documents and have the same defaults.upload_directory_parameter), which restore can't set yet.?or#, where a URL's query or fragment starts, and says the rest is hidden. A message with an@, as in a URL's user name and password, is hidden entirely. Look-alike characters such as a full-width@count as the real ones, and the text that is shown keeps its original characters.RegistryRestoreReport: restored entries, failures (kind, name, reason, message, version, dependency) and ignored files.Not in this PR.
Tests and Documentation
tests/unit/registry/test_registry_recipe_restore.py(89 tests): order across the four kinds (including converters in either of a technique's lists), loops, missing and failed dependencies, clashes with startup entries,read()building nothing andrestore()using what it returned without reading again, a startup entry found by its exact name, stored credentials, server paths refused as the API refuses them (a target's declared caller path is accepted, and only for targets), class lookup without imports, unknown classes and attack types, technique recipes matching the create request, and failure messages (queries, fragments, user info, the exact message for every form of?,#and@, original characters kept, long messages checked one character at a time).pytest tests/unit/registry tests/unit/models -n 4: 3,935 passed, 5 skipped, 4 xfailedpre-commit run --from-ref origin/main --to-ref HEAD: passed