Skip to content

FEAT: Read saved registry recipes - #3892

Open
varunj-msft wants to merge 15 commits into
microsoft:mainfrom
varunj-msft:varunj-msft/13178-Read-Registry-Recipes
Open

varunj-msft wants to merge 15 commits into
microsoft:mainfrom
varunj-msft:varunj-msft/13178-Read-Registry-Recipes

Conversation

@varunj-msft

@varunj-msft varunj-msft commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Description

This is one of the changes for saving Registry entries across restarts. It's the second of four stacked PRs and builds on #3891. Until #3891 merges, the diff here also shows its changes.

Problem. There's no file format or storage for a saved target, converter, scorer or technique.

Change.

  • pyrit/models/catalog/registry_recipe.py: the recipe models, exported from pyrit.models.catalog.
    • A target, converter or scorer recipe holds schema_version, class_module, class_name and params.
    • A technique recipe holds the fields of the technique create request, without the name, with the same limits and tag rules. The library can't import the backend, so they're copied, and the next PR adds a test that the two models accept and reject the same documents.
    • Unknown fields are rejected, so a file with an extra field such as credentials is reported, not silently read.
    • Also adds the failure and reference types.
  • pyrit/models/request_limits.py: adds MAX_TEXT_LENGTH for the technique description. FIX: Apply the request limits to request fields added since the last limits PR #3595 adds the same line, so the two merge cleanly; until it lands, backend/models/common.py keeps its own copy of the same value.
  • pyrit/registry/registry_recipe_storage.py: RegistryRecipeStorage, a FileDocumentStorage with one folder per registry (targets/, converters/, scorers/, techniques/) under one source.
    • On Blob the folder goes on the URL path, so a SAS token stays in the query.
    • list_recipes() uses the listing from the previous PR. It returns the parsed recipes with their versions (the SHA-256 of the bytes), a failure for each unreadable, invalid or newer-format file, and the ignored files.
    • The format version is checked first, so a file from a newer PyRIT says so instead of failing on a field it doesn't know.
  • Report text that isn't valid UTF-8 is escaped in backslash form, such as a local file name with invalid bytes or an entry name written with a \udcff escape inside a recipe, so the report can always be sent as JSON. Valid text is unchanged.
  • The file name is the entry name, so only names that fit the document rule (up to 64 lowercase letters, digits and underscores, starting with a letter) can be saved. The PRs that save entries will refuse other names.

What stays the same. Nothing calls this yet; the next PR rebuilds entries from these recipes. No API, config or behavior change.

Tests and Documentation

  • New tests/unit/registry/test_registry_recipe_storage.py: recipes and their versions, field limits and unknown fields, failures for unreadable, invalid and newer-format files, ignored names, Blob folders that keep a SAS token in the query, and report text that isn't UTF-8 (escaped only when sent as JSON; models keep the text they were given; valid text unchanged).
  • pytest tests/unit/registry tests/unit/models -n 4: 3,459 passed
  • pre-commit run --from-ref origin/main --to-ref HEAD: passed
  • No doc change: nothing calls this yet. JupyText: not run (no notebooks changed).

Copilot AI balanced review requested due to automatic review settings November 3, 2026 17:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues require changes to the registry recipe models or storage reader.

No open issues 🎉

What changed in this PR

Adds the models and storage reader needed to preserve registry entries across restarts. This second stacked PR builds on #3891’s document-listing support; entry rebuilding and startup restoration remain for later PRs.

Changes:

  • Defines versioned component and technique recipes, plus restore-report types with JSON-safe text.
  • Reads recipes from per-registry local or Azure Blob folders, preserving SAS queries and reporting invalid, unreadable, newer-format, and ignored files.
  • Adds unit coverage for parsing, versions, validation, storage failures, and report serialization without changing existing callers.
File Description
tests/​unit/​registry/​test_registry_recipe_storage.py Tests recipe loading, validation, versions, Blob paths, and report serialization.
tests/​unit/​registry/​test_file_document_storage.py Tests issue-aware local and Blob listings and safe error messages.
pyrit/​registry/​registry_recipe_storage.py Adds per-registry storage, recipe parsing, and failure reporting.
pyrit/​registry/​file_document_storage.py Adds issue-aware listings while retaining legacy listing behavior.
pyrit/​models/​request_limits.py Adds the shared technique-description text limit.
pyrit/​models/​catalog/​registry_recipe.py Defines recipe schemas, references, and restore-report models.
pyrit/​models/​catalog/​__init__.py Exposes the new models through lazy exports.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants