Skip to content

test(testkit): add cleanup hooks that run after pass, failure, or panic #5218

Closed

Description

@matthewgrossman

User Story

I'm moving OpenShell's driver feature suites onto testkit (#5137). Those tests create things testkit doesn't track, such as providers, profiles, and container images. I need cleanup for them that runs whether the test passes, fails, or panics, without each suite writing its own cleanup wrapper.

Description

Add cleanup hooks to testkit, as proposed in the #5137 review:

  • cx.on_cleanup(name, async move || ...) registers a hook through &self.
  • Hooks run after the test body, last registered first, whether the body passes, fails, or panics. Each hook is limited to CLEANUP_TIMEOUT and its panics are caught.
  • A hook that fails, panics, or times out is reported, and the remaining hooks still run. It fails a test whose body passed.
  • Sandbox tracking becomes the first user. Creating or tracking a sandbox registers a hook that deletes it and waits until it's gone, keyed by sandbox name. Tracking and then adopting a sandbox registers one hook, and Sandbox::delete removes it.
  • Hooks run one at a time. Running adjacent sandbox deletes together can come later if needed.
  • The Keycloak provider-refresh suite registers its provider and profile deletes right after creating them.

Context

The Keycloak suite deletes its provider and profile inline after the body, so a panic skips those deletes and a failed delete is only logged. The OCI image suite in #5137 has its own run wrapper that deletes sandboxes before images for the same reason. #5137 will rebase onto this change and move that suite to hooks.

Definition of Done

  • cx.on_cleanup and cx.cli_handle are available and documented in tests/testkit/README.md.
  • Unit tests cover hook order, failing, panicking, and timed-out hooks, and a body that panics after registering a hook that writes a marker: the test fails with the body's panic and the marker is set.
  • Testkit sandbox cleanup runs through the hooks.
  • The Keycloak provider-refresh test uses hooks instead of inline cleanup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions