Skip to content

docs: add known issues/mirroring manifest - #540

Merged
mikemckiernan merged 1 commit into
NVIDIA:mainfrom
nikitabugrovsky:nbugrovs/known-issues-mirroring-manifest
Nov 19, 2026
Merged

mikemckiernan merged 1 commit into
NVIDIA:mainfrom
nikitabugrovsky:nbugrovs/known-issues-mirroring-manifest

Conversation

@nikitabugrovsky

Copy link
Copy Markdown
Contributor

Add a new Known Issues page under openshift/ describing the container image manifest inconsistencies (uncompressed layers, and OCI index/Docker Schema2 child manifest mismatches) that can cause oc-mirror to fail mirroring some GPU Operator related images to registries such as Quay.

Includes commands to detect affected images, the current fix status as of GPU Operator v26.7.3, and the workaround for images that are still pending a corrected replacement.

Cross-link the new page from the disconnected mirroring guide and the troubleshooting page.

Refs: NVIDIA/gpu-operator#3166

@nikitabugrovsky

Copy link
Copy Markdown
Contributor Author

@mikemckiernan hi! Could you kindly take a look?

@mikemckiernan

Copy link
Copy Markdown
Member

Thanks very much for this PR. I like most of it, but would like a handful of changes. I proposed the changes in a PR to your branch: nikitabugrovsky#1.

Comment thread openshift/known-issues.rst Resolved
@nikitabugrovsky
nikitabugrovsky force-pushed the nbugrovs/known-issues-mirroring-manifest branch 2 times, most recently from e6af0af to 03d2c16 Compare November 19, 2026 16:57
@nikitabugrovsky

Copy link
Copy Markdown
Contributor Author

@mikemckiernan @rahulait could you please take another look?

@rahulait rahulait left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nikitabugrovsky , LGTM.

@mikemckiernan can we run the CI and verify everything looks fine before merging?

@mikemckiernan mikemckiernan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you again! I hope you don't mind asking your agent to review my comments and suggestions.

Comment thread openshift/known-issues.rst Resolved
Comment thread openshift/known-issues.rst Resolved
Add a Known Issues page under openshift/ documenting that oc-mirror
fails to mirror two GPU Operator bundle images to Red Hat Quay:
dra-driver-nvidia-gpu:v0.5.1 and the gdrdrv image pinned in the
bundle. Both fail because of container image manifest defects, not
because digest preservation is merely inconvenient.

Findings, confirmed by direct skopeo and oc-mirror testing against
live Red Hat Quay 3.16.2 and 3.17.3 instances:

* gdrdrv's OCI image index references Docker Schema2 child
  manifests. Red Hat Quay rejects this combination before 3.17, and
  accepts it unchanged on 3.17 and later. The Red Hat Quay 3.17
  release notes document this as a fixed limitation.
* dra-driver-nvidia-gpu uses the Docker Schema2 manifest format with
  an uncompressed layer. The Docker Schema2 specification defines no
  uncompressed layer media type, so every Red Hat Quay version
  rejects it. Clusters that use the DRA Driver for NVIDIA GPUs have
  no workaround.
* The default driver manager image, k8s-driver-manager:v0.12.1, also
  has an uncompressed layer, but its OCI manifest format permits
  uncompressed layers, so it mirrors to Red Hat Quay unchanged.
  Confirmed with a digest-identical mirror to a live Quay 3.16.2
  instance. This image is excluded from the blockedImages workaround
  on this page.

Includes an ImageSetConfiguration blockedImages procedure for the
two affected images, a tip to drop the gdrdrv block on Red Hat Quay
3.17 and later, and skopeo commands to check any image for the same
class of manifest problem.

Cross-link the new page from the disconnected mirroring guide and
the troubleshooting page.

Refs: NVIDIA/gpu-operator#3166

Signed-off-by: Nikita Bugrovsky <nbugrovs@redhat.com>
@nikitabugrovsky
nikitabugrovsky force-pushed the nbugrovs/known-issues-mirroring-manifest branch from 03d2c16 to eadcc7e Compare November 19, 2026 17:56
@github-actions

Copy link
Copy Markdown

Documentation preview

https://nvidia.github.io/cloud-native-docs/review/pr-540

@mikemckiernan
mikemckiernan merged commit 64ea876 into NVIDIA:main Nov 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants