From 25873d9027cb0888a2375f285f144ee8e4f5ef90 Mon Sep 17 00:00:00 2001 From: Nikita Bugrovsky Date: Sun, 22 Nov 2026 13:59:42 +0200 Subject: [PATCH] docs: add known issue for the default driver container on OpenShift 5.0 Newly installed OpenShift Container Platform 5.0 clusters use RHCOS 10 nodes. The GPU Operator bundle pins the default driver container image by digest to a RHEL 9 based image, and the Operator uses a digest-pinned image without per-node operating system tag resolution. The driver daemon set therefore fails on RHCOS 10 nodes. Clusters upgraded from OpenShift 4.x keep RHCOS 9 nodes and are not affected. Document the workaround of setting spec.driver.version in the cluster policy, either to a floating driver version or to the RHEL 10 driver image digest, and point the cluster policy creation steps at the known issue. Signed-off-by: Nikita Bugrovsky --- openshift/install-gpu-ocp.rst | 5 ++++ openshift/known-issues.rst | 48 ++++++++++++++++++++++++++++++++++- 2 files changed, 52 insertions(+), 1 deletion(-) diff --git a/openshift/install-gpu-ocp.rst b/openshift/install-gpu-ocp.rst index 8b04295f4..bf2fcc592 100644 --- a/openshift/install-gpu-ocp.rst +++ b/openshift/install-gpu-ocp.rst @@ -193,6 +193,11 @@ When you install the **NVIDIA GPU Operator** in the OpenShift Container Platform .. note:: If you create a ClusterPolicy that contains an empty specification such as ``spec{}``, the ClusterPolicy fails to deploy. +You must set the driver version when you create the cluster policy on a newly installed OpenShift Container Platform 5.0 cluster. +Newly installed clusters use RHCOS 10 nodes, and the default driver image does not work on these nodes. +A cluster that you upgraded from OpenShift 4.x keeps RHCOS 9 nodes and is not affected. +Refer to :ref:`openshift-known-issues-ocp5-default-driver`. + As a cluster administrator, you can create a ClusterPolicy using the OpenShift Container Platform CLI or the web console. Also, these steps differ when using **NVIDIA vGPU**. Refer to the appropriate sections below. diff --git a/openshift/known-issues.rst b/openshift/known-issues.rst index 41d733920..e41addf76 100644 --- a/openshift/known-issues.rst +++ b/openshift/known-issues.rst @@ -1,4 +1,4 @@ -.. Date: November 18 2026 +.. Date: November 22 2026 .. Author: nbugrovs .. headings are # * - = @@ -118,3 +118,49 @@ To check an image that fails to mirror, use ``skopeo`` to inspect the image mani The Docker Image Manifest V2, Schema 2 specification defines no uncompressed layer media type, so registries reject it. To report an affected image that this page does not list, add the image reference and the ``oc-mirror`` error output to `NVIDIA/gpu-operator#3166 `_. + +.. _openshift-known-issues-ocp5-default-driver: + +********************************************************************************************* +Default driver container fails on newly installed OpenShift Container Platform 5.0 clusters +********************************************************************************************* + +When you install the GPU Operator with the default cluster policy on a newly installed OpenShift Container Platform 5.0 cluster, the ``nvidia-driver-daemonset`` pods fail on the GPU nodes. + +Newly installed clusters use RHCOS 10 nodes. A cluster that you upgraded from OpenShift 4.x keeps RHCOS 9 nodes and is not affected. + +The GPU Operator bundle pins the default driver container image by digest. The image is based on RHEL 9. The Operator uses a digest-pinned driver image as-is, without per-node operating system tag resolution. The RHEL 9 driver container therefore runs on RHCOS 10 nodes and fails to load kernel modules. + +Specify the driver version in the cluster policy +================================================ + +#. Set ``spec.driver.version`` in the cluster policy to a floating driver version, such as ``${recommended}``. The Operator resolves the operating-system tag for each node and deploys ``${recommended}-rhel10`` on RHCOS 10 nodes: + + .. code-block:: yaml + + driver: + repository: nvcr.io/nvidia + image: driver + version: "${recommended}" + + Alternatively, set the field to the RHEL 10 driver image digest for driver version ``${recommended}``: + + .. code-block:: yaml + + driver: + repository: nvcr.io/nvidia + image: driver + version: "sha256:52b984c00817b08dcf5a0ed50dbba7b9f4d779614bde486a3483a4fb61dea3e5" + +#. Verify that the driver daemon set uses the RHEL 10 image: + + .. code-block:: console + + $ oc -n nvidia-gpu-operator get ds -o custom-columns=NAME:.metadata.name,IMAGE:.spec.template.spec.containers[0].image + + *Partial Output* + + .. code-block:: console + + NAME IMAGE + nvidia-driver-daemonset-10.2.20261015-0 nvcr.io/nvidia/driver:${recommended}-rhel10