Repository navigation
Conversation
A Call through an object, e.g. `q.memcpy(...)`, keeps its callee only as a string, so Uxreplace could not rename `q`. reuse_efuncs then saw two Callables differing only in that object as distinct, and kept both. Keep the callee object on the Call and let Uxreplace substitute into it.
The Intel GPU branch of get_gpu_info installed memory callbacks that always returned None, so Device.memavail() was None on Intel GPUs and anything budgeting device memory (e.g. DevitoPRO's layered snapshotting) could not work. Read `memory_physical_size_byte` and `memory_free_size_byte` from `xpu-smi discovery -d <id> -j`, and keep returning None if xpu-smi is unavailable or fails. The callbacks take a physical device ID, as with nvidia-smi, so treat ZE_AFFINITY_MASK as a visible-devices variable when mapping an MPI rank to its device, and run xpu-smi without it, since xpu-smi otherwise numbers only the devices the mask leaves visible.
The `graphics/ubuntu jammy unified` channel stops at compute-runtime 23.17. With it, the oneAPI 2026 compilers find no Level Zero device on a Data Center GPU Max 1100 and OpenMP offload images fail to load. Install the Max-series user-space stack (compute-runtime 25.18, IGC 2.11, Level Zero loader 1.21+) from `gpu/ubuntu jammy/lts/2523`, under its current package names, and add a `gpu-icx` target for OpenMP offloading to Intel GPUs.
With oneAPI 2026.0 on a Data Center GPU Max 1100 the explicit-SIMD device
compilation fails at load time for every Devito Operator: IGC 1.0.15468
(LTS 2350) rejects the image ("SLM variable usage in indirectly called
functions is not supported") and IGC 2.11 (LTS 2523, the newest for
Ubuntu 22.04) crashes. Without the flag, the GPU test files pass.
Add a pytest-gpu-omp-intel job, on self-hosted runners labelled INTELGPU, running the same test files and examples as the AMD OpenMP job. The runner selects its device through ZE_AFFINITY_MASK.
Undo the switch-off of c57bf01 so that the LTS 2523 runtime reaches devitocodes/bases:{cpu-icx,cpu-sycl,gpu-sycl}, and build the new gpu-icx image the Intel GPU job uses.
Intel MPI 2021.19 picks the shm fabric alone on a single node, and shm refuses MPI_Comm_spawn, so the DevitoPRO decoupler tests hung in the rebuilt images. Use shm:ofi.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3095 +/- ##
==========================================
+ Coverage 84.40% 84.43% +0.02%
==========================================
Files 260 260
Lines 58153 58189 +36
Branches 4948 4950 +2
==========================================
+ Hits 49086 49129 +43
+ Misses 8245 8237 -8
- Partials 822 823 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
mloubout
reviewed
Oct 27, 2026
mloubout
left a comment
Contributor
There was a problem hiding this comment.
Overall yes, just one main comment
| self.base = None | ||
| # The called object: a str or, for a call through an object such as | ||
| # `obj.method(...)`, a CallFromPointer | ||
| self.callee = name |
Contributor
There was a problem hiding this comment.
Not sure I understand why this needs to be separate from name
| else: | ||
| arguments.append(uxreplace(i, self.mapper)) | ||
| kwargs = {'arguments': arguments} | ||
| if o.base is not None: |
Contributor
There was a problem hiding this comment.
That looks like a rebuild bug being bypassed here
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes for Devito on Intel GPUs, found while bringing DevitoPRO's SYCL backend back to a green test suite on a Data Center GPU Max 1100 (devitocodespro/devitopro#252), plus a CI job for OpenMP offloading on Intel GPUs (#2370). The commits are kept separate.
1.
compiler: Substitute the object of a method Call in UxreplaceUxreplace.visit_Callsubstituted into a Call's arguments andretobj, never into its callee. For a call through an object (q.memcpy(...), built from aCallFromPointer), the objectqis part of the callee.Call.baseand the constructor arguments kept it, butCall.name(a string, which code generation and signatures use) was never rebuilt.reuse_efuncsrelies onUxreplaceto abstract every symbol of a Callable, so two Callables that differed only in that object kept distinct signatures and were both emitted.No OSS backend emits such Calls; DevitoPRO's SYCL backend does. With two streamed Functions it emitted two copies of each streaming thread function, identical apart from the queue. CUDA, HIP and OpenMP pass the stream as an argument and were never affected.
The fix makes
Call.calleeexpose the callee object, and hasUxreplacerebuild the name only when the callee actually changes. Subclasses that fix their own callee (e.g. aparallel_foron a handler that is never in the mapper) are therefore never rebuilt withname.Test:
tests/test_iet.py::test_reuse_efuncs_call_through_object, modelled ontest_reuse_efuncs_natural_numbering. It is IET-level because no OSS backend emits such a Call. It fails on main and passes with the fix.2.
arch: Query the memory of Intel GPUs through xpu-smiThe Intel branch of
get_gpu_infoinstalledmem.*callbacks that always returnedNone.Device.memavail()was thereforeNoneon Intel GPUs, andnbytes_avail_mapperraisedTypeError: unsupported operand type(s) for -=: 'NoneType' and 'Integer'. In DevitoPRO this broke every snapshotting Operator on an Intel GPU.The callbacks now read
memory_physical_size_byteandmemory_free_size_bytefromxpu-smi discovery -d <id> -j, and still returnNoneif xpu-smi is missing or fails. They take a physical device ID, as with nvidia-smi. To map MPI ranks correctly:ZE_AFFINITY_MASKis now one of the visible-devices variables.None.The old comment about "weird errors in docker" did not reproduce with the new images below, run unprivileged.
Tests:
TestGPUInfo::test_memavailandTestDeviceID::test_visible_devices[{ZE_AFFINITY_MASK: 1}]. Both fail on main and pass with the fix on a Max 1100.3. Intel GPU CI (#2370)
docker: Take the Intel GPU runtime from the LTS 2523 channel. The oldgraphics/ubuntu jammy unifiedchannel stops at compute-runtime 23.17, with which oneAPI 2026 finds no Level Zero device on a Max 1100, and OpenMP offload images fail to load. The images now take compute-runtime 25.18 / IGC 2.11 / Level Zero 1.24 fromgpu/ubuntu jammy/lts/2523, with the renamed packages. A newgpu-icxtarget setsDEVITO_PLATFORM=intelgpuX.compiler: Stop passing -fopenmp-target-simd for Intel GPUs. With oneAPI 2026.0 every Operator fails at image load:Without the flag, the job's test files pass.
ci: Run the GPU tests on Intel GPUs with OpenMP offloading. The newpytest-gpu-omp-intelmatrix entry runs the same test files and examples as the AMD OpenMP job, on self-hostedINTELGPUrunners. The runner picks the device viaZE_AFFINITY_MASK.ci: Build the Intel base images again, including gpu-icx. This undoes c57bf01 ("docker: Disable oneapi updates"). That commit stopped rebuilding the Intel bases, which froze them at a runtime (23.17) that cannot drive current Intel GPUs with the current compiler. The new job needs thegpu-icximage, and the runtime fix above only reachescpu-icx/cpu-sycl/gpu-syclthrough a rebuild. If you prefer to keep automatic rebuilds off, this commit can be dropped and the images pushed once by hand fromdocker/Dockerfile.intel.docker: Let Intel MPI spawn processes in the Intel images. Intel MPI 2021.19 uses theshmfabric alone on one node, andshmrefusesMPI_Comm_spawn. DevitoPRO's decoupler tests hung in the rebuilt images untilI_MPI_FABRICS=shm:ofiwas set.Results (ese-ponte: 2x Data Center GPU Max 1100, oneAPI 2026.0.0)
gpu-icxtarget, using the job's own commands:mpiexec -n 2examples: 46 passed.-m "not parallel"): C 3706 passed, CXX 3706 passed, openmp 3709 passed, 0 failed (gcc).pre-commit runon the touched files (isort, ruff, flake8, typos, actionlint, hadolint): passes.Before the Intel job can run
INTELGPUmust be registered. None exists yet. Both are deferred by the owner, so the new job will queue until then.Fixes #2370
Related: devitocodespro/devitopro#252 (pins this branch).
🤖 Generated with Claude Code