Adhere to Rust floating-point semantics #643

Floating-point behavior is currently determined by target defaults, which differ from Rust's and from each other. Vulkan 1.1-1.4 added features that support configuration of float behavior to match Rust's, and I'd like to use those to bring GPU float behavior close to CPU. This will make the rust-gpu Vulkan target viable for scientific computing.

WGSL does not have any float control mechanism yet, so it will have to wait. For CUDA, the defaults already stick close to Rust's semantics, although specific compilers targeting CUDA may perform inconsistent reordering optimizations like automatic parallelization of sums and products.

The scope of behavior that this spans is: algebraic reordering, automatic FMA fusing, rounding, subnormal flushing, and Inf/NaN assumptions.

Here's my proposed solution, which I've implemented in 2 stacked branches:

1. Add float control policies
2. Carry float controls through linking and vectorization

Example use-cases that do not work reliably now, but would work after this update:

• Geometric predicates (incircle test, etc)
  • Deliberate accounting for floating-point roundoff is ubiquitous in computational geometry; most meshing software can't run with float reordering enabled
• Full-precision accumulation procedures like Two-Sum and Two-Prod and improved-accuracy methods like Pairwise Sum require rigorously consistent operation ordering and roundoff behavior to function at all
• Compensated arithmetic is required in solvers and integrators for stiff/ill-conditioned systems, for example, in control systems and signal processing

Related ticket with narrower scope: #229

Proposal

Floating-point policies

A per-entrypoint annotation in [rust_math, fast_math, compat_math] determines float controls flags.
All targets maintain un-annotated defaults (compat_math) for maximum compatibility.

Usage like #[spirv(compute(threads(1), rust_math))] and so on.

Constraints

• Vulkan requires roundoff behavior and subnormal handling are configured per-entrypoint (similar to CPU environments)
  • This means nested kernels inherit the parent kernel's settings, even if they aren't the right ones
  • Ideally, each kernel would be able to control its own execution requirements, but that's not supported by APIs (and probably not supported by hardware)
  • This is somewhat achievable in the same way as in Rust on CPU: use rust_math strict mode as the blanket setting, and use explicit algebraic operations to opt out locally
• Feature requirements are module-wide
  • Must use separate output modules to deploy compat_math entry points independently of rust_math or fast_math entry points
• Runners must enable shaderFloatControls2 and support controls for every floating-point width used
• Not all targets support enough/any float controls to match Rust
  • SPIR-V>=1.4 supports adequate float controls, but not all downstream targets do
  • Vulkan 1.1–1.3 requires VK_KHR_shader_float_controls2; the interface is core in Vulkan 1.4
  • WGSL does not expose any float controls at all
    • Upstream extension requires changes to WGSL spec
  • WGPU naga does not support float controls even if targeting SPIR-V output
    • Upstream extension is possible but would not cover all output targets
• Vector operations can only have one set of flags
  • When scalar operations being combined into a vector operation have different flags, the stricter set is used
  • This avoids either (1) rejecting vectorization of scalar ops with different flags or (2) producing a less correct result from vector ops than from scalar ops
• These settings do not guarantee that the hardware has properly implemented IEEE-754 or that the drivers will comply with the flags that have been set

Details

rust_math and fast_math emit control flags that map to the Vulkan FloatControls2 API.
While this is technically a Vulkan-specific pattern, it matches closely to historical CPU flags, and can be expected to map to other float control APIs reasonably well.

rust_math is the Vulkan default. It reflects Rust's strict floating-point semantics.
fast_math is a qualitative environment. Nothing in particular can be said about its validity except that it looks fine sometimes.

compat_math emits no flags, maximizing compatibility but leaving float semantics to be determined by the target's default.
Targets' default behavior varies widely; WGSL's default is even less strict than fast_math, while CUDA is similar to
rust_math, and Vulkan's un-controlled default falls in the middle.
Hardware and firmware introduce further variance in defaults and feature support.

Target compatibility

References

Vulkan floating-point rules.
Vulkan extension requirements
WGSL floating-point rules