SIMD paths and dispatch

Contiguous dot, sum, nrm2 (via dot), axpy, scale, mul_assign, and div_assign_floor use one reduction shape. dot keeps eight partials, p0 through p7. Each product is one mul_add into its partial, then (p0+p4), (p1+p5), (p2+p6), (p3+p7), then one mul_add per tail element. axpy is a.mul_add(x, y) per element. sum uses the same pair tree with plain adds. The portable clone is that mul_add loop with no ISA feature enabled. Hardware FMA rounds once, and so does f64::mul_add, so the portable clone is the bit reference.

The simd feature compiles those loops with multiversion 0.9. The clones are x86_64+avx512f+fma (rustc >= 1.89, build.rs cfg rgmin_avx512), x86_64+avx2+fma, aarch64+sve, aarch64+neon, and the default clone. Code outside the clones stays on the compiler default, x86-64 and armv8-a. The build does not pass -C target-cpu or -march. Selection is runtime. A CPU without a clone runs the default clone. It does not execute that clone’s instructions.

RGMIN_SIMD is read once, in the OnceLock that caches the override. Accepted names are scalar, avx2, avx512, neon, and sve. An unknown name is ignored. A known name this CPU cannot run selects the portable path. The AVX2 clone requires FMA. A CPU with AVX2 and no FMA stays on mul_add. A thread-local test override is read on the calling thread before a rayon fan-out. Workers receive the path that was chosen there. They do not call getenv.

The par feature keeps the same tree. The chunk length is 4096, a multiple of 8, and does not depend on the pool size. Each contiguous chunk is folded by the active kernel, and those totals are added from the first index upward. Element updates use the same chunk length. Lengths below 65536 stay serial. Thread count does not change a bit. dot_ordered is a separate left fold of unfused products for noise projections. It is not the eight-partial tree and it is not dispatched.

The same input, including -0.0, subnormals, 1 ulp values, and NaN payloads, produces the same f64 bits on every clone and at 1, 2, 3, and 8 threads. The SHA-256 of those bytes is tests/golden/isa-hashes.toml. A clone the machine cannot execute is reported as not exercised. The scalar path is always run by forcing it.

Páll and Hess, Comput. Phys. Commun. 184, 2641 (2013), https://doi.org/10.1016/j.cpc.2013.06.003. Dröge et al., Softw. Pract. Exp. 53, 176 (2023), https://doi.org/10.1002/spe.3075.

simd is a default feature. The gate that decided that is in the changelog. Default-on required a Holm-corrected wall-time ratio interval below 1 on the atomistic bench or at n = 3e4, and no micro-bench size slower beyond its interval.