Summary
The negative-score test-data injection in #743 (rows whose post-scale attention scores land at -200..-490 — the regime real models like Qwen2.5 actually reach, and the one behind pytorch/pytorch#193893) surfaces fp8-family mismatches on B200 that classic sparse-int data never exercised:
- fp8 fwd —
test_sdpa_fp8_fwd_L0[test110] fails torch.testing.assert_close on O (an MQA config: h_q=9, h_k=1, s=3770, d_qk=192/d_v=128). 1 of 256 sampled configs.
- fp8 bwd — 3 of 128 sampled configs fail gradient tolerances.
- mxfp8 — 11 fwd + 2 bwd tests fail their exact-match comparison against the mxfp8 emulator (e.g.
test_sdpa_mxfp8_fwd_L0[test213]: "output mismatch: 43 elements differ"), i.e. kernel and reference emulation disagree bit-wise only in the negative-score regime. Needs triage into kernel-vs-emulator: either the kernel's exp/rescale path deviates for large-magnitude scores, or the emulator's modeling does.
Notes
- All observed with the SM100 decode fwd fix already applied, so these are independent of that bug.
- fp16/bf16 forward suites are fully green on the same data, and the fp16/bf16 backward counterpart finding is tracked separately (fp16-rounded score recompute signature).
- Injected values are exactly representable in fp8 e4m3/e5m2 (|k| <= 4, |q| a power of two), so quantization of the inputs themselves is exact; the divergence is do
8BD2
wnstream in the softmax/rescale path.
Repro
Check out #743, point at a cuDNN with the decode fwd fix, then:
pytest test/python/test_mhas_v2.py::test_sdpa_fp8_fwd_L0 -k test110
pytest test/python/test_mhas_v2.py::test_sdpa_mxfp8_fwd_L0 -k "test213 or test214"
Each failing test prints a self-contained test_repro dict.
🤖 Generated with Claude Code
Summary
The negative-score test-data injection in #743 (rows whose post-scale attention scores land at -200..-490 — the regime real models like Qwen2.5 actually reach, and the one behind pytorch/pytorch#193893) surfaces fp8-family mismatches on B200 that classic sparse-int data never exercised:
test_sdpa_fp8_fwd_L0[test110]failstorch.testing.assert_closeon O (an MQA config: h_q=9, h_k=1, s=3770, d_qk=192/d_v=128). 1 of 256 sampled configs.test_sdpa_mxfp8_fwd_L0[test213]: "output mismatch: 43 elements differ"), i.e. kernel and reference emulation disagree bit-wise only in the negative-score regime. Needs triage into kernel-vs-emulator: either the kernel's exp/rescale path deviates for large-magnitude scores, or the emulator's modeling does.Notes
Repro
Check out #743, point at a cuDNN with the decode fwd fix, then:
Each failing test prints a self-contained
test_reprodict.🤖 Generated with Claude Code