[Hexagon] HVX `qf32` values are silently corrupted when spilled (qfloat extended state not preserved by the register allocator)

[L-roro]

TL;DR

On HVX targets with the qfloat ops enabled (+hvx-qfloat, which is auto-enabled for any HVX v68+), the register allocator can spill (and copy) an HVX qf32 (qfloat) value with a plain vmem store/reload, and the reloaded value is then consumed by a qf32 instruction. A qf32 value carries architectural extended state that a plain vmem store/reload (or a non-qf copy) does not preserve. The result is a silent miscompile — large garbage values under register pressure.

The backend has no notion that qf32 vector values are different from ordinary sf/integer vectors (both are MVT::v32f32 in HvxVR), so the allocator/coalescer/spiller treat them as freely spillable/copyable.

I have tested a post-RA fixup pass that patches the observed cases (in a small toy model), but it is a very poor fix. I believe a more robust fix is needed.

Affected configuration

• Target: hexagon-unknown-unknown-elf, -mcpu=hexagonv79, -mattr=+hvxv79,+hvx-length128b. The qfloat ops (+hvx-qfloat) are auto-enabled by the subtarget for any HVX v68+, so no explicit feature flag is needed to reproduce.
• LLVM commit: 0a73d41
• Relevant components: HexagonQFPOptimizer, the HVX qfloat ISel patterns (HexagonPatternsHVX.td), and the generic register allocator / spiller.

Background: HVX qf32 extended state

https://docs.qualcomm.com/doc/80-N2040-61/topic/hvx-floating-point.html#programming-with-hvx-floating-point

The bug

ISel lowers HVX float ops to the spill-safe form: each op is immediately followed by V6_vconv_sf_qf32, so every inter-op value is IEEE sf (which round-trips through memory fine). HexagonQFPOptimizer then removes those conversions and rewrites consumers to the bare qf32 variants (V6_vadd_sf -> V6_vadd_qf32_mix -> V6_vadd_qf32, V6_vmpy_qf32_sf -> V6_vmpy_qf32, …; see QFPInstMap). This is optimization creates long-lived qf32 values that implicitly assume they stay in registers.

Under register pressure RAGreedy spills such a qf32 accumulator with a plain PS_vstorerv_ai / PS_vloadrv_ai (→ V6_vS32b_ai/V6_vL32b_ai), and the reload is consumed as .qf32:

v0.qf32 = vmpy(v0.sf, v1.sf)      ; q = a*b  (qf32, extended state)
...
vmem(r29+#1) = v0                 ; plain spill -> extended state dropped
call opaque_call
...
v1 = vmem(r29+#1)                 ; reload (extended state reset)
v0.qf32 = vadd(v1.qf32, v0.sf)    ; consumed as qf32 -> corrupted

Because qf32 and sf are indistinguishable at the type/regclass level (MVT::v32f32, HvxVR), nothing in the allocator/coalescer/spiller knows this spill is not value-preserving, and -verify-machineinstrs is clean (it is a valid vector spill at the MIR level).

The corruption is data-dependent: it bites when the spilled qf32 is unnormal (e.g. a near-zero a*0 product). In a tiled f32 matmul this produces output on the order of 1e31.

Minimal reproducer

A qf32 value is kept live across a call to force a spill:

; RUN: llc -mtriple=hexagon-unknown-unknown-elf -mcpu=hexagonv79 \
; RUN:   -mattr=+hvxv79,+hvx-length128b --relocation-model=pic < %s -o -
target triple = "hexagon-unknown-unknown-elf"
declare void @opaque_call()
define void @qf32_call_spill(ptr %out, <32 x float> %a, <32 x float> %b, <32 x float> %c) #0 {
  %q = fmul <32 x float> %a, %b
  call void @opaque_call()
  %r = fadd <32 x float> %q, %c
  store <32 x float> %r, ptr %out, align 128
  ret void
}
attributes #0 = { nounwind "target-cpu"="hexagonv79" "target-features"="+hvxv79,+hvx-length128b" }

The emitted .s contains vmem(r29+#N) = v<q> (plain qf32 spill) followed by a reload consumed as vadd(v<q>.qf32, …). On the simulator with inputs that make a*b near-zero, the result mismatches an IEEE reference; converting the value to sf before the spill (and reading the reload as .sf) makes it correct.

A useful confirmation: -disable-qfp-opt makes the miscompile disappear (ISel's sf-normalized chains are spill-safe), at the cost of the extra conversions. This pins HexagonQFPOptimizer as the introducer of the spillable qf32 live ranges.

Attempted fix: a post-RA legalization pass (and its limitations)

I tested HexagonQFPSpillFixup, a post-RA pass that runs while spills are still PS_v{store,load}rv_ai with frame indices. A small forward dataflow classifies each vector register's value as QF32 / NotQF32 / MaybeQF32; for slots it can fully prove, it converts the qf32 store to sf in place (V6_vconv_sf_qf32, Vd==Vu, no extra register) and retypes the reload's qf32-reading consumers to read .sf (vadd_qf32 -> vadd_qf32_mix -> vadd_sf). It needs no reload-side conversion and no v81-only opcode (v81 has a conversion operation to .sf that would make this a lot simpler on reload, since it could be achieved with a single register, which is not the case as far as I am aware in earlier versions and using multiple registers would be problematic post RA), is verifier-clean, and fixes my small model's miscompilation end-to-end.

Huge structural limitations though:

• It must enumerate patterns. Correctness depends on a hand-maintained list of qf32-producing opcodes, which operands are read as .qf32, which consumers are retypeable, and which opcodes are reloads. A missed qf32 producer would be misclassified NotQF32 and left unconverted — a latent miscompile. New ISA ops or new lowering patterns silently create blind spots.
• It bails (leaves the bug unfixed) on cases it cannot prove, including:
  • the reloaded value flowing through a COPY before its consumer;
  • chained spills (a reloaded value re-stored to another slot);
  • cross-block live-in joins (a value that is qf32 on one path and sf on another);
  • qf32 consumers with no demotable form — V6_vmpy_qf32 (no qf32 x sf multiply) and V6_vsub_qf32 (the _mix form is v81+);
  • values that are live-out of the block.
• It only handles spills, not copies. A coalescer-left plain copy of a qf32 value is equally unsafe and is not addressed.
• It only handles single-vector (HvxVR) spills, not vector pairs (HvxWR).
• It changes numerics slightly (the qf32 extended range is collapsed to sf at spill points).

In short, such a pass fixes the shapes one model happens to produce but the structural issue is harder to solve.

A more robust fix: model qf32-ness

Maybe the principled fix is to represent "this vector value is qf32 (carries non-round-trippable extended state)" as a first-class property that the register allocator, coalescer, and spiller respect — e.g. a distinct register class (HvxVR_qf32 aliasing V0–V31), a value-state/subreg flag, or a dedicated MVT. Then:

• Identification is carried, not re-derived. The property flows from the defining instruction through copies, PHIs, spills, reloads, and coalescing via existing infrastructure — no opcode enumeration, and the cross-block / chained-spill / copy-flow cases are handled for free.
• The spill/copy hooks handle every case uniformly (storeRegToStackSlot / copyPhysReg branch on the class), so there is no "unknown pattern → bail."
• It can make the spill value-preserving without retyping consumers (e.g. keep qf32 values out of plain spills via spill weight / rematerialization, or give the class a convert-and-reconvert spill), which dissolves the non-demotable-consumer (vmpy_qf32/vsub_qf32) dead-end entirely.

This does not make the underlying cost free (a plain vmem still cannot carry qf32 extended state, so the spill must still convert or be avoided), but it makes the handling sound and exhaustive rather than pattern-matched. An orthogonal option is to constrain HexagonQFPOptimizer so it does not create qf32 live ranges that can be spilled/copied across qf32 consumers.

Questions for maintainers

I am working on a completely different project and have little experience with llvm and the hexagon backend. Given that this seems like a much more complex and architectural fix, I am unsure that I will have the time to tackle it myself.

1. Is this bug something that you are aware of? Do you have any solution under development?
2. Would it be better to rework how HexagonQFPOptimizer works altogether instead of piling up patches?
3. If this is being considered and given that a solution might be simpler in v81+, will this still get fixed for earlier versions?

Links

My questionable fixup pass:
Disclaimer: This was heavily vibe-coded and I did not attempt to clean it up, since I believe it is a bad solution anyway.
https://github.com/llvm/llvm-project/compare/main...L-roro:llvm-project:hexagon-add-qfp-spill-fixup?expand=1#diffe28743586c48d1ec941ba1b02b6de259773f08225bbaf7cda4a96c406fc06832

The original ll that I was completely unable to reduce to track down the bug. It is coming from a very experimental project, and the code is highly unoptimized. I am aware that it has tremendously inefficient code. Nevertheless, this bug was particularly nasty to track down given its interplay with register allocation and pressure, along with its data dependent nature. If you want to run it instead of the minimal reproducer, the external functions will need an implementation:

dispatch.opt.ll.txt

[llvmorg-github-actions]

@llvm/issue-subscribers-backend-hexagon

Author: L-roro

Details

## TL;DR

On HVX targets with the qfloat ops enabled (+hvx-qfloat, which is auto-enabled for any HVX v68+), the register allocator can spill (and copy) an HVX qf32 (qfloat) value with a plain vmem store/reload, and the reloaded value is then consumed by a qf32 instruction. A qf32 value carries architectural extended state that a plain vmem store/reload (or a non-qf copy) does not preserve. The result is a silent miscompile — large garbage values under register pressure.

The backend has no notion that qf32 vector values are different from ordinary sf/integer vectors (both are MVT::v32f32 in HvxVR), so the allocator/coalescer/spiller treat them as freely spillable/copyable.

I have tested a post-RA fixup pass that patches the observed cases (in a small toy model), but it is a very poor fix. I believe a more robust fix is needed.

Affected configuration

• Target: hexagon-unknown-unknown-elf, -mcpu=hexagonv79, -mattr=+hvxv79,+hvx-length128b. The qfloat ops (+hvx-qfloat) are auto-enabled by the subtarget for any HVX v68+, so no explicit feature flag is needed to reproduce.
• Relevant components: HexagonQFPOptimizer, the HVX qfloat ISel patterns (HexagonPatternsHVX.td), and the generic register allocator / spiller.

Background: HVX qf32 extended state

https://docs.qualcomm.com/doc/80-N2040-61/topic/hvx-floating-point.html#programming-with-hvx-floating-point

The bug

ISel lowers HVX float ops to the spill-safe form: each op is immediately followed by V6_vconv_sf_qf32, so every inter-op value is IEEE sf (which round-trips through memory fine). HexagonQFPOptimizer then removes those conversions and rewrites consumers to the bare qf32 variants (V6_vadd_sf -> V6_vadd_qf32_mix -> V6_vadd_qf32, V6_vmpy_qf32_sf -> V6_vmpy_qf32, …; see QFPInstMap). This is optimization creates long-lived qf32 values that implicitly assume they stay in registers.

Under register pressure RAGreedy spills such a qf32 accumulator with a plain PS_vstorerv_ai / PS_vloadrv_ai (→ V6_vS32b_ai/V6_vL32b_ai), and the reload is consumed as .qf32:

v0.qf32 = vmpy(v0.sf, v1.sf)      ; q = a*b  (qf32, extended state)
...
vmem(r29+#<!-- -->1) = v0                 ; plain spill -&gt; extended state dropped
call opaque_call
...
v1 = vmem(r29+#<!-- -->1)                 ; reload (extended state reset)
v0.qf32 = vadd(v1.qf32, v0.sf)    ; consumed as qf32 -&gt; corrupted

Because qf32 and sf are indistinguishable at the type/regclass level (MVT::v32f32, HvxVR), nothing in the allocator/coalescer/spiller knows this spill is not value-preserving, and -verify-machineinstrs is clean (it is a valid vector spill at the MIR level).

The corruption is data-dependent: it bites when the spilled qf32 is unnormal (e.g. a near-zero a*0 product). In a tiled f32 matmul this produces output on the order of 1e31.

Minimal reproducer

A qf32 value is kept live across a call to force a spill:

; RUN: llc -mtriple=hexagon-unknown-unknown-elf -mcpu=hexagonv79 \
; RUN:   -mattr=+hvxv79,+hvx-length128b --relocation-model=pic < %s -o -
target triple = "hexagon-unknown-unknown-elf"
declare void @<!-- -->opaque_call()
define void @<!-- -->qf32_call_spill(ptr %out, &lt;32 x float&gt; %a, &lt;32 x float&gt; %b, &lt;32 x float&gt; %c) #<!-- -->0 {
  %q = fmul &lt;32 x float&gt; %a, %b
  call void @<!-- -->opaque_call()
  %r = fadd &lt;32 x float&gt; %q, %c
  store &lt;32 x float&gt; %r, ptr %out, align 128
  ret void
}
attributes #<!-- -->0 = { nounwind "target-cpu"="hexagonv79" "target-features"="+hvxv79,+hvx-length128b" }

The emitted .s contains vmem(r29+#N) = v<q> (plain qf32 spill) followed by a reload consumed as vadd(v<q>.qf32, …). On the simulator with inputs that make a*b near-zero, the result mismatches an IEEE reference; converting the value to sf before the spill (and reading the reload as .sf) makes it correct.

A useful confirmation: -disable-qfp-opt makes the miscompile disappear (ISel's sf-normalized chains are spill-safe), at the cost of the extra conversions. This pins HexagonQFPOptimizer as the introducer of the spillable qf32 live ranges.

Attempted fix: a post-RA legalization pass (and its limitations)

I tested HexagonQFPSpillFixup, a post-RA pass that runs while spills are still PS_v{store,load}rv_ai with frame indices. A small forward dataflow classifies each vector register's value as QF32 / NotQF32 / MaybeQF32; for slots it can fully prove, it converts the qf32 store to sf in place (V6_vconv_sf_qf32, Vd==Vu, no extra register) and retypes the reload's qf32-reading consumers to read .sf (vadd_qf32 -> vadd_qf32_mix -> vadd_sf). It needs no reload-side conversion and no v81-only opcode (v81 has a conversion operation to .sf that would make this a lot simpler on reload, since it could be achieved with a single register, which is not the case as far as I am aware in earlier versions and using multiple registers would be problematic post RA), is verifier-clean, and fixes my small model's miscompilation end-to-end.

Huge structural limitations though:

• It must enumerate patterns. Correctness depends on a hand-maintained list of qf32-producing opcodes, which operands are read as .qf32, which consumers are retypeable, and which opcodes are reloads. A missed qf32 producer would be misclassified NotQF32 and left unconverted — a latent miscompile. New ISA ops or new lowering patterns silently create blind spots.
• It bails (leaves the bug unfixed) on cases it cannot prove, including:
  • the reloaded value flowing through a COPY before its consumer;
  • chained spills (a reloaded value re-stored to another slot);
  • cross-block live-in joins (a value that is qf32 on one path and sf on another);
  • qf32 consumers with no demotable form — V6_vmpy_qf32 (no qf32 x sf multiply) and V6_vsub_qf32 (the _mix form is v81+);
  • values that are live-out of the block.
• It only handles spills, not copies. A coalescer-left plain copy of a qf32 value is equally unsafe and is not addressed.
• It only handles single-vector (HvxVR) spills, not vector pairs (HvxWR).
• It changes numerics slightly (the qf32 extended range is collapsed to sf at spill points).

In short, such a pass fixes the shapes one model happens to produce but the structural issue is harder to solve.

A more robust fix: model qf32-ness

Maybe the principled fix is to represent "this vector value is qf32 (carries non-round-trippable extended state)" as a first-class property that the register allocator, coalescer, and spiller respect — e.g. a distinct register class (HvxVR_qf32 aliasing V0–V31), a value-state/subreg flag, or a dedicated MVT. Then:

• Identification is carried, not re-derived. The property flows from the defining instruction through copies, PHIs, spills, reloads, and coalescing via existing infrastructure — no opcode enumeration, and the cross-block / chained-spill / copy-flow cases are handled for free.
• The spill/copy hooks handle every case uniformly (storeRegToStackSlot / copyPhysReg branch on the class), so there is no "unknown pattern → bail."
• It can make the spill value-preserving without retyping consumers (e.g. keep qf32 values out of plain spills via spill weight / rematerialization, or give the class a convert-and-reconvert spill), which dissolves the non-demotable-consumer (vmpy_qf32/vsub_qf32) dead-end entirely.

This does not make the underlying cost free (a plain vmem still cannot carry qf32 extended state, so the spill must still convert or be avoided), but it makes the handling sound and exhaustive rather than pattern-matched. An orthogonal option is to constrain HexagonQFPOptimizer so it does not create qf32 live ranges that can be spilled/copied across qf32 consumers.

Questions for maintainers

I am working on a completely different project and have little experience with llvm and the hexagon backend. Given that this seems like a much more complex and architectural fix, I am unsure that I will have the time to tackle it myself.

1. Is this bug something that you are aware of? Do you have any solution under development?
2. Would it be better to rework how HexagonQFPOptimizer works altogether instead of piling up patches?
3. If this is being considered and given that a solution might be simpler in v81+, will this still get fixed for earlier versions?

Links

My questionable fixup pass:
Disclaimer: This was heavily vibe-coded and I did not attempt to clean it up, since I believe it is a bad solution anyway.
https://github.com/llvm/llvm-project/compare/main...L-roro:llvm-project:hexagon-add-qfp-spill-fixup?expand=1#diffe28743586c48d1ec941ba1b02b6de259773f08225bbaf7cda4a96c406fc06832

The original ll that I was completely unable to reduce to track down the bug. It is coming from a very experimental project, and the code is highly unoptimized. I am aware that it has tremendously inefficient code. Nevertheless, this bug was particularly nasty to track down given its interplay with register allocation and pressure, along with its data dependent nature. If you want to run it instead of the minimal reproducer, the external functions will need an implementation:

dispatch.opt.ll.txt

[androm3da]

cc @SundeepKushwaha @sgundapa @fhossein-quic

[fhossein-quic]

It seems to me that this issue is fixed by #198902
This is what I get:

  ...
  v0.sf = v0.qf32                     => converted to sf before spilling
  vmem(r29+#1) = v0.new               => spilled as sf, no extended state to lose
  {
  call opaque_call
  vmem(r29+#0) = v2
  }
  {
  v0.qf32 = vadd(v1.sf,v0.sf)          => v1 (the reload) consumed as sf, not qf32
  v1.cur = vmem(r29+#1)                => reload
  }```
  
Could you please retest on top-of-tree main and confirm? If it's clean on your end too, this should be good to close..

[androm3da]

this should be good to close..

If it's fixed on main then ideally we can cherry-pick the fix commit(s) to release/23.x for closure.

[fhossein-quic]

The patch is already on 23 as well. Commit 1485386.