Casting infinity to an intiger sometimes results in a 0 value(with optimizations enabled) #690

FractalFir · 2025-05-30T15:19:54Z

This program(which runs fine under MIRI) produces incorrect results when compiled with cg_gcc.

fn main() {
    let _2 = [true, false, true, true, false];
    let _1 = 0;
    let mut _6 = 0.0;
    let RET = 0.0;
    let mut _3 = 51216f32;
    let mut _4 = _3 * _3;
    let _13 = 0;
    let mut _14 = _4;
    let _20 = &mut _14;
    _4 = *_20 * *_20 * _4;
    _6 = *_20;
    *_20 = _4;
    let mut _28 = 0.0;
    let _41 = &mut _28;
    *_41 = *_20;
    let _39 = *_20 = *_41 * *_41;
    let _12 = *_41 = -*_20;
    _3 = *_41;
    let _57 = *_41;
    let _5 = *_41 = _3 * _6 * -_57;
    let val = *_41;
    let tmp = _28 * val;
    let _78 = tmp as u128;
    println!("{_78:?}");
}

The text was updated successfully, but these errors were encountered:

FractalFir · 2025-05-30T15:51:41Z

After further reduction, all of this is just a very long way of obtaining infinity.

fn main() {
    let mut _4 = 2623078656.0;
    _4 = _4 * _4 * _4;
    let val = -(_4 * _4) * _4 * (_4);
    let inf: f32 = val * val;
    let int = inf as u64;
    println!("{inf:?} {:?} {int:?}", inf.to_bits());
}

With optimizations enabled, GCC will optimize the inf as u64 cast to return 0, instead of u64::MAX.

bjorn3 · 2025-05-30T16:05:04Z

In C this is UB. LLVM used to have it be UB too for a long time, requiring rustc to check for overflow before doing the actual cast, but now there is an intrinsic which saturates on overflow. I guess GCC has the same UB.

FractalFir · 2025-05-30T16:19:11Z

Huh, I did not expect this to remain uncaught for so long.

cg_clr has a workaround for this, but it looks like cg_gcc does not.

https://github.com/rust-lang/rustc_codegen_gcc/blob/master/src%2Fbuilder.rs#L1212-L1216

I guess we will have to implement a workaround here too. Thankfully, it is a fairly simple thing to do.

There is one thing that concerns me, tough. The x86_64 float to int casts are wrapping, right?

So, with the default settings, this should have tripped way more tests.

This suggests to me that there is a workaround in play already, but it is just broken.

@antoyo does cg_gcc set any flags that could influence this? E.g. do you switch float casts to saturating somehow?

antoyo · 2025-05-30T18:09:56Z

@antoyo does cg_gcc set any flags that could influence this? E.g. do you switch float casts to saturating somehow?

I do not think so. We set some flags like -fwrapv, but I don't think this would influence this.

Any idea of tests that should fail because of this bug?

FractalFir · 2025-05-30T18:18:34Z

I would have to check. The more fundamental issue here is wrapping vs saturating casts. I believe that on x86_64 C casts wrap around, while in Rust they saturate. It seems to me like they are saturating(in `cg_gcc`), but in a subtly wrong way. If they were wrapping that should cause almost every file generated by rustlantis to fail. Looking at the generated assembly, I might have a rough idea about the case of the bug.

…

On Fri, May 30, 2025, 20:10 antoyo ***@***.***> wrote: *antoyo* left a comment (rust-lang/rustc_codegen_gcc#690) <#690 (comment)> @antoyo <https://github.com/antoyo> does cg_gcc set any flags that could influence this? E.g. do you switch float casts to saturating somehow? I do not think so. We set some flags like -fwrapv, but I don't think this would influence this. Any idea of tests that should fail because of this bug? — Reply to this email directly, view it on GitHub <#690 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AR52NDRXOW3AEQAJFCYSF3D3BCNIXAVCNFSM6AAAAAB6IKP4BOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMRTGA3DOOJVGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

FractalFir · 2025-05-30T18:47:29Z

Analyzing the GIMPLE we generate is a bit tough - duplicate block names make it hard to see what is happening.

So, it looks like the "saturation check" is present on the GIMPLE level, the problem just is... it happens after the cast.

 if (_37 < _38) goto then; else goto else;
      then:
      selectVar = 0;
      goto after;
      else:
      _39 = loadedValue2 * loadedValue3;
      _40 = -_39;
      _41 = loadedValue4 * _40;
      _42 = loadedValue5 * _41;
      _43 = loadedValue2 * loadedValue3;
      _44 = -_43;
      _45 = loadedValue4 * _44;
      _46 = loadedValue5 * _45;
      _47 = _42 * _46;
     // That seems wrong???
      selectVar = (size_t) _47;
      goto after;
      after:
      _48 = loadedValue2 * loadedValue3;
      _49 = -_48;
      _50 = loadedValue4 * _49;
      _51 = loadedValue5 * _50;
      _52 = loadedValue2 * loadedValue3;
      _53 = -_52;
      _54 = loadedValue4 * _53;
      _55 = loadedValue5 * _54;
      _56 = _51 * _55;
      _57 = 1.844674297419792384e+19;
      if (_56 > _57) goto then; else goto else;
      then:
      selectVar = 18446744073709551615;

Here, it looks like if the float value > 0, then we cast it to a float, and then check if it is larger than the max int value after the fact(swaping the value out in that case).

Since such a cast is UB if the floating point is out of range, GCC is able to "see" the loaded value can't possibly be larger than max(since that is UB), and removes the branch.

So, it seems like all of the muck at the begining is needed to get GCC to "see" that the branch is impossible.

FractalFir · 2025-05-30T19:00:36Z

I still don't get why the casts are sometimes saturating in the first place.

A piece of code like this works(is saturating):

  for i in 0..1000 {
        println!("{:?}", i as f32 as u8);
    }

But it should not, since a bit of equivalent C, compiled with GCC, does not(it wraps around).

#include <stdio.h>
#include  <float.h> 
int main()
{
    for (int i = 0; i < 1000; i++)printf("%u\n", (unsigned int)(unsigned char)((float)i));
    return 0;
}

antoyo · 2025-05-30T20:37:48Z

it happens after the cast.

I do not know if this is the problem here, but that kind of ordering issues can happen because GCC uses an AST-based IR while LLVM uses an instruction-based IR.
Which means that sometimes, we need to explicitly tell GCC to insert some instruction right now (for instance, load the value in a variable right now, instead of just forwarding the value which could be dereferenced in a another basic block).

FractalFir · 2025-05-30T20:40:16Z

I can't find where that branch is inserted. If it is not inserted by cg_gcc(I am not sure about that), then maybe it is inserted by libgccjit?

antoyo · 2025-05-30T20:41:17Z

It could be inserted in cg_ssa as well.
I would be surprised if libgccjit inserted that.

FractalFir · 2025-05-30T21:11:07Z

Never mind, it is in cg_gcc - I just could not find it. And, I immediately see the issue:

rustc_codegen_gcc/src/builder.rs

Line 1841 in 706905b

// 1. Cast val to an integer with fpto[su]i. This may result in undef.

At least the comment here suggests the order of operations(cast then check) is simply wrong.

We can work around this by swapping the order of operations around.

For unsigned casts, maybe could do something clever with clamps or min/max, e.g. (int) min(float_max, max(val, 0,0)) which should result in the correct value, even for infinites and NaN.

FractalFir · 2025-05-31T12:21:11Z

So, I reimplemented the cast in a way that I think should be fully sound:

// unsigned cast
__attribute__((noinline, visibility ("default")))
size_t cast (<float:32> param0)
{
  size_t D.4016;
  size_t fti_cast_res;

  start:
  _1 = 1.844674297419792384e+19;
  if (param0 > _1) goto gt_max; else goto lt_max;
  lt_max:
  _2 = 0.0;
  if (param0 > _2) goto in_bounds; else goto lt_min;
  in_bounds:
  fti_cast_res = (size_t) param0;
  goto after_cast;
  gt_max:
  fti_cast_res = 18446744073709551615;
  goto after_cast;
  lt_min:
  fti_cast_res = 0;
  goto after_cast;
  after_cast:
  D.4016 = fti_cast_res;
  return D.4016;
}

But it is still broken with optimizations. Can you see anything obviously wrong with this cast?
EDIT:
@antoyo
Could you please reopen the issue?

I got fat fingers, and Github's got some short-cuts I don't know about :(. I can't reopen it myself, sadly.

FractalFir · 2025-05-31T13:10:48Z

It seems like I can replicate the issue in C. I think the following code ought to be UB free, right?

#include <stdint.h>
#include <stdio.h>
uint64_t womky() {
  uint64_t ret;
  uint64_t fti_cast_res;
  float loadedValue5;
  float loadedValue4;
  float loadedValue3;
  float loadedValue2;
  float loadedValue1;
  char stack_var_1[4];
  char *stack_var_1_12_1;
  float _19;
  float _29;

  stack_var_1_12_1 = &stack_var_1[0];
  float _2 = 3.4028234663852885981170418348451692544e+38;
  *(float *)stack_var_1_12_1 = _2;
  char *stack_var_1_13_3 = &stack_var_1[0];
  loadedValue1 = *((float *)stack_var_1_13_3);
  char *stack_var_1_14_4 = &stack_var_1[0];
  *((float *)stack_var_1_14_4) = loadedValue1;
  char *stack_var_1_15_5 = &stack_var_1[0];
  loadedValue2 = *((float *)stack_var_1_15_5);
  char *stack_var_1_16_6 = &stack_var_1[0];
  loadedValue3 = *((float *)stack_var_1_16_6);
  char *stack_var_1_17_7 = &stack_var_1[0];
  loadedValue4 = *((float *)stack_var_1_17_7);
  char *stack_var_1_18_8 = &stack_var_1[0];
  loadedValue5 = *((float *)stack_var_1_18_8);
  float _9 = loadedValue2 * loadedValue3;
  float _10 = -_9;
  float _11 = loadedValue4 * _10;
  float _12 = loadedValue5 * _11;
  float _13 = loadedValue2 * loadedValue3;
  float _14 = -_13;
  float _15 = loadedValue4 * _14;
  float _16 = loadedValue5 * _15;
  float _17 = _12 * _16;
  float _18 = 1.844674297419792384e+19;
  if (_17 > _18)
    goto gt_max;
  else
    goto lt_max;
lt_max:
  _19 = loadedValue2 * loadedValue3;
  float _20 = -_19;
  float _21 = loadedValue4 * _20;
  float _22 = loadedValue5 * _21;
  float _23 = loadedValue2 * loadedValue3;
  float _24 = -_23;
  float _25 = loadedValue4 * _24;
  float _26 = loadedValue5 * _25;
  float _27 = _22 * _26;
  float _28 = 0.0;
  if (_27 > _28)
    goto in_bounds;
  else
    goto lt_min;
in_bounds:
  _29 = loadedValue2 * loadedValue3;
  float _30 = -_29;
  float _31 = loadedValue4 * _30;
  float _32 = loadedValue5 * _31;
  float _33 = loadedValue2 * loadedValue3;
  float _34 = -_33;
  float _35 = loadedValue4 * _34;
  float _36 = loadedValue5 * _35;
  float _37 = _32 * _36;
  fti_cast_res = (uint64_t)_37;
  goto after_cast;
gt_max:
  fti_cast_res = 18446744073709551615lu;
  goto after_cast;
lt_min:
  fti_cast_res = 0;
  goto after_cast;
after_cast:
  ret = fti_cast_res;
  return ret;
}

int main() {
  printf("Hello World %lu", (uint64_t)(1.0 / 0.0));
  return 0;
}

antoyo · 2025-05-31T13:11:43Z

But it is still broken with optimizations. Can you see anything obviously wrong with this cast?

Could you please push your branch somewhere and tell me exactly which reproducer produces that GIMPLE IR?

FractalFir · 2025-05-31T13:56:12Z

The branch is here: https://github.com/FractalFir/rustc_codegen_gcc/tree/master

Both the snippet of gimple, and the GIMPLE converted to C come from this small Rust reproducer.

#[unsafe(no_mangle)]
#[inline(never)]
pub fn cast(num: f32) -> u64 {
    num as u64
}
#[unsafe(no_mangle)]
fn womky() -> u64 {
    let mut _4 = 340282346638528859811704183484516925440.000000;
    _4 = _4;
    let val = -(_4 * _4) * _4 * (_4);
    let inf: f32 = (val * val);
    let int = inf as u64;
    int
}
fn main() {
    println!("{:?}", womky());
}

The cast function exists only because I wanted to check exactly what gimple I generated for casts.

FractalFir · 2025-05-31T15:18:34Z

I further minimzed the C example, removing the cast alltogether. This function returns true in -O1, but false in -O2.

Since it only contains floating-point math(which is UB-free?), it should behave identically in both cases.

#include <stdbool.h>
#include <stdio.h>
bool womky(void) {
  float val = 3.4028234663852885981170418348451692544e+38;
  float a = val * -(val * val);
  float b = val * a;
  return b * b > 0.0;
}
int main(void) { printf("%d", womky()); }

EDIT: I think I know what is going on.

Here, we get screwed over by the sign of infinity.

If you run this Rust code:

println!("{} {}",(1.0 / 0.0) as u64, (-1.0 / 0.0) as u64);

You'll see that casting inf and -inf to a u64 results in different values.

Here, GCC's floating-point optimizations change a positive infinity to a negative one. I think this is OK in C(not sure, tough).

I don't know if this is a miscompilation according to Rust(I think so).

antoyo · 2025-05-31T15:45:44Z

I don't know if this is a miscompilation according to Rust(I think so).

There's been a lot of discussion about the float semantics in Rust. You might be able to find the answer to that question in one of those links:

FractalFir · 2025-05-31T15:45:55Z

When I looked at the bits of the float(or the float itself), they were identical.

➜  rustc_codegen_gcc git:(fuzz_support) ✗ ./gcc_debug.out
inf:inf bits:2139095040 int:18446744073709551615
➜  rustc_codegen_gcc git:(fuzz_support) ✗ ./gcc.out       
inf:inf bits:2139095040 int:0
➜  rustc_codegen_gcc git:(fuzz_support) ✗

It looks like, in this case, GCC only applied the optimizations when computing the floating-point value for the cast, but not when it got reinterpreted as bits(or passed to be displayed).

In other words, the value of that float is indeterminate(is this the right term?): each time we look at it, it changes.

FractalFir · 2025-05-31T15:50:25Z

This has the potential to royally screw us over. Consider a cast like this:

if (val > (float) ULONG_MAX) return ULONG_MAX;
else if (val < 0) return 0;
else return (uint64_t) val;

Suppose GCC optimizes the value in the 1st check to -infinity, and keeps the rest as infinity.

Now, our code looks like this:

if (-infinity > (float) ULONG_MAX) return ULONG_MAX;
else if (infinity < 0) return 0;
else return (uint64_t) infinity;

Since negative infinity is less than ULONG_MAX, we fall trough to this code:

else if (infinity < 0) return 0;
else return (uint64_t) infinity;

Positive ininfity is more than 0, so we fall trough here:

else return (uint64_t) infinity;

And now, our code has UB it did not have before. On some systems, this could cause a crash.

FractalFir · 2025-05-31T17:37:59Z

Manifesting this issue in C is quite easy. Here, we can see that:

res is positive infinity, with the bit pattern 0x7f800000(the sign bit is not set)
res is, at the same time, smaller than 0.

#include <stdbool.h>
#include <stdio.h>
#include <string.h>

unsigned int to_bits(float f){
	unsigned int ret;
	memcpy(&ret,&f, sizeof(unsigned int));
	return ret;
}
bool womky(void) {
  float val = 3.4028234663852885981170418348451692544e+38;
  float a = val * -(val * val);
  float b = val * a;
  float res = b * b;
  printf("%f %x\n",res, to_bits(res));
  return res > 0.0;
}
int main(void) { printf("%d\n", womky()); }

@antoyo do you know if this behaviour(the same variable has 2 different values when accessed) is permitted by the C standard?

antoyo · 2025-05-31T19:50:21Z

I have honestly no idea.
I don't know if the problem really is that 2 different values are observed: it seems related to the float comparison.
For instance, printing signbit(res) will show the same value for both debug and release builds.

And putting the comparison in a noinline function:

int __attribute__ ((noinline)) is_positive(float f) {
    return f > 0.0;
}

gives the correct result.

nikic · 2025-06-01T11:30:23Z

@FractalFir As far as I can tell this is just a plain GCC bug (somewhere in VRP presumably).

The result of the float calculation here is required to be +inf, and the result of the comparison must be true. All the funky float non-determinism stuff is related to NaN values (which are not involved here) and requires inspecting the bitwise representation of the float (float comparisons are not affected by NaN signs).

nikic · 2025-06-01T11:34:29Z

https://gcc.godbolt.org/z/4j66dWz6s has vrp1 printing something like this:

Global Exported: a_4 = [frange] float [+Inf, +Inf] +-NAN
Global Exported: b_5 = [frange] float [] +-NAN
Global Exported: _3 = [frange] float [] +-NAN

If I'm reading this right, it seems to think that b can only be NAN, which is ... not right.

antoyo added the bug Something isn't working label May 30, 2025

FractalFir changed the title ~~Compiler bug found using rustlantis.~~ Casting infinity to an intiger sometimes results in a 0 value(with optimizations enabled) May 30, 2025

FractalFir closed this as completed May 31, 2025

antoyo reopened this May 31, 2025

Casting infinity to an intiger sometimes results in a 0 value(with optimizations enabled) #690

Casting infinity to an intiger sometimes results in a 0 value(with optimizations enabled) #690

Comments

FractalFir commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

FractalFir commented May 30, 2025

Uh oh!

bjorn3 commented May 30, 2025

Uh oh!

FractalFir commented May 30, 2025

Uh oh!

antoyo commented May 30, 2025

Uh oh!

FractalFir commented May 30, 2025 via email • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FractalFir commented May 30, 2025

Uh oh!

FractalFir commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antoyo commented May 30, 2025

Uh oh!

FractalFir commented May 30, 2025

Uh oh!

antoyo commented May 30, 2025

Uh oh!

FractalFir commented May 30, 2025

Uh oh!

FractalFir commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FractalFir commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antoyo commented May 31, 2025

Uh oh!

FractalFir commented May 31, 2025

Uh oh!

FractalFir commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antoyo commented May 31, 2025

Uh oh!

FractalFir commented May 31, 2025

Uh oh!

FractalFir commented May 31, 2025

Uh oh!

FractalFir commented May 31, 2025

Uh oh!

antoyo commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nikic commented Jun 1, 2025

Uh oh!

nikic commented Jun 1, 2025

Uh oh!

FractalFir commented May 30, 2025 •

edited

Loading

FractalFir commented May 30, 2025 via email •

edited

Loading

FractalFir commented May 30, 2025 •

edited

Loading

FractalFir commented May 31, 2025 •

edited

Loading

FractalFir commented May 31, 2025 •

edited

Loading

FractalFir commented May 31, 2025 •

edited

Loading

antoyo commented May 31, 2025 •

edited

Loading