v1.11-rc2: Inconsistent behaviour between function call and expected behaviour

[dlfivefifty]

I have a function call where the behaviour is completely different from typing that function manually. I've tried to construct a simpler example but haven't been successful.

Here is the broken code:

julia> using InfiniteArrays, FillArrays, LinearAlgebra

julia> A = Tridiagonal(Fill(1,∞), Fill(2,∞), Fill(3,∞))
ℵ₀×ℵ₀ Tridiagonal{Int64, Fill{Int64, 1, Tuple{InfiniteArrays.OneToInf{Int64}}}} with indices OneToInf()×OneToInf():
 2  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 1  2  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  1  2  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  1  2  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  1  2  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  1  2  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  1  2  3  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  2  3  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  2  3  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  2  3  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  2  3  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  2  3     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  2     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1     
 ⋮              ⋮              ⋮        ⋱  

julia> B = 2I;

julia> A + B
ℵ₀×ℵ₀ Tridiagonal{Int64, Fill{Int64, 1, Tuple{InfiniteArrays.OneToInf{Int64}}}} with indices OneToInf()×OneToInf():
 0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1     
 ⋮              ⋮              ⋮        ⋱  

The diagonal of the result should be 4. Here is the actual offending function which I have re-implemented as myplus:

julia> @which A + B
+(A::Tridiagonal, B::UniformScaling)
     @ LinearAlgebra ~/Projects/julia-1.11/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/special.jl:230

julia> function myplus(A, B) # copied from special.jl:2030
           newd = A.d .+ Ref(B)
           Tridiagonal(typeof(newd)(A.dl), newd, typeof(newd)(A.du))
       end
myplus (generic function with 1 method)

julia> myplus(A, B) # same behaviour
ℵ₀×ℵ₀ Tridiagonal{Int64, Fill{Int64, 1, Tuple{InfiniteArrays.OneToInf{Int64}}}} with indices OneToInf()×OneToInf():
 0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0  3     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  0     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1     
 ⋮              ⋮              ⋮        ⋱  

So its not related to the macro @commutative.

Now the odd part: if I type out the actual code it works fine:

julia> newd = A.d .+ Ref(B)
ℵ₀-element Fill{Int64, 1, Tuple{InfiniteArrays.OneToInf{Int64}}} with indices OneToInf(), with entries equal to 4

julia> Tridiagonal(typeof(newd)(A.dl), newd, typeof(newd)(A.du)) 
ℵ₀×ℵ₀ Tridiagonal{Int64, Fill{Int64, 1, Tuple{InfiniteArrays.OneToInf{Int64}}}} with indices OneToInf()×OneToInf():
 4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1     
 ⋮              ⋮              ⋮        ⋱  

Running the code via Debugger also returns the correct result.

Note here is the bug in CI showing its present in all OSes: JuliaLinearAlgebra/InfiniteLinearAlgebra.jl#187

[dlfivefifty]

For completeness: this bug was not present in Julia v1.10 with the myplus implementation coming from Julia v1.11.

              _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.2 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

julia> using InfiniteArrays, FillArrays, LinearAlgebra

julia> function myplus(A, B) # copied from special.jl:2030
           newd = A.d .+ Ref(B)
           Tridiagonal(typeof(newd)(A.dl), newd, typeof(newd)(A.du))
       end
myplus (generic function with 1 method)

julia> A = Tridiagonal(Fill(1,∞), Fill(2,∞), Fill(3,∞));

julia> B = 2I;

julia> myplus(A,B)
ℵ₀×ℵ₀ Tridiagonal{Int64, Fill{Int64, 1, Tuple{InfiniteArrays.OneToInf{Int64}}}} with indices OneToInf()×OneToInf():
 4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  ⋅     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3  ⋅  …  
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4  3     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1  4     
 ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  ⋅  1     
 ⋮              ⋮              ⋮        ⋱  

[nsajko]

I'd try deleting all @assume_effects annotations from the minimal reproducer, to eliminate user error. So including from all dependencies.

Sidenote, this is a place where having #54436 would be useful.

[nsajko]

The example above (A + B) works fine if julia is started with -O0, but it's buggy for -O1.

[jishnub]

Bisected to:

81afdbc36b365fcbf3ae25b7451c6cb5798c0c3d is the first bad commit
commit 81afdbc36b365fcbf3ae25b7451c6cb5798c0c3d
Author: Jameson Nash <[email protected]>
Date:   Fri Nov 17 18:44:45 2023 -0500

    codegen: remove UB from uninitialized bitstypes in new (#52169)
    
    In the time since the creation of issue #26764, there _is_ now 'a way to
    say to llvm "I don't care what this value is, but it always has to be
    the same"' using the `freeze` instruction, so we can use that to
    instruct LLVM to not give us undefined behavior when users are using
    uninitialized memory. There should not be an impact if users were
    already avoiding this paradigm and are fully initializing their structs.
    
    Fixes #26764