Skip to content

Don't re-infer compiled CodeInstance from external images - #61479

Merged
KristofferC merged 3 commits into
JuliaLang:masterfrom
xal-0:precompile-external-linkage
Sep 1, 2026
Merged

Don't re-infer compiled CodeInstance from external images#61479
KristofferC merged 3 commits into
JuliaLang:masterfrom
xal-0:precompile-external-linkage

Conversation

@xal-0

@xal-0 xal-0 commented Apr 2, 2026

Copy link
Copy Markdown
Member

998cb27 introduced a regression that, when invoking a CodeInstance from an external image, would result in the generation of a tojlinvoke trampoline for every invoke but the first (in the order compile! infers them).

module P2

function what(x)
    try
        show(x)
    catch e
        rethrow()
    end
end

what(1) # calls julia_rethrow_88_gfthunk
what(:a) # calls julia_rethrow_88_gfthunk
what('a') # calls jlpkg_rethrow_0

end # module P2

After inferring the first use of rethrow in this example, compile! would find no source for the CodeInstance and infer it again, with source_mode=SOURCE_MODE_GET_SOURCE. It then inserts the resulting new CodeInstance into the code_cache. Before 998cb27, this went into the InternalCodeCache, which would insert it after any CodeInstances with the invoke field set. Now, it is inserted into the OverlayCodeCache, which takes priority over the InternalCodeCache. For the two remaining specializations of what, this new CodeInstance is used for the invoke.

Since it's a waste to re-infer this CodeInstance when we have external_linkage, it makes more sense to thread this through to compile! and skip any CodeInstances that are from the image.

Thanks to @mlechu for helping track down this issue and providing the MWE.

Comment thread Compiler/src/typeinfer.jl Outdated
@xal-0
xal-0 force-pushed the precompile-external-linkage branch from 7288214 to b3736c0 Compare April 10, 2026 18:10
Comment thread src/aotcompile.cpp
}

// Check the global cache for an equivalent CodeInstance with a world age range
// containing the world age range of the given CodeInstance.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this function also is supposed to check for the JL_CI_FLAGS_FROM_IMAGE marker. Not required (and generally the cache structure will be expected to surface it first anyways), but defensive?

xal-0 added 3 commits August 31, 2026 10:01
998cb27 introduced a regression that, when invoking a CodeInstance
from an external image, would result in the generation of a tojlinvoke
trampoline for every invoke but the first (in the order `compile!`
infers them).

```julia
module P2

function what(x)
    try
        show(x)
    catch e
        rethrow()
    end
end

what(1) # calls julia_rethrow_88_gfthunk
what(:a) # calls julia_rethrow_88_gfthunk
what('a') # calls jlpkg_rethrow_0

end # module P2
```

After inferring the first use of `rethrow` in this example, `compile!` would
find no source for the CodeInstance and infer it again, with
`source_mode=SOURCE_MODE_GET_SOURCE`.  It then inserts the resulting new
CodeInstance into the `code_cache`.  Before 998cb27, this went into the
InternalCodeCache, which would insert it after any CodeInstances with the
`invoke` field set.  Now, it is inserted into the OverlayCodeCache, which takes
priority over the InternalCodeCache.  For the two remaining specializations of
`what`, this new CodeInstance is used for the invoke.

Since it's a waste to re-infer this CodeInstance when we have
`external_linkage`, it makes more sense to thread this through to `compile!`
and skip any CodeInstances that are from the image.

Thanks to @mlechu for helping track down this issue and providing the
MWE.
@xal-0
xal-0 force-pushed the precompile-external-linkage branch from b3736c0 to d6a2f9c Compare August 31, 2026 17:30
@IanButterworth

Copy link
Copy Markdown
Member

This seems a more complete (and approved) fix than #62934

So I'll just share the benchmarks I did over there, for visibility, and close #62934


This gives a 20% improvement in precompilation time on 1.13 and 10% improvement on nightly

Before
1.13 = #62926 1483760
nightly = ba658eb
results

With this PR ontop of 1.13 and nightly
results

@IanButterworth IanButterworth added packages Package management and loading latency Latency backport 1.13 Change should be backported to release-1.13 labels Aug 31, 2026
@IanButterworth IanButterworth added this to the 1.13 milestone Aug 31, 2026
@KristofferC
KristofferC merged commit 21a70e4 into JuliaLang:master Sep 1, 2026
11 of 12 checks passed
@KristofferC KristofferC mentioned this pull request Sep 1, 2026
37 tasks
KristofferC pushed a commit that referenced this pull request Sep 1, 2026
998cb27 introduced a regression that, when invoking a CodeInstance
from an external image, would result in the generation of a tojlinvoke
trampoline for every invoke but the first (in the order `compile!`
infers them).

```julia
module P2

function what(x)
    try
        show(x)
    catch e
        rethrow()
    end
end

what(1) # calls julia_rethrow_88_gfthunk
what(:a) # calls julia_rethrow_88_gfthunk
what('a') # calls jlpkg_rethrow_0

end # module P2
```

After inferring the first use of `rethrow` in this example, `compile!` would
find no source for the CodeInstance and infer it again, with
`source_mode=SOURCE_MODE_GET_SOURCE`.  It then inserts the resulting new
CodeInstance into the `code_cache`.  Before 998cb27, this went into the
InternalCodeCache, which would insert it after any CodeInstances with the
`invoke` field set.  Now, it is inserted into the OverlayCodeCache, which takes
priority over the InternalCodeCache.  For the two remaining specializations of
`what`, this new CodeInstance is used for the invoke.

Since it's a waste to re-infer this CodeInstance when we have
`external_linkage`, it makes more sense to thread this through to `compile!`
and skip any CodeInstances that are from the image.

Thanks to mlechu for helping track down this issue and providing the
MWE.

(cherry picked from commit 21a70e4)

Backport note: the `src/aotcompile.cpp` hunk (`jl_get_ci_equiv_range` in
`aot_link_output`) was dropped — it depends on the `jl_codegen_output_t`
link-step refactor and `emit_pkg_plt_thunk` infrastructure that does not
exist on 1.13, where from-image `CodeInstance`s are linked externally at
codegen time in `emit_invoke` instead. The `collectinvokes!` condition was
adapted to 1.13's simpler enqueue logic.
IanButterworth added a commit to IanButterworth/Julia-TTFX-Snippets that referenced this pull request Sep 1, 2026
…build

Both dev arms are official binaries carrying the JuliaLang/julia#61479 fix:
nightly 1.14.0-DEV.3081 and `juliaup add pr62959` (1.13.0-rc3.53) labelled
1.13. The 1.13 arm additionally has JuliaLang/julia#62891, so it precompiles
with all 15 cores on Apple Silicon where earlier arms used 5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VzpF8n8QdxxtrbpjgEgW6J
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 1.13 Change should be backported to release-1.13 latency Latency packages Package management and loading

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants