make write barrier field aware - #62737
Draft
oscardssmith wants to merge 5 commits into
Draft
Conversation
topolarity
requested changes
Aug 20, 2026
topolarity
left a comment
topolarity
left a comment
Member
There was a problem hiding this comment.
The comments need to be rewritten by a human and the combined julia.write_barrier needs re-work I think but otherwise this looks largely OK to me
| // `parent` is younger than the last safepoint. A generational plan need not remember it. | ||
| // A snapshot barrier need not either: marking can only have begun at a safepoint, so a | ||
| // live snapshot cannot contain any field of `parent`, and the values being displaced are | ||
| // the uninitialised ones the allocator left behind. |
Member
There was a problem hiding this comment.
Is there a reason to include all these comments here instead of in gc-interface.h?
| // `slot` is the address of the field being written, or a null pointer where the | ||
| // caller cannot name a single field (whole-object stores, deletion barriers, and | ||
| // array copies). Only plans with a field-granularity barrier look at it; the rest | ||
| // key off `parent` alone. |
Member
There was a problem hiding this comment.
Should this be pairs of slot, child, slot, child, ...? It's not clear to me how the multi-child case can successfully provide field information - is it being deleted right now?
Member
There was a problem hiding this comment.
Another option is to describe this as a span of contiguous fields, which could be more sensible
Member
There was a problem hiding this comment.
Claude shares these references from the MMTk LXR branch:
- HotSpot emits one call per oop arraycopy —
arraycopy_prologuein the LXR field barrier assembler:
https://github.com/wenyuzhao/mmtk-openjdk/blob/0682434ae725787e38defbc17bd66dc918e4bdb7/openjdk/barriers/mmtkFieldBarrier.cpp#L130-L148 - Binding FFI shim —
mmtk_array_copy_preconverts (src, dst, count) into two address-range slices and calls the generic slice barrier:
https://github.com/wenyuzhao/mmtk-openjdk/blob/0682434ae725787e38defbc17bd66dc918e4bdb7/mmtk/src/api.rs#L477-L491 - Generic FieldBarrier routes
memory_region_copy_preto the plan's slow path:
https://github.com/wenyuzhao/mmtk-core/blob/9625c174f3a3d226fee460afe62e9c4b581f6044/src/plan/barriers.rs#L335-L341 - The LXR bulk optimization itself — the u128 unlog-bit scan with the per-slot fallback (this is the interesting one):
https://github.com/wenyuzhao/mmtk-core/blob/9625c174f3a3d226fee460afe62e9c4b581f6044/src/plan/lxr/barrier.rs#L209-L232
just for your record / inspiration
| // | ||
| // The merged call keeps the slot only when the barriers named the same field, and | ||
| // otherwise names the parent alone -- always a safe over-approximation, since a null slot | ||
| // means "something in this object changed". A field-granularity plan that wants to keep |
Member
There was a problem hiding this comment.
Ah yes, we are awkwardly dropping field information here
oscardssmith
force-pushed
the
os/wb-field-slot
branch
2 times, most recently
from
August 27, 2026 22:06
0ff9e88 to
9056272
Compare
topolarity
reviewed
Aug 31, 2026
topolarity
force-pushed
the
os/wb-field-slot
branch
from
August 31, 2026 16:10
6d9ebe7 to
fa74ff3
Compare
The `jl_gc_wb_current_task` annotations are no-ops today, but they mark where a real write barrier would have to sit, and a collector that must observe the displaced reference reads it through the task's still-current fields -- the barrier has to run before the store it annotates. Reorder the `bound_cancel_token` restores in `jl_eh_restore_state`, `jl_eh_restore_state_noexcept`, and the finalizer-block restore so the annotations sit where a real barrier would be sound. Assisted-by: Claude Code (Fable 5)
topolarity
force-pushed
the
os/wb-field-slot
branch
from
September 1, 2026 01:17
fa74ff3 to
d09ea0a
Compare
`jl_gc_wb_fresh`, `jl_gc_wb_current_task` and `jl_gc_wb_knownold` mark stores where a write barrier would ordinarily be required and state the property that lets a collector do less than the full `jl_gc_wb`. Every other barrier in gc-interface.h is declared there and defined by the collector in gc-wb-stock.h or gc-wb-mmtk.h; these three were instead given no-op bodies in the interface itself, which bakes one collector's answer into the question. Declare them like the rest and move the no-ops into gc-wb-stock.h, where they belong and where the generational reasoning that justifies them holds. That distinction matters because each property is only an argument about the value being *stored*: it is old, or perm-rooted, or the parent is already in a remset. A snapshot barrier also has to observe the value being *displaced*, and two of the three say nothing about that. `jl_gc_wb_current_task` is the sharper case -- being in a remset means the parent gets rescanned, which recovers references inserted into it but not references removed from it, since the rescan sees the field after the store. A reference moved out of the current task into an already-blackened object is then reachable only through a location the snapshot never saw. So MMTk defines those two as real barriers under MMTK_SNAPSHOT_BARRIER, and `jl_gc_wb_fresh` as a no-op for every plan: marking can only begin at a safepoint, so no field of a parent allocated since the last one can appear in a live snapshot. The three places that elide or hoist a write barrier when only the newly stored value is considered were keyed on `MMTK_PLAN_CONCURRENTIMMIX`. What actually makes them invalid is not that plan in particular but the property that the barrier has to observe every overwritten reference, which ConcurrentImmix needs for its SATB snapshot and which other plans need for other reasons. Introduce `MMTK_SNAPSHOT_BARRIER` for that property and key the guards on it. No functional change: ConcurrentImmix is the only plan that defines it. The pointer path moved the references and then invoked `jl_gc_wb_genericmemory_copy_ptr`, so a barrier that reads the destination to learn which references were displaced saw the values that had just overwritten them. Under a snapshot barrier that loses both halves of the update: the references copied in are never recorded, and the ones they displaced are never released. Invoke the barrier first, matching the ordering the boxed path above already uses. The source range the barrier reads is unaffected by the move, and the destination holds its previous, still-valid contents while the barrier runs. Assisted-by: Claude Code (Opus 5)
The write-barrier entry points for genericmemory element spans were inconsistent: `jl_gc_wb_genericmemory_copy_boxed` performed part of the copy itself (handing the remainder back to the caller through in-out pointers), while `jl_gc_wb_genericmemory_copy_ptr` only observed a copy the caller performs afterwards. Settle on one convention and name it honestly: a `jl_gc_wb_*` entry only observes a write its caller performs, and `jl_gc_genericmemory_*` operations perform the mutation themselves, fused with whatever barrier work the collector needs. A span's values cannot be passed by value the way a single store's value can, and fusing the operation is what lets a collector observe each value exactly once. `jl_gc_genericmemory_copy_boxed` and `jl_gc_genericmemory_copy_ptr` now perform the whole copy. For the inline-pointer copy this also closes a soundness race: the old barrier walked the *source* before the caller's `memmove_refs` re-read it, so a store racing with the copy could replace an old reference with a young one between the two reads, landing a young object in an old destination with no record on either side. The fused operation copies first and makes its queueing decisions from the values it actually stored. The old-source fast path survives: values copied from an old source were old, and any younger value that appears in the destination afterwards was put there by a store that carries its own barrier. `jl_gc_genericmemory_clear` is the deletion-side sibling, used by `jl_array_del_end`: clearing inserts no references, so a collector that records insertions implements it as a plain memset, while a snapshotting collector logs the overwritten span before the clear. `memmove_refs` moves to julia.h so the fused entries, which live in the public GC headers, can use it. Assisted-by: Claude Code (Fable 5)
topolarity
force-pushed
the
os/wb-field-slot
branch
from
September 1, 2026 02:49
d09ea0a to
c3d8e82
Compare
Member
|
This now contains a few fixes for the existing barriers, both stock GC + MMTk I'll try to split those out tomorrow. |
A collector that records the field being written, rather than the object containing it, has to be told which field that is. Nothing downstream of the store can recover that address, so thread it through both barrier layers. The C runtime's `jl_gc_wb` takes the written field's address as a second argument, and the slot is required: a collector may record the field, the containing object, or both, and a wrong slot is not merely slower, it is wrong. Sites that clear several named fields emit one barrier per slot, and dynamic element spans already go through the fused `jl_gc_genericmemory_*` operations of the previous commit. The one write with no nameable slot gets a dedicated entry: module `usings` lists live in malloc'd memory with no per-slot metadata, so `jl_gc_wb_module_usings` records the module, the only loggable location. The gcext test follows the `jl_gc_wb` signature as an external consumer, naming its true slots. The IR-level barrier splits by granularity claim instead of overloading one variadic intrinsic with a sentinel-null slot: - `julia.object_write_barrier(parent, children...)`: the whole parent counts as modified. This is the degradation target for stores whose written locations cannot be named -- a field-granularity collector must treat every field of the parent as written, which for a Memory parent can mean scanning the whole array -- and it is only ever materialized by passes, since transformations can forget which slot was written but never invent one. The JuliaLICM hoist demotes a field barrier whose slot is loop-varying to an object barrier carrying all of its children. - `julia.field_write_barrier.pN(parent, slot, child, ...)`: the named fields were written; the tail is additional (slot, child) pairs for a store that writes several fields at once (an inline composite). Slots are never null (the GC invariant verifier enforces the pair arity and non-null slots); a store that cannot name its fields takes the object barrier instead. The `.pN` suffix is monomorphization mangling, spelled the way LLVM's intrinsic mangler spells pointer overloads: the slot's address space is an encoding artifact, but a declared function is monomorphic in its pointer types and the GC invariant verifier permits no cast between the Derived (.p11, object-interior slots) and Loaded (.p13, array-data slots) worlds. Both monomorphizations have identical semantics and identical lowerings. Routing Loaded slots to `.p13` makes compiled array-element stores field-precise instead of degrading them to whole-object barriers. The pairs form is what keeps composite stores affordable everywhere. Emitting one field barrier per slot measured at ~2.8x the barrier-path cost of the object barrier on Vector-of-struct fills, for every collector class, and no optimizer can fold slot-keyed checks across calls: the metadata addressing is nonlinear, and alignment facts neither exist for array elements nor survive `julia.gc_loaded`. One pairs call per store instead lowers on parent-keyed collectors to byte-identical machine code with the object barrier, confirmed cost-neutral under layout-randomized benchmarking, while a slot-keyed lowering can coalesce the pair checks into a single wide masked test of the slots' metadata bits, since their relative offsets are compile-time constants. Both composite emission paths (materialized aggregates and split-represented values) emit pairs whenever the store address and layout are known; read-once safety holds because shared sources are loaded exactly once into private staging before either the barrier or the store consumes them. There is deliberately no span-granularity barrier intrinsic: a span mutation that reads its values from memory must make each barrier decision on a value read exactly once, which means fusing the barrier with the operation; those fused operations live behind the C runtime (`jl_gc_genericmemory_copy_boxed` and friends). Compiled clears with statically-known layout decompose into field barriers with null children, which the stock lowering elides statically. Renaming `julia.write_barrier` is a deliberate break for external IR producers: they must now choose a granularity rather than silently taking whole-object barriers, which can be very slow under snapshotting collectors. CancellationLowering's name matches follow the rename. Assisted-by: Claude Code (Opus 5, Fable 5)
CancellationLowering matched `julia.gc_alloc_obj`, the write barriers, `julia.pointer_from_objref`, and `julia.gc_loaded` by scanning callee names, including a substring match that any function containing "write_barrier" would satisfy. Inherit JuliaPassContext and compare against its function pointers through the family predicates instead. The setjmp/safepoint skip keeps its name match: it pairs a C symbol with calls this pass itself creates mid-run, before any declaration necessarily exists at context-initialization time. Assisted-by: Claude Code (Fable 5)
topolarity
force-pushed
the
os/wb-field-slot
branch
from
September 1, 2026 03:47
c3d8e82 to
1ac1583
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This should make generational concurrent immix and LXR a lot better.