AddressSanitizer: heap-buffer-overflow Python/optimizer.c:709:39 in _PyJit_translate_single_bytecode_to_trace

[YuanchengJiang]

Crash report

What happened?

The following code:

import json
import sys
import threading

sys.setswitchinterval(1e-6)   # makes the race frequent; it also happens without this

def worker(data, index):
    while data:
        for d in list(data):
            try:
                if len(d) > 5:
                    d.pop(next(iter(d)), None)
                else:
                    d[index] = index
            except Exception:
                pass

data = [{}, {}]
threads = [threading.Thread(target=worker, args=(data, i), daemon=True) for i in range(4)]
for t in threads:
    t.start()
for _ in range(200):
    try:
        json.dumps(data)
    except Exception:
        pass
data.clear()
for t in threads:
    t.join(1.0)
print("done")

Resulted in this output (roughly 1 run in 8; it is a race):

==1526716==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7309e9c7e398 at pc 0x5b9774fd19f9 bp 0x72a9e3cdd030 sp 0x72a9e3cdd028
READ of size 8 at 0x7309e9c7e398 thread T4
    #0 0x5b9774fd19f8 in _PyJit_translate_single_bytecode_to_trace Python/optimizer.c:709:39
    #1 0x5b9774afa38f in _PyEval_EvalFrameDefault Python/generated_cases.c.h:13173:25
    #2 0x5b9774aed857 in _PyEval_Vector Python/ceval.c:2176:12
    #3 0x5b97745a9b8d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:144:11
    #4 0x5b97745a9b8d in _PyObject_VectorcallPrepend Objects/call.c:855:20
    #5 0x5b9774c2d58a in _PyObject_VectorcallTstate Include/internal/pycore_call.h:144:11
    #6 0x5b9774c2d58a in context_run Python/context.c:802:29
    #7 0x5b9774aef7a5 in _PyCallMethodDescriptorFastWithKeywords_StackRef Python/ceval.c:885:11
    #8 0x5b9774b2afc4 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:4205:35
    #9 0x5b9774aed857 in _PyEval_Vector Python/ceval.c:2176:12
    #10 0x5b97745a9b8d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:144:11
    #11 0x5b97745a9b8d in _PyObject_VectorcallPrepend Objects/call.c:855:20
    #12 0x5b97745a613d in _PyVectorcall_Call Objects/call.c:273:16
    #13 0x5b97753c7700 in thread_run Modules/_threadmodule.c:388:21
    #14 0x5b977512a18b in pythread_wrapper Python/thread_pthread.h:236:5

0x7309e9c7e398 is located 472 bytes after 64-byte region [0x7309e9c7e180,0x7309e9c7e1c0)
allocated by thread T3 here:
    #0 malloc
    #1 _PyMem_DebugRawAlloc Objects/obmalloc.c:3103:24
    #2 _PyMem_DebugRawRealloc Objects/obmalloc.c:3179:16
    #3 get_index_for_executor Python/optimizer.c:77:33
    #4 _PyOptimizer_Optimize Python/optimizer.c:173:21
    #5 stop_tracing_and_jit Python/ceval.c:1092:15
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:13179:27

SUMMARY: AddressSanitizer: heap-buffer-overflow Python/optimizer.c:709:39 in _PyJit_translate_single_bytecode_to_trace

What happens

_PyJit_translate_single_bytecode_to_trace (Python/optimizer.c:700-713) pairs an operand
recorded earlier with an opcode read now:

int oparg = tracer->prev_state.instr_oparg;   // recorded after the instruction ran
int opcode = this_instr->op.code;             // read from the code object now
...
if (opcode == ENTER_EXECUTOR) {
    _PyExecutorObject *executor = old_code->co_executors->executors[oparg & 255];

The tail of every traced instruction stores prev_state.instr = next_instr and
prev_state.instr_oparg = oparg (Python/bytecodes.c, the tracing epilogue), and the
instruction is only translated on the next epilogue, after the following instruction has
run. If the GIL changes hands in that window and another thread finishes its own trace of the
same loop, insert_executor rewrites that instruction in place to ENTER_EXECUTOR with
op.arg = <executor index>. The first thread then sees ENTER_EXECUTOR but still uses the
JUMP_BACKWARD distance it recorded as the executor index.

Captured under gdb at the faulting line, from the same program:

oparg (tracer->prev_state.instr_oparg) = 66
this_instr->op                          = {code = ENTER_EXECUTOR, arg = 1}
*old_code->co_executors                 = {size = 2, capacity = 4, ...}
old_code->co_name = "worker", instruction offset 102

and dis of worker at byte offset 204 (code unit 102) is JUMP_BACKWARD 66. So the
recorded operand is the jump distance, the live instruction is ENTER_EXECUTOR 1, and
executors[66] is read from a 2-element array.

Recording the opcode together with the operand in prev_state (and aborting the trace if the
live opcode no longer matches), or taking the executor index from the live op.arg instead of
the recorded operand, would close the window.

To reproduce:

./python min.py      # repeat; ~10% of runs. Same with -OO. Without the setswitchinterval line it is rare.

Commit:

e682b4478c2ad09de0bbe821127f451f02f6e5ca  (main, 2026-09-16, "Fix comment formatting in dictobject.c (#157597)")

Build configuration:

../configure --with-pydebug --enable-experimental-jit=yes --with-address-sanitizer --with-undefined-behavior-sanitizer
CC=clang-21 (Clang 21.1.8); default GIL build (not free-threaded)

Operating System:

Ubuntu 22.04.5 LTS, x86_64, in Docker (image fusion-fuzz-cpython:latest)

This bug was found by fusion-fuzz

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response