Skip to content

gh-140936: Fix JIT assertion crash at finalization if some generator is alive #140969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

gh-140936: Fix JIT assertion crash at finalization if some generator is alive #140969

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cc1f1e9
Possible solution
efimov-mikhail Nov 3, 2025
937b990
Test added
efimov-mikhail Nov 3, 2025
688f063
Another solution
efimov-mikhail Nov 3, 2025
39f9bc3
New line fix in test
efimov-mikhail Nov 3, 2025
28c07d6
Simplify test
efimov-mikhail Nov 4, 2025
e7a261a
Test improvement
efimov-mikhail Nov 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion Lib/test/test_capi/test_opt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@
import gc
import os
import types
import tempfile
import subprocess

import _opcode

from test.support import (script_helper, requires_specialization,
import_helper, Py_GIL_DISABLED, requires_jit_enabled,
reset_code)
reset_code, requires_subprocess)

_testinternalcapi = import_helper.import_module("_testinternalcapi")

Expand Down Expand Up @@ -2660,7 +2662,38 @@ def f():

f()

@requires_subprocess()
def test_interpreter_finalization_with_generator_alive(self):
code = textwrap.dedent("""
import sys
def simple_for():
for x in (1, 2):
Copy link
Member

@Fidget-Spinner Fidget-Spinner Nov 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The length of this tuple must be >= SPECIALIZATION_THRESHOLD and < JIT_BACKWARD_SPECIALIZATION_COUNTER (names are wrong but they are in pycore_backoff.h). Could you somehow make it so that we rely on those numbers instead of this? Otherwise the test will break if we change the specialization threshold or JIT threshold.

Copy link
Member Author

@efimov-mikhail efimov-mikhail Nov 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we can't use something bigger than SPECIALIZATION_THRESHOLD, it breaks the test with no assertion.
But we can use exactly SPECIALIZATION_THRESHOLD value.

I've checked that new variant of test is worked properly with any values of ADAPTIVE_WARMUP_VALUE. I've checked 1, 2, 3, 6, 100, 1000, 4000. And since SPECIALIZATION_THRESHOLD = ADAPTIVE_WARMUP_VALUE + 1, this test is now correct, I hope.

x

def gen():
try:
yield
except:
simple_for()

sys.settrace(lambda *args: None)
simple_for()
g = gen()
next(g)
print("finished", end='')
""")

tmp = tempfile.NamedTemporaryFile(delete=False, suffix='.py')
tmp.write(code.encode('utf-8'))
tmp.close()
try:
p = subprocess.Popen([sys.executable, tmp.name], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
p.wait()
out = p.stdout.read()
finally:
os.remove(tmp.name)
p.stdout.close()
self.assertEqual(b"finished", out)

def global_identity(x):
return x
Expand Down
4 changes: 3 additions & 1 deletion Python/optimizer.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,9 @@ _PyOptimizer_Optimize(
{
_PyStackRef *stack_pointer = frame->stackpointer;
PyInterpreterState *interp = _PyInterpreterState_GET();
assert(interp->jit);
if (!interp->jit) {
return 0;
}
assert(!interp->compiling);
#ifndef Py_GIL_DISABLED
interp->compiling = true;
Expand Down
Loading