-
Notifications
You must be signed in to change notification settings - Fork 467
chore(iast): fix iast pytest collection error #15204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 237 ± 2 ms. The average import time from base is: 241 ± 4 ms. The import time difference between this PR and base is: -3.3 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate avara1986/APPSEC-59497_pytest_error_signature (4ee4944) with baseline main (1c3361c) 🟡 Near SLO Breach (4 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 20.464ms (SLO: <22.300ms -8.2%) vs baseline: ~same Memory: ✅ 66.080MB (SLO: <67.000MB 🟡 -1.4%) vs baseline: +5.0% ✅ exception-replay-enabledTime: ✅ 1.339ms (SLO: <1.450ms -7.6%) vs baseline: -0.9% Memory: ✅ 64.247MB (SLO: <67.000MB -4.1%) vs baseline: +4.7% ✅ iastTime: ✅ 20.466ms (SLO: <22.250ms -8.0%) vs baseline: +0.1% Memory: ✅ 66.082MB (SLO: <67.000MB 🟡 -1.4%) vs baseline: +5.0% ✅ profilerTime: ✅ 15.532ms (SLO: <16.550ms -6.2%) vs baseline: ~same Memory: ✅ 54.078MB (SLO: <54.500MB 🟡 -0.8%) vs baseline: +5.2% ✅ resource-renamingTime: ✅ 20.520ms (SLO: <21.750ms -5.7%) vs baseline: ~same Memory: ✅ 66.178MB (SLO: <67.000MB 🟡 -1.2%) vs baseline: +5.1% ✅ span-code-originTime: ✅ 25.351ms (SLO: <28.200ms 📉 -10.1%) vs baseline: -0.4% Memory: ✅ 68.266MB (SLO: <69.500MB 🟡 -1.8%) vs baseline: +5.9% ✅ tracerTime: ✅ 20.553ms (SLO: <21.750ms -5.5%) vs baseline: ~same Memory: ✅ 66.088MB (SLO: <67.000MB 🟡 -1.4%) vs baseline: +5.0% ✅ tracer-and-profilerTime: ✅ 22.663ms (SLO: <23.500ms -3.6%) vs baseline: -0.3% Memory: ✅ 67.894MB (SLO: <68.000MB 🟡 -0.2%) vs baseline: +5.1% ✅ tracer-dont-create-db-spansTime: ✅ 19.350ms (SLO: <21.500ms -10.0%) vs baseline: ~same Memory: ✅ 66.151MB (SLO: <67.000MB 🟡 -1.3%) vs baseline: +4.9% ✅ tracer-minimalTime: ✅ 16.627ms (SLO: <17.500ms -5.0%) vs baseline: ~same Memory: ✅ 66.159MB (SLO: <67.000MB 🟡 -1.3%) vs baseline: +5.1% ✅ tracer-nativeTime: ✅ 20.438ms (SLO: <21.750ms -6.0%) vs baseline: -0.4% Memory: ✅ 67.987MB (SLO: <72.500MB -6.2%) vs baseline: +5.0% ✅ tracer-no-cachesTime: ✅ 18.527ms (SLO: <19.650ms -5.7%) vs baseline: +0.5% Memory: ✅ 66.030MB (SLO: <67.000MB 🟡 -1.4%) vs baseline: +4.8% ✅ tracer-no-databasesTime: ✅ 18.780ms (SLO: <20.100ms -6.6%) vs baseline: -0.1% Memory: ✅ 65.945MB (SLO: <67.000MB 🟡 -1.6%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 20.179ms (SLO: <21.500ms -6.1%) vs baseline: +0.4% Memory: ✅ 65.942MB (SLO: <67.000MB 🟡 -1.6%) vs baseline: +4.7% ✅ tracer-no-templatesTime: ✅ 20.257ms (SLO: <22.000ms -7.9%) vs baseline: ~same Memory: ✅ 66.055MB (SLO: <67.000MB 🟡 -1.4%) vs baseline: +4.6% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.043ms (SLO: <19.850ms -9.1%) vs baseline: +0.3% Memory: ✅ 65.961MB (SLO: <66.500MB 🟡 -0.8%) vs baseline: +4.9% ✅ errortracking-enabled-userTime: ✅ 18.116ms (SLO: <19.400ms -6.6%) vs baseline: +0.4% Memory: ✅ 66.003MB (SLO: <66.500MB 🟡 -0.7%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 18.082ms (SLO: <19.450ms -7.0%) vs baseline: +0.2% Memory: ✅ 65.914MB (SLO: <66.500MB 🟡 -0.9%) vs baseline: +4.8% 🟡 errortrackingflasksqli - 6/6✅ errortracking-enabled-allTime: ✅ 2.070ms (SLO: <2.300ms 📉 -10.0%) vs baseline: -0.4% Memory: ✅ 52.612MB (SLO: <53.500MB 🟡 -1.7%) vs baseline: +4.9% ✅ errortracking-enabled-userTime: ✅ 2.072ms (SLO: <2.250ms -7.9%) vs baseline: ~same Memory: ✅ 52.652MB (SLO: <53.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 2.074ms (SLO: <2.300ms -9.8%) vs baseline: ~same Memory: ✅ 52.593MB (SLO: <53.500MB 🟡 -1.7%) vs baseline: +5.0% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 4.595ms (SLO: <4.750ms -3.3%) vs baseline: -0.2% Memory: ✅ 62.317MB (SLO: <65.000MB -4.1%) vs baseline: +4.7% ✅ appsec-postTime: ✅ 6.623ms (SLO: <6.750ms 🟡 -1.9%) vs baseline: -0.1% Memory: ✅ 62.397MB (SLO: <65.000MB -4.0%) vs baseline: +5.0% ✅ appsec-telemetryTime: ✅ 4.593ms (SLO: <4.750ms -3.3%) vs baseline: ~same Memory: ✅ 62.365MB (SLO: <65.000MB -4.1%) vs baseline: +4.8% ✅ debuggerTime: ✅ 1.856ms (SLO: <2.000ms -7.2%) vs baseline: -0.3% Memory: ✅ 45.275MB (SLO: <47.000MB -3.7%) vs baseline: +4.9% ✅ iast-getTime: ✅ 1.878ms (SLO: <2.000ms -6.1%) vs baseline: +1.2% Memory: ✅ 42.069MB (SLO: <49.000MB 📉 -14.1%) vs baseline: +4.9% ✅ profilerTime: ✅ 1.919ms (SLO: <2.100ms -8.6%) vs baseline: +0.3% Memory: ✅ 46.631MB (SLO: <47.000MB 🟡 -0.8%) vs baseline: +4.9% ✅ resource-renamingTime: ✅ 3.370ms (SLO: <3.650ms -7.7%) vs baseline: +0.3% Memory: ✅ 52.569MB (SLO: <53.500MB 🟡 -1.7%) vs baseline: +4.8% ✅ tracerTime: ✅ 3.351ms (SLO: <3.650ms -8.2%) vs baseline: -0.3% Memory: ✅ 52.554MB (SLO: <53.500MB 🟡 -1.8%) vs baseline: +4.8% ✅ tracer-nativeTime: ✅ 3.354ms (SLO: <3.650ms -8.1%) vs baseline: ~same Memory: ✅ 54.309MB (SLO: <60.000MB -9.5%) vs baseline: +4.8%
|
What Does This Do?
This PR continues the work of the following PRs
#15175
#15141
Fixes a bug where IAST's
post_preload()function drops theinspectmodule fromsys.modules, causing pytest's test collection phase to fail with signature introspection errorsMotivation
When IAST is enabled and
post_preload()drops theinspectmodule from memory, pytest encounters errors during test collection:Root Cause
The issue occurs because:
inspectearly and createsSignatureobjects during its initializationpost_preload()removesinspectfromsys.modulesto avoid Gevent conflictsinspectwhen it continues to use inspect functionsSignatureclass from the old module doesn't match the new oneThis is a module identity mismatch problem:
old_inspect.Signature≠new_inspect.SignatureWhen pytest tries to validate signatures with
isinstance(sig, inspect.Signature), it fails because the signature object's class is from the old (deleted) module, butinspect.Signaturerefers to the class from the new (reimported) module.Gevent Compatibility Note
While the original implementation dropped
inspectto prevent Gevent conflicts, this created a worse problem by breaking pytest entirely.Trade-off Decision:
inspect) are rare and workaroundableIf Gevent issues resurface with the
inspectmodule, they should be addressed through:inspectwithin IAST functions (as originally intended)