Skip to content

Add line probe exploration tests #8741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Add line probe exploration tests #8741

wants to merge 1 commit into from

Conversation

jpbempel
Copy link
Member

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Apr 28, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1745844142 1745844511
end_time 2025-04-28T12:43:41 2025-04-28T12:49:48
git_branch master jpbempel/itw-line-probe
git_commit_sha c55dc46 b995155
start_time 2025-04-28T12:42:23 2025-04-28T12:48:32
See matching parameters
Baseline Candidate
ci_job_id 914759253 914759253
ci_pipeline_id 63481469 63481469
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1745842799 1745842799

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-20846.877ns; +20957.531ns] or [-7.633%; +7.674%]		 unstable
[-33.768µs; +31.176µs] or [-10.797%; +9.968%]		 unstable
[-44.746µs; +42.621µs] or [-13.680%; +13.030%]		 unstable
[-102.193µs; +145.072µs] or [-16.953%; +24.067%]		 same
scenario:basic unsure
[+1.125µs; +7.388µs] or [+0.403%; +2.646%]		 same same unstable
[+18.704µs; +87.761µs] or [+3.718%; +17.444%]		 unstable
[-150.011op/s; +150.011op/s] or [-6.000%; +6.000%]
scenario:loop unsure
[-18.154µs; -5.054µs] or [-0.182%; -0.051%]		 same same same same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (312.741 µs) : 288, 337
.   : milestone, 313,
basic (308.314 µs) : 300, 316
.   : milestone, 308,
loop (10.131 ms) : 10108, 10154
.   : milestone, 10131,
section candidate
noprobe (311.445 µs) : 275, 348
.   : milestone, 311,
basic (315.048 µs) : 306, 324
.   : milestone, 315,
loop (10.124 ms) : 10094, 10154
.   : milestone, 10124,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 312.741 µs [288.186 µs, 337.297 µs]
basic 308.314 µs [300.406 µs, 316.222 µs]
loop 10.131 ms [10.108 ms, 10.154 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 311.445 µs [275.22 µs, 347.671 µs]
basic 315.048 µs [305.925 µs, 324.171 µs]
loop 10.124 ms [10.094 ms, 10.154 ms]

@pr-commenter
Copy link

pr-commenter bot commented Apr 28, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/itw-line-probe
git_commit_date 1745843547 1745842799
git_commit_sha c55dc46 b995155
release_version 1.49.0-SNAPSHOT~c55dc46d28 1.49.0-SNAPSHOT~b99515547d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745845870 1745845870
ci_job_id 914759247 914759247
ci_pipeline_id 63481469 63481469
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ekfrqbyz-project-304-concurrent-0-3125bxqd 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ekfrqbyz-project-304-concurrent-0-3125bxqd 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 7 performance improvements and 22 performance regressions! Performance is the same for 32 metrics, 10 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start worse
[+46.086ms; +77.051ms] or [+4.070%; +6.805%]		 1.194s 1.132s
scenario:startup:insecure-bank:iast:BytebuddyAgent worse
[+47.469ms; +72.324ms] or [+6.032%; +9.190%]		 846.896ms 787.000ms
scenario:startup:insecure-bank:iast:Debugger better
[-1.695ms; -1.469ms] or [-29.052%; -25.178%]		 4.253ms 5.836ms
scenario:startup:insecure-bank:iast:Telemetry worse
[+664.622µs; +1064.044µs] or [+8.392%; +13.435%]		 8.784ms 7.920ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:BytebuddyAgent worse
[+23.347ms; +55.513ms] or [+2.919%; +6.942%]		 839.154ms 799.724ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:AppSec better
[-7.057ms; -1.520ms] or [-12.429%; -2.678%]		 52.488ms 56.776ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Debugger better
[-1.819ms; -1.558ms] or [-30.692%; -26.292%]		 4.237ms 5.926ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Telemetry worse
[+475.081µs; +876.125µs] or [+5.935%; +10.946%]		 8.680ms 8.004ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start worse
[+28.122ms; +69.397ms] or [+2.466%; +6.085%]		 1.189s 1.140s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:BytebuddyAgent worse
[+34.927ms; +69.738ms] or [+4.410%; +8.806%]		 844.307ms 791.975ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Debugger better
[-1.816ms; -1.589ms] or [-30.778%; -26.939%]		 4.198ms 5.900ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry worse
[+613.964µs; +1022.058µs] or [+7.868%; +13.097%]		 8.622ms 7.804ms
scenario:startup:insecure-bank:tracing:Agent.start worse
[+29.881ms; +64.672ms] or [+2.949%; +6.382%]		 1.061s 1.013s
scenario:startup:insecure-bank:tracing:BytebuddyAgent worse
[+37.802ms; +64.085ms] or [+5.601%; +9.496%]		 725.811ms 674.867ms
scenario:startup:petclinic:appsec:Agent.start worse
[+31.634ms; +62.568ms] or [+2.742%; +5.423%]		 1.201s 1.154s
scenario:startup:petclinic:appsec:BytebuddyAgent worse
[+40.638ms; +60.436ms] or [+5.878%; +8.742%]		 741.865ms 691.328ms
scenario:startup:petclinic:appsec:Debugger better
[-1.756ms; -1.636ms] or [-29.714%; -27.681%]		 4.213ms 5.909ms
scenario:startup:petclinic:iast:Agent.start worse
[+42.889ms; +77.582ms] or [+3.774%; +6.827%]		 1.197s 1.136s
scenario:startup:petclinic:iast:BytebuddyAgent worse
[+47.355ms; +73.107ms] or [+6.005%; +9.270%]		 848.872ms 788.641ms
scenario:startup:petclinic:iast:Debugger better
[-1.824ms; -1.539ms] or [-30.732%; -25.926%]		 4.255ms 5.936ms
scenario:startup:petclinic:iast:Telemetry worse
[+0.802ms; +1.245ms] or [+10.185%; +15.816%]		 8.894ms 7.871ms
scenario:startup:petclinic:profiling:Agent.start worse
[+51.501ms; +60.159ms] or [+4.120%; +4.812%]		 1.306s 1.250s
scenario:startup:petclinic:profiling:ProfilingAgent worse
[+4.568ms; +6.168ms] or [+4.743%; +6.405%]		 101.673ms 96.305ms
scenario:startup:petclinic:profiling:BytebuddyAgent worse
[+48.454ms; +52.530ms] or [+7.327%; +7.943%]		 711.840ms 661.348ms
scenario:startup:petclinic:profiling:Debugger better
[-1.798ms; -1.669ms] or [-29.611%; -27.484%]		 4.338ms 6.072ms
scenario:startup:petclinic:profiling:Telemetry worse
[+611.888µs; +847.352µs] or [+7.551%; +10.457%]		 8.833ms 8.103ms
scenario:startup:petclinic:profiling:Profiling worse
[+4.569ms; +6.169ms] or [+4.743%; +6.404%]		 101.698ms 96.329ms
scenario:startup:petclinic:tracing:Agent.start worse
[+47.757ms; +65.204ms] or [+4.762%; +6.502%]		 1.059s 1.003s
scenario:startup:petclinic:tracing:BytebuddyAgent worse
[+48.033ms; +64.205ms] or [+7.184%; +9.602%]		 724.778ms 668.659ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.013 s) : 0, 1013386
Total [baseline] (8.634 s) : 0, 8633965
Agent [candidate] (1.061 s) : 0, 1060663
Total [candidate] (8.656 s) : 0, 8656451
section iast
Agent [baseline] (1.132 s) : 0, 1132347
Total [baseline] (9.163 s) : 0, 9162890
Agent [candidate] (1.194 s) : 0, 1193915
Total [candidate] (9.249 s) : 0, 9248542
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.15 s) : 0, 1150132
Total [baseline] (9.169 s) : 0, 9169421
Agent [candidate] (1.184 s) : 0, 1184432
Total [candidate] (9.214 s) : 0, 9214249
section iast_TELEMETRY_OFF
Agent [baseline] (1.14 s) : 0, 1140482
Total [baseline] (9.198 s) : 0, 9198162
Agent [candidate] (1.189 s) : 0, 1189242
Total [candidate] (9.229 s) : 0, 9228893
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.013 s -
Agent iast 1.132 s 118.96 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.15 s 136.746 ms (13.5%)
Agent iast_TELEMETRY_OFF 1.14 s 127.096 ms (12.5%)
Total tracing 8.634 s -
Total iast 9.163 s 528.925 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.169 s 535.456 ms (6.2%)
Total iast_TELEMETRY_OFF 9.198 s 564.197 ms (6.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.194 s 133.253 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.184 s 123.769 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.189 s 128.579 ms (12.1%)
Total tracing 8.656 s -
Total iast 9.249 s 592.091 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.214 s 557.798 ms (6.4%)
Total iast_TELEMETRY_OFF 9.229 s 572.442 ms (6.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.867 ms) : 0, 674867
BytebuddyAgent [candidate] (725.811 ms) : 0, 725811
GlobalTracer [baseline] (241.219 ms) : 0, 241219
GlobalTracer [candidate] (239.447 ms) : 0, 239447
AppSec [baseline] (54.741 ms) : 0, 54741
AppSec [candidate] (55.472 ms) : 0, 55472
Debugger [baseline] (6.894 ms) : 0, 6894
Debugger [candidate] (5.113 ms) : 0, 5113
Remote Config [baseline] (729.255 µs) : 0, 729
Remote Config [candidate] (702.75 µs) : 0, 703
Telemetry [baseline] (11.423 ms) : 0, 11423
Telemetry [candidate] (10.735 ms) : 0, 10735
section iast
BytebuddyAgent [baseline] (787.0 ms) : 0, 787000
BytebuddyAgent [candidate] (846.896 ms) : 0, 846896
GlobalTracer [baseline] (228.948 ms) : 0, 228948
GlobalTracer [candidate] (230.422 ms) : 0, 230422
IAST [baseline] (22.841 ms) : 0, 22841
IAST [candidate] (22.919 ms) : 0, 22919
AppSec [baseline] (55.993 ms) : 0, 55993
AppSec [candidate] (56.495 ms) : 0, 56495
Debugger [baseline] (5.836 ms) : 0, 5836
Debugger [candidate] (4.253 ms) : 0, 4253
Remote Config [baseline] (580.298 µs) : 0, 580
Remote Config [candidate] (668.444 µs) : 0, 668
Telemetry [baseline] (7.92 ms) : 0, 7920
Telemetry [candidate] (8.784 ms) : 0, 8784
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (799.724 ms) : 0, 799724
BytebuddyAgent [candidate] (839.154 ms) : 0, 839154
GlobalTracer [baseline] (232.302 ms) : 0, 232302
GlobalTracer [candidate] (229.791 ms) : 0, 229791
IAST [baseline] (23.149 ms) : 0, 23149
IAST [candidate] (23.804 ms) : 0, 23804
AppSec [baseline] (56.776 ms) : 0, 56776
AppSec [candidate] (52.488 ms) : 0, 52488
Debugger [baseline] (5.926 ms) : 0, 5926
Debugger [candidate] (4.237 ms) : 0, 4237
Remote Config [baseline] (599.716 µs) : 0, 600
Remote Config [candidate] (609.492 µs) : 0, 609
Telemetry [baseline] (8.004 ms) : 0, 8004
Telemetry [candidate] (8.68 ms) : 0, 8680
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (791.975 ms) : 0, 791975
BytebuddyAgent [candidate] (844.307 ms) : 0, 844307
GlobalTracer [baseline] (231.418 ms) : 0, 231418
GlobalTracer [candidate] (229.497 ms) : 0, 229497
IAST [baseline] (22.577 ms) : 0, 22577
IAST [candidate] (23.931 ms) : 0, 23931
AppSec [baseline] (56.746 ms) : 0, 56746
AppSec [candidate] (54.698 ms) : 0, 54698
Debugger [baseline] (5.9 ms) : 0, 5900
Debugger [candidate] (4.198 ms) : 0, 4198
Remote Config [baseline] (594.302 µs) : 0, 594
Remote Config [candidate] (654.545 µs) : 0, 655
Telemetry [baseline] (7.804 ms) : 0, 7804
Telemetry [candidate] (8.622 ms) : 0, 8622
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.003 s) : 0, 1002772
Total [baseline] (10.514 s) : 0, 10514062
Agent [candidate] (1.059 s) : 0, 1059252
Total [candidate] (10.447 s) : 0, 10446761
section appsec
Agent [baseline] (1.154 s) : 0, 1153695
Total [baseline] (10.61 s) : 0, 10610182
Agent [candidate] (1.201 s) : 0, 1200796
Total [candidate] (10.78 s) : 0, 10779801
section iast
Agent [baseline] (1.136 s) : 0, 1136428
Total [baseline] (10.815 s) : 0, 10814841
Agent [candidate] (1.197 s) : 0, 1196663
Total [candidate] (11.026 s) : 0, 11025979
section profiling
Agent [baseline] (1.25 s) : 0, 1250118
Total [baseline] (10.704 s) : 0, 10703548
Agent [candidate] (1.306 s) : 0, 1305948
Total [candidate] (10.879 s) : 0, 10878719
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.003 s -
Agent appsec 1.154 s 150.923 ms (15.1%)
Agent iast 1.136 s 133.656 ms (13.3%)
Agent profiling 1.25 s 247.346 ms (24.7%)
Total tracing 10.514 s -
Total appsec 10.61 s 96.121 ms (0.9%)
Total iast 10.815 s 300.779 ms (2.9%)
Total profiling 10.704 s 189.486 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.201 s 141.543 ms (13.4%)
Agent iast 1.197 s 137.411 ms (13.0%)
Agent profiling 1.306 s 246.695 ms (23.3%)
Total tracing 10.447 s -
Total appsec 10.78 s 333.039 ms (3.2%)
Total iast 11.026 s 579.218 ms (5.5%)
Total profiling 10.879 s 431.958 ms (4.1%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.659 ms) : 0, 668659
BytebuddyAgent [candidate] (724.778 ms) : 0, 724778
GlobalTracer [baseline] (239.541 ms) : 0, 239541
GlobalTracer [candidate] (239.322 ms) : 0, 239322
AppSec [baseline] (54.417 ms) : 0, 54417
AppSec [candidate] (54.53 ms) : 0, 54530
Debugger [baseline] (6.117 ms) : 0, 6117
Debugger [candidate] (5.8 ms) : 0, 5800
Remote Config [baseline] (712.749 µs) : 0, 713
Remote Config [candidate] (684.148 µs) : 0, 684
Telemetry [baseline] (9.954 ms) : 0, 9954
Telemetry [candidate] (10.582 ms) : 0, 10582
section appsec
BytebuddyAgent [baseline] (691.328 ms) : 0, 691328
BytebuddyAgent [candidate] (741.865 ms) : 0, 741865
GlobalTracer [baseline] (237.279 ms) : 0, 237279
GlobalTracer [candidate] (236.078 ms) : 0, 236078
AppSec [baseline] (175.908 ms) : 0, 175908
AppSec [candidate] (175.361 ms) : 0, 175361
Debugger [baseline] (5.909 ms) : 0, 5909
Debugger [candidate] (4.213 ms) : 0, 4213
Remote Config [baseline] (635.065 µs) : 0, 635
Remote Config [candidate] (627.067 µs) : 0, 627
Telemetry [baseline] (8.21 ms) : 0, 8210
Telemetry [candidate] (8.138 ms) : 0, 8138
IAST [baseline] (21.912 ms) : 0, 21912
IAST [candidate] (21.952 ms) : 0, 21952
section iast
BytebuddyAgent [baseline] (788.641 ms) : 0, 788641
BytebuddyAgent [candidate] (848.872 ms) : 0, 848872
GlobalTracer [baseline] (230.393 ms) : 0, 230393
GlobalTracer [candidate] (231.195 ms) : 0, 231195
AppSec [baseline] (56.569 ms) : 0, 56569
AppSec [candidate] (52.27 ms) : 0, 52270
Debugger [baseline] (5.936 ms) : 0, 5936
Debugger [candidate] (4.255 ms) : 0, 4255
Remote Config [baseline] (601.795 µs) : 0, 602
Remote Config [candidate] (624.966 µs) : 0, 625
Telemetry [baseline] (7.871 ms) : 0, 7871
Telemetry [candidate] (8.894 ms) : 0, 8894
IAST [baseline] (22.979 ms) : 0, 22979
IAST [candidate] (22.956 ms) : 0, 22956
section profiling
BytebuddyAgent [baseline] (661.348 ms) : 0, 661348
BytebuddyAgent [candidate] (711.84 ms) : 0, 711840
GlobalTracer [baseline] (373.541 ms) : 0, 373541
GlobalTracer [candidate] (375.251 ms) : 0, 375251
AppSec [baseline] (54.09 ms) : 0, 54090
AppSec [candidate] (53.226 ms) : 0, 53226
Debugger [baseline] (6.072 ms) : 0, 6072
Debugger [candidate] (4.338 ms) : 0, 4338
Remote Config [baseline] (632.303 µs) : 0, 632
Remote Config [candidate] (651.687 µs) : 0, 652
Telemetry [baseline] (8.103 ms) : 0, 8103
Telemetry [candidate] (8.833 ms) : 0, 8833
ProfilingAgent [baseline] (96.305 ms) : 0, 96305
ProfilingAgent [candidate] (101.673 ms) : 0, 101673
Profiling [baseline] (96.329 ms) : 0, 96329
Profiling [candidate] (101.698 ms) : 0, 101698
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-28T12:41:32 2025-04-28T12:49:14
git_branch master jpbempel/itw-line-probe
git_commit_date 1745843547 1745842799
git_commit_sha c55dc46 b995155
release_version 1.49.0-SNAPSHOT~c55dc46d28 1.49.0-SNAPSHOT~b99515547d
start_time 2025-04-28T12:41:18 2025-04-28T12:49:00
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745844951 1745844951
ci_job_id 914759248 914759248
ci_pipeline_id 63481469 63481469
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ekfrqbyz-project-304-concurrent-1-ghstnko7 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ekfrqbyz-project-304-concurrent-1-ghstnko7 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1334, 1371
.   : milestone, 1353,
appsec (1.725 ms) : 1700, 1749
.   : milestone, 1725,
appsec_no_iast (1.721 ms) : 1697, 1745
.   : milestone, 1721,
code_origins (1.697 ms) : 1670, 1724
.   : milestone, 1697,
iast (1.521 ms) : 1497, 1546
.   : milestone, 1521,
profiling (1.503 ms) : 1479, 1526
.   : milestone, 1503,
tracing (1.484 ms) : 1459, 1509
.   : milestone, 1484,
section candidate
no_agent (1.359 ms) : 1340, 1379
.   : milestone, 1359,
appsec (1.741 ms) : 1717, 1764
.   : milestone, 1741,
appsec_no_iast (1.741 ms) : 1718, 1765
.   : milestone, 1741,
code_origins (1.695 ms) : 1668, 1721
.   : milestone, 1695,
iast (1.502 ms) : 1478, 1527
.   : milestone, 1502,
profiling (1.51 ms) : 1486, 1534
.   : milestone, 1510,
tracing (1.492 ms) : 1467, 1517
.   : milestone, 1492,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.334 ms, 1.371 ms] -
appsec 1.725 ms [1.7 ms, 1.749 ms] 371.797 µs (27.5%)
appsec_no_iast 1.721 ms [1.697 ms, 1.745 ms] 368.103 µs (27.2%)
code_origins 1.697 ms [1.67 ms, 1.724 ms] 344.017 µs (25.4%)
iast 1.521 ms [1.497 ms, 1.546 ms] 168.722 µs (12.5%)
profiling 1.503 ms [1.479 ms, 1.526 ms] 149.997 µs (11.1%)
tracing 1.484 ms [1.459 ms, 1.509 ms] 131.095 µs (9.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.34 ms, 1.379 ms] -
appsec 1.741 ms [1.717 ms, 1.764 ms] 381.317 µs (28.1%)
appsec_no_iast 1.741 ms [1.718 ms, 1.765 ms] 382.058 µs (28.1%)
code_origins 1.695 ms [1.668 ms, 1.721 ms] 335.704 µs (24.7%)
iast 1.502 ms [1.478 ms, 1.527 ms] 142.947 µs (10.5%)
profiling 1.51 ms [1.486 ms, 1.534 ms] 150.515 µs (11.1%)
tracing 1.492 ms [1.467 ms, 1.517 ms] 133.044 µs (9.8%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28
    dateFormat X
    axisFormat %s
section baseline
no_agent (376.573 µs) : 357, 397
.   : milestone, 377,
iast (515.351 µs) : 492, 539
.   : milestone, 515,
iast_FULL (732.392 µs) : 709, 756
.   : milestone, 732,
iast_GLOBAL (564.926 µs) : 542, 588
.   : milestone, 565,
iast_HARDCODED_SECRET_DISABLED (518.086 µs) : 495, 541
.   : milestone, 518,
iast_INACTIVE (462.931 µs) : 441, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (504.238 µs) : 481, 527
.   : milestone, 504,
tracing (458.914 µs) : 437, 481
.   : milestone, 459,
section candidate
no_agent (378.473 µs) : 358, 399
.   : milestone, 378,
iast (506.743 µs) : 485, 528
.   : milestone, 507,
iast_FULL (728.301 µs) : 706, 750
.   : milestone, 728,
iast_GLOBAL (557.402 µs) : 536, 579
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (506.309 µs) : 485, 528
.   : milestone, 506,
iast_INACTIVE (465.487 µs) : 444, 487
.   : milestone, 465,
iast_TELEMETRY_OFF (496.198 µs) : 475, 518
.   : milestone, 496,
tracing (457.579 µs) : 436, 479
.   : milestone, 458,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 376.573 µs [356.623 µs, 396.523 µs] -
iast 515.351 µs [491.874 µs, 538.827 µs] 138.777 µs (36.9%)
iast_FULL 732.392 µs [709.107 µs, 755.677 µs] 355.818 µs (94.5%)
iast_GLOBAL 564.926 µs [541.961 µs, 587.891 µs] 188.353 µs (50.0%)
iast_HARDCODED_SECRET_DISABLED 518.086 µs [494.912 µs, 541.26 µs] 141.513 µs (37.6%)
iast_INACTIVE 462.931 µs [440.689 µs, 485.173 µs] 86.358 µs (22.9%)
iast_TELEMETRY_OFF 504.238 µs [481.228 µs, 527.247 µs] 127.664 µs (33.9%)
tracing 458.914 µs [436.748 µs, 481.079 µs] 82.34 µs (21.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.473 µs [357.642 µs, 399.304 µs] -
iast 506.743 µs [485.118 µs, 528.367 µs] 128.27 µs (33.9%)
iast_FULL 728.301 µs [706.18 µs, 750.422 µs] 349.828 µs (92.4%)
iast_GLOBAL 557.402 µs [535.676 µs, 579.128 µs] 178.929 µs (47.3%)
iast_HARDCODED_SECRET_DISABLED 506.309 µs [484.554 µs, 528.063 µs] 127.836 µs (33.8%)
iast_INACTIVE 465.487 µs [444.251 µs, 486.723 µs] 87.014 µs (23.0%)
iast_TELEMETRY_OFF 496.198 µs [474.804 µs, 517.592 µs] 117.725 µs (31.1%)
tracing 457.579 µs [436.073 µs, 479.085 µs] 79.106 µs (20.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/itw-line-probe
git_commit_date 1745843547 1745842799
git_commit_sha c55dc46 b995155
release_version 1.49.0-SNAPSHOT~c55dc46d28 1.49.0-SNAPSHOT~b99515547d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1745845355 1745845355
ci_job_id 914759249 914759249
ci_pipeline_id 63481469 63481469
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ekfrqbyz-project-304-concurrent-2-6s405b7k 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ekfrqbyz-project-304-concurrent-2-6s405b7k 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.199 s) : 15199000, 15199000
.   : milestone, 15199000,
appsec (14.776 s) : 14776000, 14776000
.   : milestone, 14776000,
iast (18.913 s) : 18913000, 18913000
.   : milestone, 18913000,
iast_GLOBAL (18.002 s) : 18002000, 18002000
.   : milestone, 18002000,
profiling (15.66 s) : 15660000, 15660000
.   : milestone, 15660000,
tracing (14.564 s) : 14564000, 14564000
.   : milestone, 14564000,
section candidate
no_agent (14.691 s) : 14691000, 14691000
.   : milestone, 14691000,
appsec (14.982 s) : 14982000, 14982000
.   : milestone, 14982000,
iast (18.639 s) : 18639000, 18639000
.   : milestone, 18639000,
iast_GLOBAL (18.354 s) : 18354000, 18354000
.   : milestone, 18354000,
profiling (15.503 s) : 15503000, 15503000
.   : milestone, 15503000,
tracing (14.901 s) : 14901000, 14901000
.   : milestone, 14901000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.199 s [15.199 s, 15.199 s] -
appsec 14.776 s [14.776 s, 14.776 s] -423.0 ms (-2.8%)
iast 18.913 s [18.913 s, 18.913 s] 3.714 s (24.4%)
iast_GLOBAL 18.002 s [18.002 s, 18.002 s] 2.803 s (18.4%)
profiling 15.66 s [15.66 s, 15.66 s] 461.0 ms (3.0%)
tracing 14.564 s [14.564 s, 14.564 s] -635.0 ms (-4.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.691 s [14.691 s, 14.691 s] -
appsec 14.982 s [14.982 s, 14.982 s] 291.0 ms (2.0%)
iast 18.639 s [18.639 s, 18.639 s] 3.948 s (26.9%)
iast_GLOBAL 18.354 s [18.354 s, 18.354 s] 3.663 s (24.9%)
profiling 15.503 s [15.503 s, 15.503 s] 812.0 ms (5.5%)
tracing 14.901 s [14.901 s, 14.901 s] 210.0 ms (1.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~b99515547d, baseline=1.49.0-SNAPSHOT~c55dc46d28
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (2.376 ms) : 2329, 2423
.   : milestone, 2376,
iast (2.149 ms) : 2090, 2208
.   : milestone, 2149,
iast_GLOBAL (2.199 ms) : 2140, 2259
.   : milestone, 2199,
profiling (2.012 ms) : 1964, 2060
.   : milestone, 2012,
tracing (1.98 ms) : 1934, 2025
.   : milestone, 1980,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.38 ms) : 2333, 2427
.   : milestone, 2380,
iast (2.149 ms) : 2090, 2208
.   : milestone, 2149,
iast_GLOBAL (2.189 ms) : 2129, 2248
.   : milestone, 2189,
profiling (2.01 ms) : 1963, 2057
.   : milestone, 2010,
tracing (1.973 ms) : 1928, 2019
.   : milestone, 1973,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 2.376 ms [2.329 ms, 2.423 ms] 901.586 µs (61.1%)
iast 2.149 ms [2.09 ms, 2.208 ms] 674.337 µs (45.7%)
iast_GLOBAL 2.199 ms [2.14 ms, 2.259 ms] 724.892 µs (49.2%)
profiling 2.012 ms [1.964 ms, 2.06 ms] 537.416 µs (36.4%)
tracing 1.98 ms [1.934 ms, 2.025 ms] 505.103 µs (34.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.38 ms [2.333 ms, 2.427 ms] 908.162 µs (61.7%)
iast 2.149 ms [2.09 ms, 2.208 ms] 677.318 µs (46.0%)
iast_GLOBAL 2.189 ms [2.129 ms, 2.248 ms] 716.92 µs (48.7%)
profiling 2.01 ms [1.963 ms, 2.057 ms] 538.071 µs (36.6%)
tracing 1.973 ms [1.928 ms, 2.019 ms] 501.819 µs (34.1%)

@jpbempel jpbempel added the tag: no release notes Changes to exclude from release notes label Apr 28, 2025
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch from 393eaa1 to b995155 Compare April 28, 2025 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jpbempel