Separate V1/V2 custom events to validate metrics sent via telemetry #8640

manuel-alvarez-alvarez · 2025-03-28T17:40:33Z

What Does This Do

Sends separated telemetry payloads for V1/V2 custom events.

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-03-28T18:31:07Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	malvarez/ato-custom-event-metric
git_commit_date	1743686401	1743686342
git_commit_sha	`6d284ce`	`5ec4ad6`
release_version	1.48.0-SNAPSHOT~6d284ce182	1.48.0-SNAPSHOT~5ec4ad63ac

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1743689436	1743689436
ci_job_id	879308116	879308116
ci_pipeline_id	60950918	60950918
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-f8yuutny-project-304-concurrent-0-30w7etbt 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-f8yuutny-project-304-concurrent-0-30w7etbt 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~5ec4ad63ac, baseline=1.48.0-SNAPSHOT~6d284ce182

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059346
Total [baseline] (8.693 s) : 0, 8693334
Agent [candidate] (1.066 s) : 0, 1065653
Total [candidate] (8.668 s) : 0, 8668249
section iast
Agent [baseline] (1.189 s) : 0, 1188755
Total [baseline] (9.273 s) : 0, 9272548
Agent [candidate] (1.178 s) : 0, 1177950
Total [candidate] (9.228 s) : 0, 9227799
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.181 s) : 0, 1181486
Total [baseline] (9.227 s) : 0, 9226604
Agent [candidate] (1.18 s) : 0, 1180394
Total [candidate] (9.195 s) : 0, 9195499
section iast_TELEMETRY_OFF
Agent [baseline] (1.178 s) : 0, 1177553
Total [baseline] (9.236 s) : 0, 9235966
Agent [candidate] (1.176 s) : 0, 1175819
Total [candidate] (9.229 s) : 0, 9228687

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	iast	1.189 s	129.409 ms (12.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.181 s	122.14 ms (11.5%)
Agent	iast_TELEMETRY_OFF	1.178 s	118.207 ms (11.2%)
Total	tracing	8.693 s	-
Total	iast	9.273 s	579.215 ms (6.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.227 s	533.27 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.236 s	542.633 ms (6.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	iast	1.178 s	112.297 ms (10.5%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.18 s	114.741 ms (10.8%)
Agent	iast_TELEMETRY_OFF	1.176 s	110.166 ms (10.3%)
Total	tracing	8.668 s	-
Total	iast	9.228 s	559.55 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.195 s	527.25 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.229 s	560.438 ms (6.5%)

gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~5ec4ad63ac, baseline=1.48.0-SNAPSHOT~6d284ce182

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (723.624 ms) : 0, 723624
BytebuddyAgent [candidate] (728.755 ms) : 0, 728755
GlobalTracer [baseline] (241.4 ms) : 0, 241400
GlobalTracer [candidate] (242.356 ms) : 0, 242356
AppSec [baseline] (56.234 ms) : 0, 56234
AppSec [candidate] (57.179 ms) : 0, 57179
Debugger [baseline] (4.375 ms) : 0, 4375
Debugger [candidate] (4.448 ms) : 0, 4448
Remote Config [baseline] (707.089 µs) : 0, 707
Remote Config [candidate] (702.161 µs) : 0, 702
Telemetry [baseline] (12.307 ms) : 0, 12307
Telemetry [candidate] (11.422 ms) : 0, 11422
section iast
BytebuddyAgent [baseline] (845.054 ms) : 0, 845054
BytebuddyAgent [candidate] (836.063 ms) : 0, 836063
GlobalTracer [baseline] (230.501 ms) : 0, 230501
GlobalTracer [candidate] (229.292 ms) : 0, 229292
IAST [baseline] (22.913 ms) : 0, 22913
IAST [candidate] (22.615 ms) : 0, 22615
AppSec [baseline] (56.143 ms) : 0, 56143
AppSec [candidate] (56.0 ms) : 0, 56000
Debugger [baseline] (4.143 ms) : 0, 4143
Debugger [candidate] (4.155 ms) : 0, 4155
Remote Config [baseline] (608.283 µs) : 0, 608
Remote Config [candidate] (596.675 µs) : 0, 597
Telemetry [baseline] (8.758 ms) : 0, 8758
Telemetry [candidate] (8.775 ms) : 0, 8775
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (838.712 ms) : 0, 838712
BytebuddyAgent [candidate] (837.733 ms) : 0, 837733
GlobalTracer [baseline] (230.349 ms) : 0, 230349
GlobalTracer [candidate] (230.125 ms) : 0, 230125
IAST [baseline] (23.657 ms) : 0, 23657
IAST [candidate] (22.818 ms) : 0, 22818
AppSec [baseline] (54.679 ms) : 0, 54679
AppSec [candidate] (55.774 ms) : 0, 55774
Debugger [baseline] (4.146 ms) : 0, 4146
Debugger [candidate] (4.097 ms) : 0, 4097
Remote Config [baseline] (610.306 µs) : 0, 610
Remote Config [candidate] (596.592 µs) : 0, 597
Telemetry [baseline] (8.792 ms) : 0, 8792
Telemetry [candidate] (8.711 ms) : 0, 8711
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (836.168 ms) : 0, 836168
BytebuddyAgent [candidate] (834.759 ms) : 0, 834759
GlobalTracer [baseline] (229.511 ms) : 0, 229511
GlobalTracer [candidate] (229.226 ms) : 0, 229226
IAST [baseline] (23.07 ms) : 0, 23070
IAST [candidate] (22.269 ms) : 0, 22269
AppSec [baseline] (55.072 ms) : 0, 55072
AppSec [candidate] (55.855 ms) : 0, 55855
Debugger [baseline] (4.138 ms) : 0, 4138
Debugger [candidate] (4.172 ms) : 0, 4172
Remote Config [baseline] (593.299 µs) : 0, 593
Remote Config [candidate] (600.149 µs) : 0, 600
Telemetry [baseline] (8.558 ms) : 0, 8558
Telemetry [candidate] (8.511 ms) : 0, 8511

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~5ec4ad63ac, baseline=1.48.0-SNAPSHOT~6d284ce182

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1047622
Total [baseline] (10.421 s) : 0, 10420808
Agent [candidate] (1.054 s) : 0, 1053704
Total [candidate] (10.48 s) : 0, 10480192
section appsec
Agent [baseline] (1.192 s) : 0, 1191992
Total [baseline] (10.729 s) : 0, 10728600
Agent [candidate] (1.19 s) : 0, 1190068
Total [candidate] (10.765 s) : 0, 10764535
section iast
Agent [baseline] (1.179 s) : 0, 1179495
Total [baseline] (11.039 s) : 0, 11039260
Agent [candidate] (1.18 s) : 0, 1179686
Total [candidate] (10.942 s) : 0, 10941620
section profiling
Agent [baseline] (1.281 s) : 0, 1281299
Total [baseline] (10.929 s) : 0, 10928791
Agent [candidate] (1.276 s) : 0, 1275791
Total [candidate] (10.883 s) : 0, 10883419

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.192 s	144.37 ms (13.8%)
Agent	iast	1.179 s	131.874 ms (12.6%)
Agent	profiling	1.281 s	233.678 ms (22.3%)
Total	tracing	10.421 s	-
Total	appsec	10.729 s	307.792 ms (3.0%)
Total	iast	11.039 s	618.452 ms (5.9%)
Total	profiling	10.929 s	507.983 ms (4.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.054 s	-
Agent	appsec	1.19 s	136.365 ms (12.9%)
Agent	iast	1.18 s	125.983 ms (12.0%)
Agent	profiling	1.276 s	222.087 ms (21.1%)
Total	tracing	10.48 s	-
Total	appsec	10.765 s	284.343 ms (2.7%)
Total	iast	10.942 s	461.428 ms (4.4%)
Total	profiling	10.883 s	403.228 ms (3.8%)

gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~5ec4ad63ac, baseline=1.48.0-SNAPSHOT~6d284ce182

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.923 ms) : 0, 717923
BytebuddyAgent [candidate] (719.145 ms) : 0, 719145
GlobalTracer [baseline] (238.801 ms) : 0, 238801
GlobalTracer [candidate] (239.707 ms) : 0, 239707
AppSec [baseline] (53.983 ms) : 0, 53983
AppSec [candidate] (54.368 ms) : 0, 54368
Debugger [baseline] (4.365 ms) : 0, 4365
Debugger [candidate] (4.384 ms) : 0, 4384
Remote Config [baseline] (693.989 µs) : 0, 694
Remote Config [candidate] (694.881 µs) : 0, 695
Telemetry [baseline] (11.42 ms) : 0, 11420
Telemetry [candidate] (14.834 ms) : 0, 14834
section appsec
BytebuddyAgent [baseline] (736.506 ms) : 0, 736506
BytebuddyAgent [candidate] (735.446 ms) : 0, 735446
GlobalTracer [baseline] (235.338 ms) : 0, 235338
GlobalTracer [candidate] (235.082 ms) : 0, 235082
AppSec [baseline] (175.836 ms) : 0, 175836
AppSec [candidate] (175.408 ms) : 0, 175408
Debugger [baseline] (4.287 ms) : 0, 4287
Debugger [candidate] (4.243 ms) : 0, 4243
Remote Config [baseline] (636.548 µs) : 0, 637
Remote Config [candidate] (633.764 µs) : 0, 634
Telemetry [baseline] (8.212 ms) : 0, 8212
Telemetry [candidate] (8.17 ms) : 0, 8170
IAST [baseline] (21.447 ms) : 0, 21447
IAST [candidate] (21.376 ms) : 0, 21376
section iast
BytebuddyAgent [baseline] (837.957 ms) : 0, 837957
BytebuddyAgent [candidate] (838.1 ms) : 0, 838100
GlobalTracer [baseline] (229.54 ms) : 0, 229540
GlobalTracer [candidate] (229.54 ms) : 0, 229540
AppSec [baseline] (55.572 ms) : 0, 55572
AppSec [candidate] (55.714 ms) : 0, 55714
Debugger [baseline] (4.124 ms) : 0, 4124
Debugger [candidate] (4.094 ms) : 0, 4094
Remote Config [baseline] (600.24 µs) : 0, 600
Remote Config [candidate] (594.238 µs) : 0, 594
Telemetry [baseline] (8.63 ms) : 0, 8630
Telemetry [candidate] (8.606 ms) : 0, 8606
IAST [baseline] (22.658 ms) : 0, 22658
IAST [candidate] (22.6 ms) : 0, 22600
section profiling
BytebuddyAgent [baseline] (714.235 ms) : 0, 714235
BytebuddyAgent [candidate] (711.275 ms) : 0, 711275
GlobalTracer [baseline] (349.52 ms) : 0, 349520
GlobalTracer [candidate] (348.785 ms) : 0, 348785
AppSec [baseline] (54.936 ms) : 0, 54936
AppSec [candidate] (53.337 ms) : 0, 53337
Debugger [baseline] (4.261 ms) : 0, 4261
Debugger [candidate] (4.254 ms) : 0, 4254
Remote Config [baseline] (697.702 µs) : 0, 698
Remote Config [candidate] (692.384 µs) : 0, 692
Telemetry [baseline] (9.05 ms) : 0, 9050
Telemetry [candidate] (8.924 ms) : 0, 8924
ProfilingAgent [baseline] (102.646 ms) : 0, 102646
ProfilingAgent [candidate] (102.444 ms) : 0, 102444
Profiling [baseline] (102.671 ms) : 0, 102671
Profiling [candidate] (102.47 ms) : 0, 102470

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-04-03T13:40:52	2025-04-03T13:48:40
git_branch	master	malvarez/ato-custom-event-metric
git_commit_date	1743686401	1743686342
git_commit_sha	`6d284ce`	`5ec4ad6`
release_version	1.48.0-SNAPSHOT~6d284ce182	1.48.0-SNAPSHOT~5ec4ad63ac
start_time	2025-04-03T13:40:38	2025-04-03T13:48:26

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1743688521	1743688521
ci_job_id	879308117	879308117
ci_pipeline_id	60950918	60950918
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-f8yuutny-project-304-concurrent-1-xwt9qf2j 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-f8yuutny-project-304-concurrent-1-xwt9qf2j 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~5ec4ad63ac, baseline=1.48.0-SNAPSHOT~6d284ce182
    dateFormat X
    axisFormat %s
section baseline
no_agent (383.772 µs) : 364, 404
.   : milestone, 384,
iast (519.971 µs) : 498, 542
.   : milestone, 520,
iast_FULL (736.431 µs) : 715, 758
.   : milestone, 736,
iast_GLOBAL (559.685 µs) : 538, 581
.   : milestone, 560,
iast_HARDCODED_SECRET_DISABLED (526.293 µs) : 504, 548
.   : milestone, 526,
iast_INACTIVE (473.252 µs) : 452, 495
.   : milestone, 473,
iast_TELEMETRY_OFF (508.588 µs) : 487, 530
.   : milestone, 509,
tracing (478.681 µs) : 457, 500
.   : milestone, 479,
section candidate
no_agent (387.311 µs) : 366, 408
.   : milestone, 387,
iast (519.491 µs) : 498, 541
.   : milestone, 519,
iast_FULL (739.78 µs) : 717, 762
.   : milestone, 740,
iast_GLOBAL (563.32 µs) : 541, 585
.   : milestone, 563,
iast_HARDCODED_SECRET_DISABLED (518.271 µs) : 497, 540
.   : milestone, 518,
iast_INACTIVE (476.049 µs) : 454, 498
.   : milestone, 476,
iast_TELEMETRY_OFF (507.669 µs) : 486, 529
.   : milestone, 508,
tracing (463.815 µs) : 443, 485
.   : milestone, 464,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	383.772 µs [363.762 µs, 403.783 µs]	-
iast	519.971 µs [498.233 µs, 541.71 µs]	136.199 µs (35.5%)
iast_FULL	736.431 µs [714.512 µs, 758.349 µs]	352.658 µs (91.9%)
iast_GLOBAL	559.685 µs [538.08 µs, 581.29 µs]	175.912 µs (45.8%)
iast_HARDCODED_SECRET_DISABLED	526.293 µs [504.281 µs, 548.305 µs]	142.521 µs (37.1%)
iast_INACTIVE	473.252 µs [451.622 µs, 494.883 µs]	89.48 µs (23.3%)
iast_TELEMETRY_OFF	508.588 µs [486.871 µs, 530.304 µs]	124.815 µs (32.5%)
tracing	478.681 µs [457.257 µs, 500.106 µs]	94.909 µs (24.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	387.311 µs [366.254 µs, 408.368 µs]	-
iast	519.491 µs [497.718 µs, 541.264 µs]	132.18 µs (34.1%)
iast_FULL	739.78 µs [717.109 µs, 762.45 µs]	352.469 µs (91.0%)
iast_GLOBAL	563.32 µs [541.421 µs, 585.219 µs]	176.009 µs (45.4%)
iast_HARDCODED_SECRET_DISABLED	518.271 µs [496.525 µs, 540.016 µs]	130.96 µs (33.8%)
iast_INACTIVE	476.049 µs [454.203 µs, 497.895 µs]	88.739 µs (22.9%)
iast_TELEMETRY_OFF	507.669 µs [486.073 µs, 529.265 µs]	120.358 µs (31.1%)
tracing	463.815 µs [442.651 µs, 484.979 µs]	76.504 µs (19.8%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~5ec4ad63ac, baseline=1.48.0-SNAPSHOT~6d284ce182
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.355 ms) : 1335, 1374
.   : milestone, 1355,
appsec (1.751 ms) : 1728, 1775
.   : milestone, 1751,
appsec_no_iast (1.742 ms) : 1718, 1765
.   : milestone, 1742,
code_origins (1.705 ms) : 1678, 1732
.   : milestone, 1705,
iast (1.559 ms) : 1534, 1583
.   : milestone, 1559,
profiling (1.523 ms) : 1500, 1547
.   : milestone, 1523,
tracing (1.544 ms) : 1520, 1568
.   : milestone, 1544,
section candidate
no_agent (1.359 ms) : 1339, 1379
.   : milestone, 1359,
appsec (1.766 ms) : 1743, 1790
.   : milestone, 1766,
appsec_no_iast (1.745 ms) : 1721, 1768
.   : milestone, 1745,
code_origins (1.7 ms) : 1674, 1727
.   : milestone, 1700,
iast (1.542 ms) : 1518, 1567
.   : milestone, 1542,
profiling (1.552 ms) : 1529, 1575
.   : milestone, 1552,
tracing (1.508 ms) : 1482, 1533
.   : milestone, 1508,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.355 ms [1.335 ms, 1.374 ms]	-
appsec	1.751 ms [1.728 ms, 1.775 ms]	396.674 µs (29.3%)
appsec_no_iast	1.742 ms [1.718 ms, 1.765 ms]	387.111 µs (28.6%)
code_origins	1.705 ms [1.678 ms, 1.732 ms]	350.256 µs (25.9%)
iast	1.559 ms [1.534 ms, 1.583 ms]	203.833 µs (15.0%)
profiling	1.523 ms [1.5 ms, 1.547 ms]	168.694 µs (12.5%)
tracing	1.544 ms [1.52 ms, 1.568 ms]	189.103 µs (14.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.359 ms [1.339 ms, 1.379 ms]	-
appsec	1.766 ms [1.743 ms, 1.79 ms]	407.248 µs (30.0%)
appsec_no_iast	1.745 ms [1.721 ms, 1.768 ms]	385.657 µs (28.4%)
code_origins	1.7 ms [1.674 ms, 1.727 ms]	341.274 µs (25.1%)
iast	1.542 ms [1.518 ms, 1.567 ms]	183.042 µs (13.5%)
profiling	1.552 ms [1.529 ms, 1.575 ms]	192.932 µs (14.2%)
tracing	1.508 ms [1.482 ms, 1.533 ms]	148.597 µs (10.9%)

Dacapo

manuel-alvarez-alvarez added type: enhancement tag: no release notes Changes to exclude from release notes comp: asm waf Application Security Management (WAF) labels Mar 28, 2025

manuel-alvarez-alvarez requested a review from a team as a code owner March 28, 2025 17:40

manuel-alvarez-alvarez requested review from smola and jandro996 March 28, 2025 17:40

jandro996 approved these changes Mar 28, 2025

View reviewed changes

smola approved these changes Apr 3, 2025

View reviewed changes

Separate V1/V2 custom events

5ec4ad6

manuel-alvarez-alvarez force-pushed the malvarez/ato-custom-event-metric branch from bc2bae0 to 5ec4ad6 Compare April 3, 2025 13:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Separate V1/V2 custom events to validate metrics sent via telemetry #8640

Separate V1/V2 custom events to validate metrics sent via telemetry #8640

manuel-alvarez-alvarez commented Mar 28, 2025 •

edited

Loading

pr-commenter bot commented Mar 28, 2025 •

edited

Loading

Separate V1/V2 custom events to validate metrics sent via telemetry #8640

Are you sure you want to change the base?

Separate V1/V2 custom events to validate metrics sent via telemetry #8640

Conversation

manuel-alvarez-alvarez commented Mar 28, 2025 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Mar 28, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

manuel-alvarez-alvarez commented Mar 28, 2025 •

edited

Loading

pr-commenter bot commented Mar 28, 2025 •

edited

Loading