Update system tests pinning trust policy #9941

sarahchen6 · 2025-11-12T15:44:49Z

What Does This Do

Update the trust policy to account for cases when the job_workflow_ref is triggered by a git tag.

Motivation

The previous trust policy failed in the latest release (link), so this should fix it. The updates here pass the recommended command For local debugging.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/APMLP-598

github-actions · 2025-11-12T15:45:02Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

datadog-datadog-prod-us1 · 2025-11-12T16:00:00Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 59.60% (-0.02%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 24a6fa1 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

pr-commenter · 2025-11-12T16:43:03Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-system-tests-pinning-trust-policy
git_commit_date	1762962102	1762961728
git_commit_sha	`c17d0e8`	`24a6fa1`
release_version	1.56.0-SNAPSHOT~c17d0e83ec	1.56.0-SNAPSHOT~24a6fa1bf0

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1762965745	1762965745
ci_job_id	1230414657	1230414657
ci_pipeline_id	82107198	82107198
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-1aa647qc 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-1aa647qc 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 12 performance improvements and 9 performance regressions! Performance is the same for 34 metrics, 10 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start	better [-75.763ms; -38.337ms] or [-6.103%; -3.088%]	1.184s	1.241s
scenario:startup:insecure-bank:iast:Debugger	better [-59.422ms; -58.619ms] or [-91.329%; -90.094%]	6.043ms	65.064ms
scenario:startup:insecure-bank:iast:Remote Config	worse [+46.651µs; +80.288µs] or [+8.491%; +14.613%]	612.888µs	549.419µs
scenario:startup:insecure-bank:iast:Flare Poller	worse [+447.817µs; +711.036µs] or [+12.686%; +20.143%]	4.109ms	3.530ms
scenario:startup:insecure-bank:tracing:Agent.start	better [-62.175ms; -49.518ms] or [-5.630%; -4.484%]	1.049s	1.104s
scenario:startup:insecure-bank:tracing:Debugger	better [-62.517ms; -61.604ms] or [-91.336%; -90.001%]	6.387ms	68.448ms
scenario:startup:insecure-bank:tracing:Remote Config	worse [+20.107µs; +76.396µs] or [+3.148%; +11.962%]	686.929µs	638.678µs
scenario:startup:petclinic:appsec:Agent.start	better [-65.334ms; -33.040ms] or [-5.111%; -2.585%]	1.229s	1.278s
scenario:startup:petclinic:appsec:Debugger	better [-55.377ms; -54.370ms] or [-90.878%; -89.226%]	6.062ms	60.936ms
scenario:startup:petclinic:appsec:Telemetry	better [-585.412µs; -222.868µs] or [-7.011%; -2.669%]	7.946ms	8.350ms
scenario:startup:petclinic:iast:Agent.start	better [-66.832ms; -55.212ms] or [-5.375%; -4.441%]	1.182s	1.243s
scenario:startup:petclinic:iast:Debugger	better [-60.154ms; -59.428ms] or [-91.286%; -90.185%]	6.105ms	65.896ms
scenario:startup:petclinic:iast:Telemetry	worse [+201.888µs; +665.219µs] or [+2.614%; +8.614%]	8.156ms	7.722ms
scenario:startup:petclinic:iast:Flare Poller	worse [+560.619µs; +682.006µs] or [+15.888%; +19.329%]	4.150ms	3.528ms
scenario:startup:petclinic:profiling:ProfilingAgent	worse [+12.354ms; +15.292ms] or [+12.701%; +15.722%]	111.091ms	97.267ms
scenario:startup:petclinic:profiling:Debugger	better [-62.791ms; -59.296ms] or [-91.615%; -86.516%]	7.495ms	68.538ms
scenario:startup:petclinic:profiling:Flare Poller	worse [+277.176µs; +437.550µs] or [+7.318%; +11.552%]	4.145ms	3.788ms
scenario:startup:petclinic:profiling:Profiling	worse [+12.415ms; +15.365ms] or [+12.687%; +15.702%]	111.746ms	97.856ms
scenario:startup:petclinic:tracing:Agent.start	better [-73.408ms; -43.140ms] or [-6.615%; -3.887%]	1.051s	1.110s
scenario:startup:petclinic:tracing:Debugger	better [-63.543ms; -61.769ms] or [-92.028%; -89.458%]	6.392ms	69.048ms
scenario:startup:petclinic:tracing:Remote Config	worse [+28.348µs; +67.491µs] or [+4.472%; +10.646%]	681.862µs	633.943µs

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~c17d0e83ec

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.104 s) : 0, 1104349
Total [baseline] (8.822 s) : 0, 8822379
Agent [candidate] (1.049 s) : 0, 1048502
Total [candidate] (8.64 s) : 0, 8639816
section iast
Agent [baseline] (1.241 s) : 0, 1241342
Total [baseline] (9.558 s) : 0, 9558371
Agent [candidate] (1.184 s) : 0, 1184292
Total [candidate] (9.317 s) : 0, 9316963

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.104 s	-
Agent	iast	1.241 s	136.994 ms (12.4%)
Total	tracing	8.822 s	-
Total	iast	9.558 s	735.992 ms (8.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	iast	1.184 s	135.79 ms (13.0%)
Total	tracing	8.64 s	-
Total	iast	9.317 s	677.147 ms (7.8%)

gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~c17d0e83ec

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.476 ms) : 0, 1476
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (706.344 ms) : 0, 706344
BytebuddyAgent [candidate] (703.376 ms) : 0, 703376
GlobalTracer [baseline] (248.396 ms) : 0, 248396
GlobalTracer [candidate] (248.714 ms) : 0, 248714
AppSec [baseline] (32.327 ms) : 0, 32327
AppSec [candidate] (32.468 ms) : 0, 32468
Debugger [baseline] (68.448 ms) : 0, 68448
Debugger [candidate] (6.387 ms) : 0, 6387
Remote Config [baseline] (638.678 µs) : 0, 639
Remote Config [candidate] (686.929 µs) : 0, 687
Telemetry [baseline] (8.219 ms) : 0, 8219
Telemetry [candidate] (13.442 ms) : 0, 13442
Flare Poller [baseline] (3.68 ms) : 0, 3680
Flare Poller [candidate] (7.338 ms) : 0, 7338
section iast
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (829.391 ms) : 0, 829391
BytebuddyAgent [candidate] (831.097 ms) : 0, 831097
GlobalTracer [baseline] (237.824 ms) : 0, 237824
GlobalTracer [candidate] (236.946 ms) : 0, 236946
IAST [baseline] (28.938 ms) : 0, 28938
IAST [candidate] (32.406 ms) : 0, 32406
AppSec [baseline] (32.112 ms) : 0, 32112
AppSec [candidate] (28.715 ms) : 0, 28715
Debugger [baseline] (65.064 ms) : 0, 65064
Debugger [candidate] (6.043 ms) : 0, 6043
Remote Config [baseline] (549.419 µs) : 0, 549
Remote Config [candidate] (612.888 µs) : 0, 613
Telemetry [baseline] (7.699 ms) : 0, 7699
Telemetry [candidate] (8.04 ms) : 0, 8040
Flare Poller [baseline] (3.53 ms) : 0, 3530
Flare Poller [candidate] (4.109 ms) : 0, 4109

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~c17d0e83ec

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.11 s) : 0, 1109772
Total [baseline] (10.701 s) : 0, 10700843
Agent [candidate] (1.051 s) : 0, 1051498
Total [candidate] (10.831 s) : 0, 10831480
section appsec
Agent [baseline] (1.278 s) : 0, 1278253
Total [baseline] (10.996 s) : 0, 10995611
Agent [candidate] (1.229 s) : 0, 1229066
Total [candidate] (10.938 s) : 0, 10938019
section iast
Agent [baseline] (1.243 s) : 0, 1243335
Total [baseline] (11.282 s) : 0, 11281955
Agent [candidate] (1.182 s) : 0, 1182314
Total [candidate] (11.206 s) : 0, 11206210
section profiling
Agent [baseline] (1.248 s) : 0, 1248484
Total [baseline] (11.031 s) : 0, 11031140
Agent [candidate] (1.199 s) : 0, 1199436
Total [candidate] (10.984 s) : 0, 10984391

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.11 s	-
Agent	appsec	1.278 s	168.481 ms (15.2%)
Agent	iast	1.243 s	133.563 ms (12.0%)
Agent	profiling	1.248 s	138.712 ms (12.5%)
Total	tracing	10.701 s	-
Total	appsec	10.996 s	294.767 ms (2.8%)
Total	iast	11.282 s	581.112 ms (5.4%)
Total	profiling	11.031 s	330.296 ms (3.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	appsec	1.229 s	177.568 ms (16.9%)
Agent	iast	1.182 s	130.815 ms (12.4%)
Agent	profiling	1.199 s	147.938 ms (14.1%)
Total	tracing	10.831 s	-
Total	appsec	10.938 s	106.54 ms (1.0%)
Total	iast	11.206 s	374.73 ms (3.5%)
Total	profiling	10.984 s	152.911 ms (1.4%)

gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~c17d0e83ec

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.478 ms) : 0, 1478
crashtracking [candidate] (1.47 ms) : 0, 1470
BytebuddyAgent [baseline] (709.812 ms) : 0, 709812
BytebuddyAgent [candidate] (706.042 ms) : 0, 706042
GlobalTracer [baseline] (249.567 ms) : 0, 249567
GlobalTracer [candidate] (249.531 ms) : 0, 249531
AppSec [baseline] (32.4 ms) : 0, 32400
AppSec [candidate] (32.381 ms) : 0, 32381
Debugger [baseline] (69.048 ms) : 0, 69048
Debugger [candidate] (6.392 ms) : 0, 6392
Remote Config [baseline] (633.943 µs) : 0, 634
Remote Config [candidate] (681.862 µs) : 0, 682
Telemetry [baseline] (8.209 ms) : 0, 8209
Telemetry [candidate] (15.37 ms) : 0, 15370
Flare Poller [baseline] (3.727 ms) : 0, 3727
Flare Poller [candidate] (4.874 ms) : 0, 4874
section appsec
crashtracking [baseline] (1.455 ms) : 0, 1455
crashtracking [candidate] (1.466 ms) : 0, 1466
BytebuddyAgent [baseline] (728.54 ms) : 0, 728540
BytebuddyAgent [candidate] (732.333 ms) : 0, 732333
GlobalTracer [baseline] (240.647 ms) : 0, 240647
GlobalTracer [candidate] (241.886 ms) : 0, 241886
IAST [baseline] (24.908 ms) : 0, 24908
IAST [candidate] (24.936 ms) : 0, 24936
AppSec [baseline] (174.172 ms) : 0, 174172
AppSec [candidate] (174.835 ms) : 0, 174835
Debugger [baseline] (60.936 ms) : 0, 60936
Debugger [candidate] (6.062 ms) : 0, 6062
Remote Config [baseline] (651.056 µs) : 0, 651
Remote Config [candidate] (659.435 µs) : 0, 659
Telemetry [baseline] (8.35 ms) : 0, 8350
Telemetry [candidate] (7.946 ms) : 0, 7946
Flare Poller [baseline] (3.811 ms) : 0, 3811
Flare Poller [candidate] (3.996 ms) : 0, 3996
section iast
crashtracking [baseline] (1.462 ms) : 0, 1462
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (830.269 ms) : 0, 830269
BytebuddyAgent [candidate] (827.987 ms) : 0, 827987
GlobalTracer [baseline] (238.194 ms) : 0, 238194
GlobalTracer [candidate] (237.399 ms) : 0, 237399
IAST [baseline] (30.561 ms) : 0, 30561
IAST [candidate] (31.885 ms) : 0, 31885
AppSec [baseline] (30.44 ms) : 0, 30440
AppSec [candidate] (29.727 ms) : 0, 29727
Debugger [baseline] (65.896 ms) : 0, 65896
Debugger [candidate] (6.105 ms) : 0, 6105
Remote Config [baseline] (569.497 µs) : 0, 569
Remote Config [candidate] (625.66 µs) : 0, 626
Telemetry [baseline] (7.722 ms) : 0, 7722
Telemetry [candidate] (8.156 ms) : 0, 8156
Flare Poller [baseline] (3.528 ms) : 0, 3528
Flare Poller [candidate] (4.15 ms) : 0, 4150
section profiling
crashtracking [baseline] (1.455 ms) : 0, 1455
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (740.863 ms) : 0, 740863
BytebuddyAgent [candidate] (733.28 ms) : 0, 733280
GlobalTracer [baseline] (224.499 ms) : 0, 224499
GlobalTracer [candidate] (223.062 ms) : 0, 223062
AppSec [baseline] (32.961 ms) : 0, 32961
AppSec [candidate] (32.591 ms) : 0, 32591
Debugger [baseline] (68.538 ms) : 0, 68538
Debugger [candidate] (7.495 ms) : 0, 7495
Remote Config [baseline] (651.495 µs) : 0, 651
Remote Config [candidate] (689.261 µs) : 0, 689
Telemetry [baseline] (8.006 ms) : 0, 8006
Telemetry [candidate] (15.369 ms) : 0, 15369
Flare Poller [baseline] (3.788 ms) : 0, 3788
Flare Poller [candidate] (4.145 ms) : 0, 4145
ProfilingAgent [baseline] (97.267 ms) : 0, 97267
ProfilingAgent [candidate] (111.091 ms) : 0, 111091
Profiling [baseline] (97.856 ms) : 0, 97856
Profiling [candidate] (111.746 ms) : 0, 111746

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-system-tests-pinning-trust-policy
git_commit_date	1762961860	1762961728
git_commit_sha	`d422f6d`	`24a6fa1`
release_version	1.56.0-SNAPSHOT~d422f6d41e	1.56.0-SNAPSHOT~24a6fa1bf0

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1762964223	1762964223
ci_job_id	1230256714	1230256714
ci_pipeline_id	82107198	82107198
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-z5ifodfd 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-z5ifodfd 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 0 performance regressions! Performance is the same for 19 metrics, 15 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:profiling:high_load	better [-1288.929µs; -687.078µs] or [-6.679%; -3.560%]	better [-2.036ms; -0.835ms] or [-6.514%; -2.673%]	unstable [-13.552op/s; +36.427op/s] or [-5.684%; +15.279%]	18.310ms	29.819ms	249.844op/s	19.298ms	31.254ms	238.406op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~d422f6d41e
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.229 ms) : 18045, 18412
.   : milestone, 18229,
appsec (19.488 ms) : 19288, 19687
.   : milestone, 19488,
code_origins (17.55 ms) : 17377, 17723
.   : milestone, 17550,
iast (17.687 ms) : 17513, 17862
.   : milestone, 17687,
profiling (19.584 ms) : 19384, 19784
.   : milestone, 19584,
tracing (17.446 ms) : 17274, 17619
.   : milestone, 17446,
section candidate
no_agent (18.994 ms) : 18801, 19187
.   : milestone, 18994,
appsec (18.79 ms) : 18600, 18980
.   : milestone, 18790,
code_origins (17.723 ms) : 17547, 17900
.   : milestone, 17723,
iast (17.736 ms) : 17562, 17910
.   : milestone, 17736,
profiling (18.681 ms) : 18495, 18867
.   : milestone, 18681,
tracing (17.556 ms) : 17379, 17733
.   : milestone, 17556,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.229 ms [18.045 ms, 18.412 ms]	-
appsec	19.488 ms [19.288 ms, 19.687 ms]	1.259 ms (6.9%)
code_origins	17.55 ms [17.377 ms, 17.723 ms]	-678.881 µs (-3.7%)
iast	17.687 ms [17.513 ms, 17.862 ms]	-541.292 µs (-3.0%)
profiling	19.584 ms [19.384 ms, 19.784 ms]	1.355 ms (7.4%)
tracing	17.446 ms [17.274 ms, 17.619 ms]	-782.224 µs (-4.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.994 ms [18.801 ms, 19.187 ms]	-
appsec	18.79 ms [18.6 ms, 18.98 ms]	-204.058 µs (-1.1%)
code_origins	17.723 ms [17.547 ms, 17.9 ms]	-1.271 ms (-6.7%)
iast	17.736 ms [17.562 ms, 17.91 ms]	-1.258 ms (-6.6%)
profiling	18.681 ms [18.495 ms, 18.867 ms]	-312.903 µs (-1.6%)
tracing	17.556 ms [17.379 ms, 17.733 ms]	-1.438 ms (-7.6%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~d422f6d41e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.18 ms) : 1168, 1191
.   : milestone, 1180,
iast (3.204 ms) : 3169, 3240
.   : milestone, 3204,
iast_FULL (5.956 ms) : 5895, 6016
.   : milestone, 5956,
iast_GLOBAL (3.624 ms) : 3569, 3679
.   : milestone, 3624,
profiling (2.09 ms) : 2071, 2109
.   : milestone, 2090,
tracing (1.863 ms) : 1847, 1879
.   : milestone, 1863,
section candidate
no_agent (1.178 ms) : 1167, 1190
.   : milestone, 1178,
iast (3.242 ms) : 3196, 3288
.   : milestone, 3242,
iast_FULL (5.865 ms) : 5808, 5923
.   : milestone, 5865,
iast_GLOBAL (3.559 ms) : 3508, 3611
.   : milestone, 3559,
profiling (1.992 ms) : 1974, 2010
.   : milestone, 1992,
tracing (1.774 ms) : 1759, 1789
.   : milestone, 1774,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.18 ms [1.168 ms, 1.191 ms]	-
iast	3.204 ms [3.169 ms, 3.24 ms]	2.025 ms (171.7%)
iast_FULL	5.956 ms [5.895 ms, 6.016 ms]	4.776 ms (404.9%)
iast_GLOBAL	3.624 ms [3.569 ms, 3.679 ms]	2.445 ms (207.3%)
profiling	2.09 ms [2.071 ms, 2.109 ms]	910.43 µs (77.2%)
tracing	1.863 ms [1.847 ms, 1.879 ms]	683.165 µs (57.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.178 ms [1.167 ms, 1.19 ms]	-
iast	3.242 ms [3.196 ms, 3.288 ms]	2.064 ms (175.2%)
iast_FULL	5.865 ms [5.808 ms, 5.923 ms]	4.687 ms (397.7%)
iast_GLOBAL	3.559 ms [3.508 ms, 3.611 ms]	2.381 ms (202.1%)
profiling	1.992 ms [1.974 ms, 2.01 ms]	813.357 µs (69.0%)
tracing	1.774 ms [1.759 ms, 1.789 ms]	595.428 µs (50.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-system-tests-pinning-trust-policy
git_commit_date	1762962102	1762961728
git_commit_sha	`c17d0e8`	`24a6fa1`
release_version	1.56.0-SNAPSHOT~c17d0e83ec	1.56.0-SNAPSHOT~24a6fa1bf0

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1762965045	1762965045
ci_job_id	1230256715	1230256715
ci_pipeline_id	82107198	82107198
cpu_model	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-onz3cphe 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-onz3cphe 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.588ms; -1.186ms] or [-38.089%; -28.453%]	2.782ms	4.169ms

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~c17d0e83ec
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.484 ms) : 1472, 1496
.   : milestone, 1484,
appsec (4.169 ms) : 3918, 4420
.   : milestone, 4169,
iast (2.566 ms) : 2466, 2666
.   : milestone, 2566,
iast_GLOBAL (2.609 ms) : 2509, 2709
.   : milestone, 2609,
profiling (2.418 ms) : 2332, 2504
.   : milestone, 2418,
tracing (2.342 ms) : 2261, 2423
.   : milestone, 2342,
section candidate
no_agent (1.488 ms) : 1476, 1500
.   : milestone, 1488,
appsec (2.782 ms) : 2699, 2865
.   : milestone, 2782,
iast (2.568 ms) : 2468, 2668
.   : milestone, 2568,
iast_GLOBAL (2.614 ms) : 2514, 2715
.   : milestone, 2614,
profiling (2.409 ms) : 2324, 2495
.   : milestone, 2409,
tracing (2.349 ms) : 2268, 2431
.   : milestone, 2349,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.484 ms [1.472 ms, 1.496 ms]	-
appsec	4.169 ms [3.918 ms, 4.42 ms]	2.685 ms (181.0%)
iast	2.566 ms [2.466 ms, 2.666 ms]	1.082 ms (72.9%)
iast_GLOBAL	2.609 ms [2.509 ms, 2.709 ms]	1.125 ms (75.8%)
profiling	2.418 ms [2.332 ms, 2.504 ms]	934.414 µs (63.0%)
tracing	2.342 ms [2.261 ms, 2.423 ms]	857.754 µs (57.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.488 ms [1.476 ms, 1.5 ms]	-
appsec	2.782 ms [2.699 ms, 2.865 ms]	1.294 ms (86.9%)
iast	2.568 ms [2.468 ms, 2.668 ms]	1.08 ms (72.5%)
iast_GLOBAL	2.614 ms [2.514 ms, 2.715 ms]	1.126 ms (75.7%)
profiling	2.409 ms [2.324 ms, 2.495 ms]	921.313 µs (61.9%)
tracing	2.349 ms [2.268 ms, 2.431 ms]	861.312 µs (57.9%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~24a6fa1bf0, baseline=1.56.0-SNAPSHOT~c17d0e83ec
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.575 s) : 14575000, 14575000
.   : milestone, 14575000,
appsec (14.101 s) : 14101000, 14101000
.   : milestone, 14101000,
iast (18.358 s) : 18358000, 18358000
.   : milestone, 18358000,
iast_GLOBAL (17.352 s) : 17352000, 17352000
.   : milestone, 17352000,
profiling (15.176 s) : 15176000, 15176000
.   : milestone, 15176000,
tracing (14.423 s) : 14423000, 14423000
.   : milestone, 14423000,
section candidate
no_agent (14.88 s) : 14880000, 14880000
.   : milestone, 14880000,
appsec (14.23 s) : 14230000, 14230000
.   : milestone, 14230000,
iast (17.821 s) : 17821000, 17821000
.   : milestone, 17821000,
iast_GLOBAL (17.158 s) : 17158000, 17158000
.   : milestone, 17158000,
profiling (14.367 s) : 14367000, 14367000
.   : milestone, 14367000,
tracing (14.193 s) : 14193000, 14193000
.   : milestone, 14193000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.575 s [14.575 s, 14.575 s]	-
appsec	14.101 s [14.101 s, 14.101 s]	-474.0 ms (-3.3%)
iast	18.358 s [18.358 s, 18.358 s]	3.783 s (26.0%)
iast_GLOBAL	17.352 s [17.352 s, 17.352 s]	2.777 s (19.1%)
profiling	15.176 s [15.176 s, 15.176 s]	601.0 ms (4.1%)
tracing	14.423 s [14.423 s, 14.423 s]	-152.0 ms (-1.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.88 s [14.88 s, 14.88 s]	-
appsec	14.23 s [14.23 s, 14.23 s]	-650.0 ms (-4.4%)
iast	17.821 s [17.821 s, 17.821 s]	2.941 s (19.8%)
iast_GLOBAL	17.158 s [17.158 s, 17.158 s]	2.278 s (15.3%)
profiling	14.367 s [14.367 s, 14.367 s]	-513.0 ms (-3.4%)
tracing	14.193 s [14.193 s, 14.193 s]	-687.0 ms (-4.6%)

PerfectSlayer

📝 notes: Make sense as the related pipeline is running on this trigger:

dd-trace-java/.github/workflows/create-release-branch.yaml

Line 6 in 640a4bd

- 'v[0-9]+.[0-9]+.0' # Trigger on minor release tags (e.g. v1.54.0)

sarahchen6 · 2025-11-13T18:08:07Z

Closing this because there is no need to merge the PR with #9954. Given the prevailing Resource not accessible by integration errors, I'm simplifying the workflow to only automate the steps that create the release branch, leaving the PR that pins system tests to be manually created. That means this trust policy will no longer be needed.

PerfectSlayer · 2025-11-17T06:30:00Z

Sounds good to me. Can you add a quick update to the .github/workflow/README.md about the create-release-branch workflow then? 🙏

Update system tests pinning trust policy

24a6fa1

sarahchen6 requested a review from a team as a code owner November 12, 2025 15:44

sarahchen6 requested review from PerfectSlayer and removed request for a team November 12, 2025 15:44

bric3 approved these changes Nov 12, 2025

View reviewed changes

sarahchen6 added type: bug Bug report and fix tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Nov 12, 2025

sarahchen6 enabled auto-merge (squash) November 12, 2025 15:48

PerfectSlayer approved these changes Nov 13, 2025

View reviewed changes

sarahchen6 disabled auto-merge November 13, 2025 17:40

sarahchen6 closed this Nov 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update system tests pinning trust policy #9941

Update system tests pinning trust policy #9941

Uh oh!

sarahchen6 commented Nov 12, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 12, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 12, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Nov 12, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

sarahchen6 commented Nov 13, 2025

Uh oh!

PerfectSlayer commented Nov 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Update system tests pinning trust policy #9941

Update system tests pinning trust policy #9941

Uh oh!

Conversation

sarahchen6 commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Nov 12, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Nov 12, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

sarahchen6 commented Nov 13, 2025

Uh oh!

PerfectSlayer commented Nov 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sarahchen6 commented Nov 12, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Nov 12, 2025 •

edited

Loading