The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Repository	Step Name	Console log	Pull Request
3003557	dotnet-aspnetcore	Run build.sh	Log
1471688	dotnet/roslyn	Run Unit Tests	Log
1471399	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#129580
1470338	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67190
1471605	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#66487
1471672	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#35462
1471442	dotnet/maui	Controls Layout	Log
1470559	dotnet/aspnetcore	Build	Log	dotnet/aspnetcore#67267
1470558	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67267
1471302	dotnet/runtime	Run Trimming Tests	Log	dotnet/runtime#129299
1471436	dotnet/aspnetcore	Build	Log	dotnet/aspnetcore#67249
1471435	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67249
1471499	dotnet/aspnetcore	Run Quarantined Tests	Log
3003492	dotnet-efcore	Build	Log
1471419	dotnet/aspnetcore	Run build.sh	Log
1471288	dotnet/maui	Controls Shell	Log	dotnet/maui#35878
1469912	dotnet/aspnetcore	Build	Log	dotnet/aspnetcore#49080
1471315	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67307
3003433	dotnet-runtime	Send job to Helix (Windows)	Log
1471086	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#66808
1471224	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65772
1471286	dotnet/runtime	Zip Build Assets	Log
1471214	dotnet/maui	Controls Shell	Log	dotnet/maui#34758
1471289	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#35878
3003414	dotnet-sdk	Checkout dotnet-sdk@refs/pull/62222/merge to s	Log	#62222
1471131	dotnet/maui	Controls ScrollView	Log
1471049	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#129579
1471218	dotnet/aspnetcore	Build	Log	dotnet/aspnetcore#67258
1471217	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67258
1471108	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67119
3003401	dotnet-aspnetcore	Run build.sh	Log
1471196	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#129586
3003370	dotnet-dotnet-monitor	Build	Log
1470630	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#127520
1471087	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66808
1466790	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#129207
1471203	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7625
1470679	dotnet/maui	Controls ViewBaseTests,VisualStateManager,Window	Log	dotnet/maui#35945
1471107	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67119
1471144	dotnet/roslyn	Run Unit Tests	Log
1471100	dotnet/aspnetcore	Run build.sh	Log
3002526	dotnet-dotnet	Run Binary Analysis Scan Optimization (Expand Archives) (1ES PT)	Log
1471083	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129582
1471064	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129533
1471136	dotnet/performance	Send job to Helix (Unix)	Log
3003176	dotnet-dotnet-buildtools-prereqs-docker	🔒 Start Network Isolation	Log
1471078	dotnet/aspnetcore	Build	Log
3003346	dotnet-efcore	Build	Log
3003343	dotnet-installer	Build	Log	#61739
3003342	dotnet-performance	Run performance job script	Log
3003338	dotnet-installer	🛡 Guardian: AntiMalware Scanner (Source)	Log	#61739
3003333	dotnet-installer	NuGetAuthenticate@1 (debian12)	Log	#61739
1471032	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129577
1470237	dotnet/maui	Controls Shell	Log
1470996	dotnet/maui	Controls Image,ImageButton,IndicatorView,InputTransparent,IsEnabled,IsVisible	Log	dotnet/maui#33365
1471035	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67296
3003320	dotnet-aspnetcore	Run build.sh	Log
3003195	dotnet-maui	Run Integration Tests - MultiProject	Log
1466181	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127690
1470980	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129549
1471010	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#66259
1470759	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#129572
1470407	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129539
1470997	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#33365
1470376	dotnet/runtime	Send to Helix	Log
1470924	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129321
1470878	dotnet/runtime	Send cDAC Dump Tests to Helix (Windows)	Log	dotnet/runtime#129520
1470806	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#129454
1470791	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#129574
1470873	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#129520
1470859	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129119
1470804	dotnet/runtime	Publish Build Assets	Log
1470076	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129502
1470092	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127582
1470770	dotnet/runtime	Build product	Log	dotnet/runtime#129570
1470667	dotnet/runtime	Publish	Log	dotnet/runtime#129302
1470729	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#129216
3003163	dotnet-maui	Run Integration Tests - Build	Log
3003097	dotnet-dotnet-buildtools-prereqs-docker	🔒 Start Network Isolation	Log
1470765	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67265
1470710	dotnet/runtime	Send cDAC Dump Tests to Helix (Windows)	Log	dotnet/runtime#129232
1470811	dotnet/roslyn	Run Unit Tests	Log
1470766	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67265
1470756	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128598
1470700	dotnet/maui	Controls Shell	Log
1465451	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129279
1470619	dotnet/runtime	Publish Build Assets	Log
3003191	dotnet-dotnet	Build	Log
1470715	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67098
1470707	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129232
1470681	dotnet/maui	Controls Shell	Log	dotnet/maui#35960
1470723	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67295
1470724	dotnet/aspnetcore	Build	Log	dotnet/aspnetcore#67295
1470762	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7625
1470676	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67293
1470714	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67098
1470682	dotnet/maui	Controls Shell	Log	dotnet/maui#35491
1470688	dotnet/maui	DeviceTestsWindows (Windows)	Log
1470703	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67294
1470623	dotnet/runtime	Send to Helix	Log
Displaying 100 of 4871 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
247	1316	4871

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez