Update the iOS scenarios build flow to match android

[LoopedBard3]

Update the iOS scenarios to use the same flow as android, bringing them up to running net10. This copies the build flow update from #4770 to the iOS scenarios.

Successful test run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2689882&view=results

[matouskozak]

Thanks a lot for porting the changes to iOS as well. I would propose to make a few more changes if possible.

1. Unifying the scenario naming between maui_scenarios_ios.proj and maui_scenarios_android.proj so that they follow the same naming conventions so the subsequent PowerBI queries are easier to implement.
2. Add codeGenType: FullAOT to Mono scenarios in sdk-perf-jobs.yml. (I'm unsure if we log somewhere that iOS jobs use LLVM backend. I think we log it only for dotnet/runtime perf measurements not scenarios. It would be nice to log this as well here.)
3. For CoreCLR, I wonder if we should use runtimeFlavor: coreclr + codeGenType: NativeAOT or use just runtimeFlavor: nativeaot. @ivanpovazan do you have insight into what is the proper naming here? I'm more in favor of the 1st one, but I'm unsure whether NativeAOT is still considered as part of CoreCLR runtime (it lives in the same subfolder in dotnet/runtime but...)

[e-kharion]

would it make sense to get channels from a (default) value of a new template parameter? So it's kept uniform across all template

[LoopedBard3]

That probably makes sense for most cases, although mobile is currently a special case as we only run main and not each main, 9.0, and 8.0.

[e-kharion]

probably out of scope for this PR but would it be possible and would it make sense to unify all the pre.py scripts to avoid code/configs duplications to avoid possible misconfigurations/bugs in the future?

[e-kharion]

LGTM in general, few comments left though

[LoopedBard3]

Thanks a lot for porting the changes to iOS as well. I would propose to make a few more changes if possible.

1. Unifying the scenario naming between maui_scenarios_ios.proj and maui_scenarios_android.proj so that they follow the same naming conventions so the subsequent PowerBI queries are easier to implement.
2. Add codeGenType: FullAOT to Mono scenarios in sdk-perf-jobs.yml. (I'm unsure if we log somewhere that iOS jobs use LLVM backend. I think we log it only for dotnet/runtime perf measurements not scenarios. It would be nice to log this as well here.)
3. For CoreCLR, I wonder if we should use runtimeFlavor: coreclr + codeGenType: NativeAOT or use just runtimeFlavor: nativeaot. @ivanpovazan do you have insight into what is the proper naming here? I'm more in favor of the 1st one, but I'm unsure whether NativeAOT is still considered as part of CoreCLR runtime (it lives in the same subfolder in dotnet/runtime but...)

For 1, I have a commit ready locally but also need the configuration updates so we can use it.

For 2 and partially 3, I am looking for confirmation of the correct configuration values to use.

• One part is whether Mono should have FullAOT, ProfiledAOT like the maui android scenarios, or just AOT like the runtime pipeline android scenarios?
• Another part is confirming what the coreclr/native aot should have, assuming CoreCLR is the correct RuntimeType and not Mono or NativeAOT? We do use the RuntimeType NativeAOT for the runtime mobile testing. (@ivanpovazan, do you have insight or preference here?)
• For including whether a build is LLVM, what would the preferred value be for the iOS Mono and iOS CoreCLR configurations be? For the runtime builds, we already pass iOSLlvmBuild as a configuration parameter and could use it here as well.

Once we have figured out what we want for the configurations, I think making those changes will be pretty fast.

[matouskozak]

• One part is whether Mono should have FullAOT, ProfiledAOT like the maui android scenarios, or just AOT like the runtime pipeline android scenarios?

In fact, all three use different setup:

• runtime android sample uses Mono AOT (AOT most of the methods but allows JIT for execution)
• maui android scenarios use Mono ProfiledAOT (AOT methods based on a profile and the rest is JIT)
• maui ios scenarios use Mono FullAOT (AOT everything, no JIT is allowed)

Each configuration differs in the amount of code that is AOTed before the app execution.

• Another part is confirming what the coreclr/native aot should have, assuming CoreCLR is the correct RuntimeType and not Mono or NativeAOT? We do use the RuntimeType NativeAOT for the runtime mobile testing. (@ivanpovazan, do you have insight or preference here?)

I would prefer the RuntimeType to be CoreCLR and CodegenType to be NativeAOT to provide a clear separation from Mono RuntimeType. We could then filter by RuntimeType to get all Mono or non-Mono (CoreCLR) configurations. However, I'll let Ivan decide as he has the insight into this domain.

• For including whether a build is LLVM, what would the preferred value be for the iOS Mono and iOS CoreCLR configurations be? For the runtime builds, we already pass iOSLlvmBuild as a configuration parameter and could use it here as well.

LLVM is only relevant to Mono. In current setup, only affects iOS builds (we do allow LLVM with Android builds, but it is not a recommended scenario, and we don't measure it at the moment).

[ivanpovazan]

Another part is confirming what the coreclr/native aot should have, assuming CoreCLR is the correct RuntimeType and not Mono or NativeAOT? We do use the RuntimeType NativeAOT for the runtime mobile testing. (@ivanpovazan, do you have insight or preference here?)

@LoopedBard3 Are you referring to RuntimeFlavor when you say RuntimeType ?

I agree it is confusing and is probably misused in some places, like using RuntimeFlavor=CoreCLR even for NativeAOT builds/tests, so we had to added extra properties to distinguish between the runtimes in our dotnet/runtime infra.
I opened an issue about unifying it : dotnet/runtime#111923 where I suggested TargetRuntime/RuntimeType is used, but it is more involved work and doesn't mean we should block this PR on it.

I am more inclined towards having runtimeFlavor: coreclr + codeGenType: NativeAOT even if codeGenType is not an ideal name, just because it matches more what we have in other areas of testing: RuntimeFlavor=CoreCLR + some-other-property-saying-it-is-NativeAOT.

We can then revisit this again once we tackle dotnet/runtime#111923

[LoopedBard3]

@LoopedBard3 Are you referring to RuntimeFlavor when you say RuntimeType ?

Yes. We end up passing the value through these scripts like that due to hitting an issue when trying to use the RuntimeType parameter directly in one of the yamls somewhere (I think it was in the runtime mobile yamls). RuntimeType was already a defined parameter that influenced the templates and I had issues passing it through all the way to our scripts so instead used RuntimeFlavor as the parameter in it's place.

[LoopedBard3]

Latest successful test run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2689882&view=results

[LoopedBard3]

As I am not 100% sure on the correct value for the llvm configuration value for iOS, I will confirm it and add it in another PR.

[matouskozak]

LGTM. Thanks for the changes.