Skip to content

Update the iOS scenarios build flow to match android #4834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 17, 2025
Merged

Update the iOS scenarios build flow to match android #4834

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
812e633
Update the iOS scenarios to use the same flow as android, bringing th…
LoopedBard3 Apr 14, 2025
2314fac
Update iOS builds to use the latest macos and xcode version.
LoopedBard3 Apr 14, 2025
3bc225c
Update the net_ios dll used for getting the ios version for mobile te…
LoopedBard3 Apr 14, 2025
b91e7c2
Test building SOD and Startup for x64.
LoopedBard3 Apr 14, 2025
67232b6
Standardize the iOS Scenario names to match android name convention.
LoopedBard3 Apr 16, 2025
e07f332
Add CodegeType to mobile iOS runs and setup RuntimeType and CodegenTy…
LoopedBard3 Apr 17, 2025
50bf9dd
Don't include the TTFD in the Blazor test name, especially as we are …
LoopedBard3 Apr 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 8 additions & 11 deletions eng/performance/maui_scenarios_ios.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@
<PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' != 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)/</PreparePayloadWorkItemBaseDirectory>

<NativeAOTCommandProps Condition="'$(RuntimeFlavor)' == 'coreclr'">--nativeaot true</NativeAOTCommandProps>
<!-- For non-default configurations, add the configuration to the name of the test (in the matching format), otherwise didn't add anything.
This will ensure that PowerBI's using the test name rather than the run configuration for the data continue to work properly and defaults are connected -->
<RunConfigsString>$(RuntimeFlavor)</RunConfigsString>
</PropertyGroup>

<Target Name="RemoveDotnetFromCorrelationStaging" BeforeTargets="BeforeTest">
Expand All @@ -29,19 +26,19 @@
</ItemDefinitionGroup>

<ItemGroup>
<MAUIiOSScenario Include=".NET iOS Default $(RunConfigsString)">
<MAUIiOSScenario Include=".NET iOS Default Template">
<ScenarioDirectoryName>netios</ScenarioDirectoryName>
<PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
<IPAName>NetiOSDefault</IPAName>
<PackageName>com.companyname.NetiOSDefault</PackageName>
</MAUIiOSScenario>
<MAUIiOSScenario Include="Maui iOS Default $(RunConfigsString)">
<MAUIiOSScenario Include="Maui iOS Default Template">
<ScenarioDirectoryName>mauiios</ScenarioDirectoryName>
<PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
<IPAName>MauiiOSDefault</IPAName>
<PackageName>net.dot.mauitesting</PackageName>
</MAUIiOSScenario>
<MAUIiOSScenario Include="Maui Blazor iOS Default $(RunConfigsString)">
<MAUIiOSScenario Include="Maui Blazor iOS Default Template">
<ScenarioDirectoryName>mauiblazorios</ScenarioDirectoryName>
<PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
<IPAName>MauiBlazoriOSDefault</IPAName>
Expand All @@ -51,7 +48,7 @@

<ItemGroup>
<PreparePayloadWorkItem Include="@(MAUIiOSScenario)">
<Command>sudo xcode-select -s /Applications/Xcode_16.2.app; $(Python) pre.py publish -f $(PERFLAB_Framework)-ios --self-contained -c Release -r ios-arm64 $(NativeAOTCommandProps) -o $(PreparePayloadWorkItemBaseDirectory)%(PreparePayloadWorkItem.ScenarioDirectoryName); cd ../; zip -r %(PreparePayloadWorkItem.ScenarioDirectoryName).zip %(PreparePayloadWorkItem.ScenarioDirectoryName)</Command>
<Command>sudo xcode-select -s /Applications/Xcode_16.3.app; $(Python) pre.py publish -f $(PERFLAB_Framework)-ios --self-contained -c Release -r ios-arm64 $(NativeAOTCommandProps) -o $(PreparePayloadWorkItemBaseDirectory)%(PreparePayloadWorkItem.ScenarioDirectoryName); cd ../; zip -r %(PreparePayloadWorkItem.ScenarioDirectoryName).zip %(PreparePayloadWorkItem.ScenarioDirectoryName)</Command>
<WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory>
</PreparePayloadWorkItem>
</ItemGroup>
Expand All @@ -61,11 +58,11 @@
<PreCommands>cp -r $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/%(HelixWorkItem.ScenarioDirectoryName) $HELIX_WORKITEM_ROOT/pub</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
</HelixWorkItem>
<HelixWorkItem Include="@(MAUIiOSScenario -> 'SOD - %(Identity) Unzipped')">
<HelixWorkItem Include="@(MAUIiOSScenario -> 'SOD - %(Identity) Extracted Size')">
<PreCommands>cp -r $HELIX_CORRELATION_PAYLOAD/$(PreparePayloadOutDirectoryName)/%(HelixWorkItem.ScenarioDirectoryName) $HELIX_WORKITEM_ROOT/pub; mv $HELIX_WORKITEM_ROOT/pub/%(HelixWorkItem.IPAName).ipa $HELIX_WORKITEM_ROOT/pub/%(HelixWorkItem.IPAName).zip; unzip -d $HELIX_WORKITEM_ROOT/pub $HELIX_WORKITEM_ROOT/pub/%(HelixWorkItem.IPAName).zip; rm $HELIX_WORKITEM_ROOT/pub/%(HelixWorkItem.IPAName).zip</PreCommands>
<Command>$(Python) test.py sod --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
</HelixWorkItem>
<XHarnessAppBundleToTest Include="Device Startup - iOS .NET Default $(RunConfigsString)">
<XHarnessAppBundleToTest Include="Device Startup - iOS .NET Default Template">
<AppBundlePath>$(ScenariosDir)netios.zip</AppBundlePath>
<WorkItemTimeout>00:15:00</WorkItemTimeout>
<TestTarget>ios-device</TestTarget>
Expand Down Expand Up @@ -93,7 +90,7 @@
]]>
</CustomCommands>
</XHarnessAppBundleToTest>
<XHarnessAppBundleToTest Include="Device Startup - iOS Maui Default $(RunConfigsString)">
<XHarnessAppBundleToTest Include="Device Startup - iOS Maui Default Template">
<AppBundlePath>$(ScenariosDir)mauiios.zip</AppBundlePath>
<WorkItemTimeout>00:15:00</WorkItemTimeout>
<TestTarget>ios-device</TestTarget>
Expand Down Expand Up @@ -122,7 +119,7 @@
</CustomCommands>
</XHarnessAppBundleToTest>
<!-- Now also disabled for normal mono runs. (Reenable for mono and native aot once fixed). Tracking issue: https://github.com/dotnet/performance/issues/3148 -->
<!-- <XHarnessAppBundleToTest Condition="'$(RuntimeFlavor)' == 'mono'" Include="Device Startup - iOS Maui Blazor Default $(RunConfigsString)">
<!-- <XHarnessAppBundleToTest Condition="'$(RuntimeFlavor)' == 'mono'" Include="Device Startup - iOS Maui Blazor Default Template">
<AppBundlePath>$(ScenariosDir)mauiblazorios.zip</AppBundlePath>
<WorkItemTimeout>00:15:00</WorkItemTimeout>
<TestTarget>ios-device</TestTarget>
Expand Down
10 changes: 4 additions & 6 deletions eng/pipelines/sdk-perf-jobs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,11 +436,10 @@ jobs:
jobParameters:
runKind: maui_scenarios_ios
projectFileName: maui_scenarios_ios.proj
dotnetVersionsLinks:
9.0: ./eng/Version.Details.xml
channels:
- 8.0
- main
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to get channels from a (default) value of a new template parameter? So it's kept uniform across all template

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That probably makes sense for most cases, although mobile is currently a special case as we only run main and not each main, 9.0, and 8.0.

runtimeFlavor: mono
codeGenType: FullAOT
additionalJobIdentifier: Mono
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}
Expand All @@ -455,11 +454,10 @@ jobs:
jobParameters:
runKind: maui_scenarios_ios
projectFileName: maui_scenarios_ios.proj
dotnetVersionsLinks:
9.0: ./eng/Version.Details.xml
channels:
- 8.0
- main
runtimeFlavor: coreclr
codeGenType: NativeAOT
additionalJobIdentifier: CoreCLR
${{ each parameter in parameters.jobParameters }}:
${{ parameter.key }}: ${{ parameter.value }}
Expand Down
4 changes: 2 additions & 2 deletions eng/pipelines/templates/build-machine-matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,9 @@ jobs:
parameters:
osGroup: osx
archType: x64
osVersion: 14
osVersion: 15
pool:
vmImage: 'macos-14'
vmImage: 'macos-15'
queue: OSX.13.Amd64.Iphone.Perf
machinePool: iPhoneMini12
${{ insert }}: ${{ parameters.jobParameters }}
7 changes: 7 additions & 0 deletions scripts/run_performance_job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -548,6 +548,13 @@ def run_performance_job(args: RunPerformanceJobArgs):
configurations["CodegenType"] = str(args.codegen_type)
configurations["RuntimeType"] = str(args.runtime_flavor)

# .NET iOS and .NET MAUI iOS sample app scenarios
if args.run_kind == "maui_scenarios_ios":
if not args.runtime_flavor in ("mono", "coreclr"):
raise Exception("Runtime flavor must be specified for maui_scenarios_ios")
configurations["CodegenType"] = str(args.codegen_type)
configurations["RuntimeType"] = str(args.runtime_flavor)

if ios_mono:
runtime_type = "Mono"
configurations["iOSLlvmBuild"] = str(args.ios_llvm_build)
Expand Down
15 changes: 7 additions & 8 deletions src/scenarios/mauiblazorios/pre.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,15 @@
import sys
from performance.logger import setup_loggers, getLogger
from shared import const
from shared.mauisharedpython import remove_aab_files, install_versioned_maui
from shared.mauisharedpython import remove_aab_files, install_latest_maui
from shared.precommands import PreCommands
from shared.versionmanager import versions_write_json, get_version_from_dll_powershell_ios
from shared.versionmanager import versions_write_json, get_sdk_versions
from test import EXENAME

setup_loggers(True)
precommands = PreCommands()
install_versioned_maui(precommands)
install_latest_maui(precommands)
precommands.print_dotnet_info()

# Setup the Maui folder
precommands.new(template='maui-blazor',
Expand All @@ -39,17 +40,15 @@

# Build the IPA
# NuGet.config file cannot be in the build directory due same cause as to https://github.com/dotnet/aspnetcore/issues/41397
shutil.copy('./MauiNuGet.config', './Nuget.config')
precommands.execute(['/p:_RequireCodeSigning=false', '/p:ApplicationId=net.dot.mauiblazortesting'])

output_dir = const.PUBDIR
if precommands.output:
output_dir = precommands.output
remove_aab_files(output_dir)

# Copy the MauiVersion to a file so we have it on the machine
maui_version = get_version_from_dll_powershell_ios(rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked/Microsoft.Maui.dll")
version_dict = { "mauiVersion": maui_version }
# Extract the versions of used SDKs from the linked folder DLLs
version_dict = get_sdk_versions(rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked", False)
versions_write_json(version_dict, rf"{output_dir}/versions.json")
print(f"Versions: {version_dict} from location " + rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked/Microsoft.Maui.dll")
print(f"Versions: {version_dict} from location " + rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked")

15 changes: 7 additions & 8 deletions src/scenarios/mauiios/pre.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@
import subprocess
from performance.logger import setup_loggers, getLogger
from shared import const
from shared.mauisharedpython import remove_aab_files, install_versioned_maui
from shared.mauisharedpython import remove_aab_files, install_latest_maui
from shared.precommands import PreCommands
from shared.versionmanager import versions_write_json, get_version_from_dll_powershell_ios
from shared.versionmanager import versions_write_json, get_sdk_versions
from test import EXENAME

setup_loggers(True)

precommands = PreCommands()
install_versioned_maui(precommands)
install_latest_maui(precommands)
precommands.print_dotnet_info()

# Setup the Maui folder
precommands.new(template='maui',
Expand All @@ -25,7 +26,6 @@
no_restore=False)

# Build the APK
shutil.copy('./MauiNuGet.config', './app/Nuget.config')
precommands.execute(['/p:_RequireCodeSigning=false', '/p:ApplicationId=net.dot.mauitesting'])

# Remove the aab files as we don't need them, this saves space
Expand All @@ -34,8 +34,7 @@
output_dir = precommands.output
remove_aab_files(output_dir)

# Copy the MauiVersion to a file so we have it on the machine
maui_version = get_version_from_dll_powershell_ios(rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked/Microsoft.Maui.dll")
version_dict = { "mauiVersion": maui_version }
# Extract the versions of used SDKs from the linked folder DLLs
version_dict = get_sdk_versions(rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked", False)
versions_write_json(version_dict, rf"{output_dir}/versions.json")
print(f"Versions: {version_dict} from location " + rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked/Microsoft.Maui.dll")
print(f"Versions: {version_dict} from location " + rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked")
17 changes: 8 additions & 9 deletions src/scenarios/netios/pre.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,18 @@
'''
import shutil
import sys
from performance.logger import setup_loggers
from performance.logger import setup_loggers, getLogger
from shared import const
from shared.mauisharedpython import remove_aab_files, install_versioned_maui
from shared.mauisharedpython import remove_aab_files, install_latest_maui
from shared.precommands import PreCommands
from shared.versionmanager import versions_write_json, get_version_from_dll_powershell_ios
from shared.versionmanager import versions_write_json, get_sdk_versions
from test import EXENAME

setup_loggers(True)

precommands = PreCommands()
install_versioned_maui(precommands)
install_latest_maui(precommands)
precommands.print_dotnet_info()

# Setup the .NET iOS folder
precommands.new(template='ios',
Expand All @@ -24,7 +25,6 @@
no_restore=False)

# Build the APK
shutil.copy('./MauiNuGet.config', './app/Nuget.config')
precommands.execute(['/p:_RequireCodeSigning=false', '/p:ApplicationId=net.dot.xamarintesting'])

# Remove the aab files as we don't need them, this saves space
Expand All @@ -33,8 +33,7 @@
output_dir = precommands.output
remove_aab_files(output_dir)

# Copy the XamarinVersion to a file so we have it on the machine
net_ios_version = get_version_from_dll_powershell_ios(rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked/Microsoft.iOS.dll")
version_dict = { "netIosVersion": net_ios_version }
# Extract the versions of used SDKs from the linked folder DLLs
version_dict = get_sdk_versions(rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked", False)
versions_write_json(version_dict, rf"{output_dir}/versions.json")
print(f"Versions: {version_dict} from location " + rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked/Microsoft.iOS.dll")
print(f"Versions: {version_dict} from location " + rf"./{const.APPDIR}/obj/Release/{precommands.framework}/ios-arm64/linked")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably out of scope for this PR but would it be possible and would it make sense to unify all the pre.py scripts to avoid code/configs duplications to avoid possible misconfigurations/bugs in the future?

2 changes: 1 addition & 1 deletion src/scenarios/shared/versionmanager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def parse_version_output(output: str) -> tuple[str, str]:

mobile_sdks = {
"net_android": "Mono.Android.dll",
"net_ios": "Mono.iOS.dll",
"net_ios": "Microsoft.iOS.dll",
"net_maui": "Microsoft.Maui.dll",
"runtime": "System.Runtime.dll"
}
Expand Down
Loading