Unhandled crashes are not been reported in release mode on iOS apps

[mackayn]

Package

Sentry.Maui

.NET Flavor

.NET

.NET Version

9.0.200

OS

iOS

SDK Version

5.2.0

Self-Hosted Sentry Version

No response

Steps to Reproduce

Setup Sentry in builder

		builder
			.UseMauiApp<App>()
			.UseSentry(options => {
				options.Dsn = "DSN";
#if DEBUG
        options.Debug = true;
        options.TracesSampleRate = 1.0;
#endif
				options.IsGlobalModeEnabled = true;
				options.StackTraceMode = StackTraceMode.Enhanced;
			})

iOS Release config

  <PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net9.0-ios|AnyCPU'">
    <CodesignProvision>Automatic</CodesignProvision>
    <CodesignKey>iPhone Distribution</CodesignKey>
    <TreatWarningsAsErrors>false</TreatWarningsAsErrors>
    <RuntimeIdentifier>ios-arm64</RuntimeIdentifier>
    <ProvisioningType>manual</ProvisioningType>
    <UseInterpreter>false</UseInterpreter>
    <MtouchLink>SdkOnly</MtouchLink>
    <ArchiveOnBuild>true</ArchiveOnBuild>
    <BuildIpa>True</BuildIpa>
    <IsAotCompatible>true</IsAotCompatible>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

Expected Result

Unhandled exceptions in debug profile are caught and shown in the sentry dashboard for the iOS .NET9 MAUI (9.0.40) app (AOT compiled, doesn't work either is not AOT compiled).

In release mode, unhandled exceptions is captured.

Actual Result

Unhandled exceptions in debug profile are caught and shown in the sentry dashboard for the .NET9 MAUI (9.0.40) app.

In release mode, nothing is captured on unhandled exceptions making it very difficult to diagnose crashes since porting to .NET9.

Is there a way to manually add debug symbols like you could with AppCenter, in this case the builds are not generated via a pipeline.

[bruno-garcia]

In release mode, nothing is captured on unhandled exceptions making it very difficult to diagnose crashes since porting to .NET9.

Do you have any logs of the sdk you can share? you can turn it on with Debug=true and get them from the mac console streaming ur phones logs.

also if you have a link to your Sentry project I can see if there's data ben dropped. You can see that too under Stats in the left nav bar

Is there a way to manually add debug symbols like you could with AppCenter, in this case the builds are not generated via a pipeline.

sentry can upload during msbuild build (https://docs.sentry.io/platforms/dotnet/configuration/msbuild/ ) but if you want to upload outside of msbuild you can with our CLI (which is what we use under the hood from the sdk too): https://docs.sentry.io/platforms/apple/dsym/

[mackayn]

In release mode, nothing is captured on unhandled exceptions making it very difficult to diagnose crashes since porting to .NET9.

Do you have any logs of the sdk you can share? you can turn it on with Debug=true and get them from the mac console streaming ur phones logs.

also if you have a link to your Sentry project I can see if there's data ben dropped. You can see that too under Stats in the left nav bar

Is there a way to manually add debug symbols like you could with AppCenter, in this case the builds are not generated via a pipeline.

sentry can upload during msbuild build (https://docs.sentry.io/platforms/dotnet/configuration/msbuild/ ) but if you want to upload outside of msbuild you can with our CLI (which is what we use under the hood from the sdk too): https://docs.sentry.io/platforms/apple/dsym/

I can't share the URL here but do you have a mailbox I can send the URL's to ?
There is data in stats but as mentioned, it's only debug crashes with a debugger attached

thanks for the link for the symbols but a bit of a clunky process if your a VS2022 Windows users and you have to look for the build manually on the mac host

[jamescrosswell]

thanks for the link for the symbols but a bit of a clunky process if your a VS2022 Windows users and you have to look for the build manually on the mac host

If you're doing it via MS Build, the transitive build file should run all the necessary CLI commands for you from wherever the build is running... and you should get logs of all that happening in your Debug window as the build occurs. That experience should be pretty similar to what it would feel like to do this directly on a Mac using Jetbrains Rider or the command line.

If you want to run the CLI commands yourself manually then yes, using a remote Mac will make it a bit more painful. But then everything is a bit more painful when you try to build iOS apps from Windows. That's basically a consequence of the requirement to use Xcode to build iOS apps (so not specific to Sentry).

[mackayn]

thanks for the link for the symbols but a bit of a clunky process if your a VS2022 Windows users and you have to look for the build manually on the mac host

If you're doing it via MS Build, the transitive build file should run all the necessary CLI commands for you from wherever the build is running... and you should get logs of all that happening in your Debug window as the build occurs. That experience should be pretty similar to what it would feel like to do this directly on a Mac using Jetbrains Rider or the command line.

If you want to run the CLI commands yourself manually then yes, using a remote Mac will make it a bit more painful. But then everything is a bit more painful when you try to build iOS apps from Windows. That's basically a consequence of the requirement to use Xcode to build iOS apps (so not specific to Sentry).

That's not particularly helpful. It doesn't explain why unhandled crashes (race condition I suspect as it was ok in .NET8 release mode) aren't caught but they are in debug build config, we need to have confidence it does that basics before we can release the apps.

[mackayn]

Something odd I did notice was that a couple of debug crashes that occurred while I was refactoring on Friday didn't show but they are showing up this morning, I'm just using the developer plan for evaluation. Just wondering if there's any limitations reporting wise for that plan ?

[jamescrosswell]

That's not particularly helpful. It doesn't explain why unhandled crashes (race condition I suspect as it was ok in .NET8 release mode) aren't caught but they are in debug build config

To work that out, I think we really need to see what is being logged when you set Debug=true (when initialising the Sentry SDK). There would typically be something logged when the SDK tries to send the crash to Sentry... some kind of error message explaining why the crash isn't being sent.

Unhandled crashes are definitely captured on iOS (you can run the Sentry samples to verify this)... So it's a matter of working out why it's not working for your particular configuration.

One thing that isn't super intuitive is that crashes which terminate the process might not be sent to Sentry before the process terminates... Typically they get written to disk and sent the next time your application starts. That might explain why you got crash reports from Friday on Monday morning. There is also an ingestion pipeline at Sentry and it can take some time for events that the SDK has sent to show in the Sentry dashboard... but typically it's minutes (not days) in that case.

[mackayn]

That's not particularly helpful. It doesn't explain why unhandled crashes (race condition I suspect as it was ok in .NET8 release mode) aren't caught but they are in debug build config

To work that out, I think we really need to see what is being logged when you set Debug=true (when initialising the Sentry SDK). There would typically be something logged when the SDK tries to send the crash to Sentry... some kind of error message explaining why the crash isn't being sent.

Unhandled crashes are definitely captured on iOS (you can run the Sentry samples to verify this)... So it's a matter of working out why it's not working for your particular configuration.

One thing that isn't super intuitive is that crashes which terminate the process might not be sent to Sentry before the process terminates... Typically they get written to disk and sent the next time your application starts. That might explain why you got crash reports from Friday on Monday morning. There is also an ingestion pipeline at Sentry and it can take some time for events that the SDK has sent to show in the Sentry dashboard... but typically it's minutes (not days) in that case.

After every crash, I always re-run the app and give it time to send any captured logs, I was doing that as AppCenter worked in a similar fashion.

[mackayn]

That's not particularly helpful. It doesn't explain why unhandled crashes (race condition I suspect as it was ok in .NET8 release mode) aren't caught but they are in debug build config

To work that out, I think we really need to see what is being logged when you set Debug=true (when initialising the Sentry SDK). There would typically be something logged when the SDK tries to send the crash to Sentry... some kind of error message explaining why the crash isn't being sent.

Unhandled crashes are definitely captured on iOS (you can run the Sentry samples to verify this)... So it's a matter of working out why it's not working for your particular configuration.

One thing that isn't super intuitive is that crashes which terminate the process might not be sent to Sentry before the process terminates... Typically they get written to disk and sent the next time your application starts. That might explain why you got crash reports from Friday on Monday morning. There is also an ingestion pipeline at Sentry and it can take some time for events that the SDK has sent to show in the Sentry dashboard... but typically it's minutes (not days) in that case.

Ok, some progress, catching the crash in release mode if I add this in the constructor of the app.xaml.cs of my app

AppDomain.CurrentDomain.FirstChanceException += CurrentDomainOnFirstChanceException;

private void CurrentDomainOnFirstChanceException(object sender, FirstChanceExceptionEventArgs e)
{
	SentrySdk.CaptureException(e.Exception);
}

Found via slack.

[jamescrosswell]

Hm, interesting. We're using Runtime.MarshalManagedException to intercept exceptions... In all the tests I ran that caught everything, but it seems not. Thanks heaps @mackayn - we'll try to reproduce locally and find a way to wire up the FirstChanceException without that resulting in duplicates.

[mackayn]

Hm, interesting. We're using Runtime.MarshalManagedException to intercept exceptions... In all the tests I ran that caught everything, but it seems not. Thanks heaps @mackayn - we'll try to reproduce locally and find a way to wire up the FirstChanceException without that resulting in duplicates.

Just for a little context, here's the two errors that were captured

System.AggregateException
A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread.

System.NotSupportedException
ConstructorContainsNullParameterNames, System.Collections.Generic.KeyValuePair`2[System.String,System.Boolean]

The release build in .NET8 in the iOS client ran okay without the interpreter enabled but we had to enable it for .NET9 and all was good, as the .NET9 AOT was so fast already, I didn't see any performance degredation from enabling it

  <PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net9.0-ios|AnyCPU'">
    <CodesignProvision>Automatic</CodesignProvision>
    <CodesignKey>iPhone Distribution</CodesignKey>
    <TreatWarningsAsErrors>false</TreatWarningsAsErrors>
    <RuntimeIdentifier>ios-arm64</RuntimeIdentifier>
    <ProvisioningType>manual</ProvisioningType>
    <UseInterpreter>true</UseInterpreter>
    <MtouchLink>SdkOnly</MtouchLink>
    <ArchiveOnBuild>true</ArchiveOnBuild>
    <BuildIpa>True</BuildIpa>
    <IsAotCompatible>true</IsAotCompatible>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

Hooking up the event handler does cause Android SIM deployment to fail in debug config so it would be a release only scenario.

Overall, really impressed with Sentry, console is great and the breadcrumb feature is awesome.

[bruno-garcia]

we definitely don't want to send FirstChanceException out of the box. since that means if the user suppresses those we'd anyway use up their quota.
If the exception passed through FirstChance and didn't get handled. it should end up on UnhandledExceptionHandler and we should get it then.

[jamescrosswell]

The trouble is that UnhandledExceptionHandler does not always fire on MAUI.

See here:

• Issues with unhandled exceptions dotnet/macios#15252 (comment)

The UnhandledException hook isn't fired when using CoreCLR (on macOS or when using NativeAOT)... which is why we're using Runtime.MarshalManagedException.

Apparently @mackayn has found a scenario where Runtime.MarshalManagedException isn't firing either though. I haven't had a chance to dig into this yet or reproduce/confirm the issue.

@mackayn have you made any progress understanding where the crashes are coming from and how we might reproduce this? Simple scenarios like the unhandled crashes caused by our sample app are captured by Sentry, so we need to work out what's happening in your scenario that's not happening in the context of that sample app to be able to reproduce and resolve the issue.

[mackayn]

The trouble is that UnhandledExceptionHandler does not always fire on MAUI.

See here:

• Issues with unhandled exceptions dotnet/macios#15252 (comment)

The UnhandledException hook isn't fired when using CoreCLR (on macOS or when using NativeAOT)... which is why we're using Runtime.MarshalManagedException.

Apparently @mackayn has found a scenario where Runtime.MarshalManagedException isn't firing either though. I haven't had a chance to dig into this yet or reproduce/confirm the issue.

@mackayn have you made any progress understanding where the crashes are coming from and how we might reproduce this? Simple scenarios like the unhandled crashes caused by our sample app are captured by Sentry, so we need to work out what's happening in your scenario that's not happening in the context of that sample app to be able to reproduce and resolve the issue.

No, as mentioned, the crash was low level. Identical C# code that worked in .NET8 was crashing the app in .NET9, compiling with the interpreter enabled in iOS solves the issue, something was been stripped out. It's worth mentioning that the crash occurred whether compiling with or without AOT compilation. We're only trimming the Sdk (SdkOnly).

System.NotSupportedException
ConstructorContainsNullParameterNames, System.Collections.Generic.KeyValuePair`2[System.String,System.Boolean]

Having looked at macios/#15252, If this is a .NET issue and not a Sentry issue, I am happy for this to be closed, oddly UnhandledExceptionHandler helped me get a clue, maybe on that test I'd compiled with AOT disabled but from reading the macOS issue, maybe it's not a great idea to hook it up as we're using AOT compilation in iOS now as it improves app startup performance.

[bruno-garcia]

Even with this being a dotnet issue we could keep this ticket open and try to find a work around. such as using FirstChanceException and letting some types of errors through, for example.