Skip to content

Perf_Regex_Industry_SliceSlice.Count Benchmarks OOM under public CI with [MemoryRandomization] #4025

@DrewScoggins

Description

@DrewScoggins
[2024/03/01 19:31:47][INFO] // Benchmark: Perf_Regex_Industry_SliceSlice.Count: Job-PTIUIJ(OutlierMode=DontRemove, PowerPlanMode=00000000-0000-0000-0000-000000000000, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MemoryRandomization=True, MinIterationCount=3, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0) [Options=IgnoreCase, NonBacktracking]
[2024/03/01 19:31:47][INFO] // *** Execute ***
[2024/03/01 19:31:47][INFO] // Launch: 1 / 1
[2024/03/01 19:31:47][INFO] // Execute: dotnet Job-MMSQNV.dll --anonymousPipes 2816 680 --benchmarkName "System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, NonBacktracking)" --job "OutlierMode=DontRemove, PowerPlanMode=00000000-0000-0000-0000-000000000000, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MemoryRandomization=True, MinIterationCount=3, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0" --benchmarkId 401 in C:\h\w\ADC00983\w\C8BA0A26\e\performance\artifacts\bin\MicroBenchmarks\Release\net9.0\Job-MMSQNV\bin\Release\net9.0
[2024/03/01 19:31:47][INFO] // BeforeAnythingElse
[2024/03/01 19:31:47][INFO] 
[2024/03/01 19:31:47][INFO] // Benchmark Process Environment Information:
[2024/03/01 19:31:47][INFO] // BenchmarkDotNet v0.13.13-nightly.20240114.129
[2024/03/01 19:31:47][INFO] // Runtime=.NET 9.0.0 (9.0.24.12902), X86 RyuJIT AVX2
[2024/03/01 19:31:47][INFO] // GC=Concurrent Workstation
[2024/03/01 19:31:47][INFO] // HardwareIntrinsics=AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT VectorSize=256
[2024/03/01 19:31:47][INFO] // Job: Job-FCDMOX(OutlierMode=DontRemove, PowerPlanMode=00000000-0000-0000-0000-000000000000, InvocationCount=1, IterationCount=1, IterationTime=250.0000 ms, MaxIterationCount=20, MemoryRandomization=True, MinIterationCount=3, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=0)
[2024/03/01 19:31:47][INFO] 
[2024/03/01 19:31:54][INFO] // BeforeActualRun
[2024/03/01 19:32:01][INFO] 
[2024/03/01 19:32:01][INFO] OutOfMemoryException!
[2024/03/01 19:32:01][INFO] BenchmarkDotNet continues to run additional iterations until desired accuracy level is achieved. It's possible only if the benchmark method doesn't have any side-effects.
[2024/03/01 19:32:01][INFO] If your benchmark allocates memory and keeps it alive, you are creating a memory leak.
[2024/03/01 19:32:01][INFO] You should redesign your benchmark and remove the side-effects. You can use `OperationsPerInvoke`, `IterationSetup` and `IterationCleanup` to do that.
[2024/03/01 19:32:01][INFO] 
[2024/03/01 19:32:01][INFO] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
[2024/03/01 19:32:01][INFO]  ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
[2024/03/01 19:32:01][INFO]    at System.Collections.Generic.Dictionary`2.Resize(Int32 newSize, Boolean forceNewHashCodes)
[2024/03/01 19:32:01][INFO]    at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.CharSetSolver.ApplyBinaryOp(BooleanOperation op, BDD set1, BDD set2)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.RegexNodeConverter.<CreateBDDFromSetString>g__Compute|5_0(String set)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.RegexNodeConverter.ConvertToSymbolicRegexNode(RegexNode root)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Symbolic.SymbolicRegexRunnerFactory..ctor(RegexTree regexTree, RegexOptions options, TimeSpan matchTimeout)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Regex..ctor(String pattern, RegexOptions options, TimeSpan matchTimeout, CultureInfo culture)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Regex..ctor(String pattern, RegexOptions options)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.<Setup>b__7_1(String w) in C:\h\w\ADC00983\w\C8BA0A26\e\performance\src\benchmarks\micro\libraries\System.Text.RegularExpressions\Perf.Regex.Industry.cs:line 109
[2024/03/01 19:32:01][INFO]    at System.Linq.Enumerable.IteratorSelectIterator`2.LazyToArray()
[2024/03/01 19:32:01][INFO]    at System.Linq.Enumerable.IteratorSelectIterator`2.ToArray()
[2024/03/01 19:32:01][INFO]    at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
[2024/03/01 19:32:01][INFO]    at System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Setup() in C:\h\w\ADC00983\w\C8BA0A26\e\performance\src\benchmarks\micro\libraries\System.Text.RegularExpressions\Perf.Regex.Industry.cs:line 109
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.Engine.RandomizeManagedHeapMemory()
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.Engine.RunIteration(IterationData data)
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.EngineStage.RunIteration(IterationMode mode, IterationStage stage, Int32 index, Int64 invokeCount, Int32 unrollFactor)
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.EngineActualStage.RunSpecific(Int64 invokeCount, IterationMode iterationMode, Int32 iterationCount, Int32 unrollFactor)
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.EngineActualStage.Run(Int64 invokeCount, IterationMode iterationMode, Boolean runAuto, Int32 unrollFactor, Boolean forceSpecific)
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.EngineActualStage.RunWorkload(Int64 invokeCount, Int32 unrollFactor, Boolean forceSpecific)
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Engines.Engine.Run()
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Autogenerated.Runnable_401.Run(IHost host, String benchmarkName) in C:\h\w\ADC00983\w\C8BA0A26\e\performance\artifacts\bin\MicroBenchmarks\Release\net9.0\Job-MMSQNV\Job-MMSQNV.notcs:line 70934
[2024/03/01 19:32:01][INFO]    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[2024/03/01 19:32:01][INFO]    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
[2024/03/01 19:32:01][INFO]    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
[2024/03/01 19:32:01][INFO]    --- End of inner exception stack trace ---
[2024/03/01 19:32:01][INFO]    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
[2024/03/01 19:32:01][INFO]    at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
[2024/03/01 19:32:01][INFO]    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
[2024/03/01 19:32:01][INFO]    at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
[2024/03/01 19:32:01][INFO]    at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) in C:\h\w\ADC00983\w\C8BA0A26\e\performance\artifacts\bin\MicroBenchmarks\Release\net9.0\Job-MMSQNV\Job-MMSQNV.notcs:line 57
[2024/03/01 19:32:01][INFO] // AfterAll
[2024/03/01 19:32:01][INFO] No Workload Results were obtained from the run.
[2024/03/01 19:32:01][INFO] // Benchmark Process 3516 has exited with code -1.
[2024/03/01 19:32:01][INFO] 
[2024/03/01 19:32:01][INFO] // ** Remained 10 (2.4%) benchmark(s) to run. Estimated finish 2024-03-01 19:32 (0h 0m from now) **
[2024/03/01 19:32:01][INFO] �]9;4;1;97�\// ***** BenchmarkRunner: Finish  *****

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions