In Allure.NUnit 2.10.0 package, an error began to occur in the AllureNUnitAttribute.AfterTest method

[AlexZhirkov1]

I'm submitting a ...

• bug report

What is the current behavior?

After updating Allure.NUnit to 2.10.0, tests began to fail. In the output I see that the scenario passed, but then an error occurred in the Allure code.
The output looks like this

Dodo.Tracker.ComponentTests.BDD.Scenarios.Consumers.WhenConsumeDataCatalogEntities<Country,Int32>.EntityIsNotUpdated_WhenPreviousVersionConsumed(auto<Int32>)`

System.Text.EncoderFallbackException : Unable to translate Unicode character \\uD8BC at index 655 to specified code page.
   at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknown, Int32 index)
   at System.Text.EncoderFallbackBuffer.InternalFallback(ReadOnlySpan`1 chars, Int32& charsConsumed)
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar, EncoderNLS encoder)
   at System.Text.Encoding.GetBytes(Char* pChars, Int32 charCount, Byte* pBytes, Int32 byteCount, EncoderNLS encoder)
   at System.Text.EncoderNLS.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.Dispose(Boolean disposing)
   at System.IO.TextWriter.Dispose()
   at Allure.Net.Commons.Writer.FileSystemResultsWriter.Write(Object allureObject, String fileSuffix)
   at Allure.Net.Commons.Writer.FileSystemResultsWriter.Write(TestResult testResult)
   at Allure.Net.Commons.AllureLifecycle.WriteTestCase()
   at NUnit.Allure.Core.AllureNUnitHelper.StopTestCase()
   at NUnit.Allure.Core.AllureNUnitAttribute.<>c__DisplayClass4_0.<AfterTest>b__0()
   at NUnit.Allure.Core.AllureNUnitAttribute.RunHookInRestoredAllureContext(ITest test, Action action)
   at NUnit.Allure.Core.AllureNUnitAttribute.AfterTest(ITest test)
   at NUnit.Framework.Internal.Commands.TestActionCommand.<>c__DisplayClass0_0.<.ctor>b__1(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.<>c__DisplayClass1_0.<Execute>b__1()
   at NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)


Fi=001,Fa=000,Pe=001 #  2> SCENARIO: EntityIsNotUpdated WhenPreviousVersionConsumed [id: "242"]
Fi=001,Fa=000,Pe=001 #  2>   STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "242" consumed...
Fi=001,Fa=000,Pe=001 #  2>   STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "242" consumed (Passed after 9ms)
Fi=001,Fa=000,Pe=001 #  2>   STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country"...
Fi=001,Fa=000,Pe=001 #  2>   STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after 3ms)
Fi=001,Fa=000,Pe=001 #  2>   STEP 3/3: THEN tracker entity "242" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country"...
Fi=001,Fa=000,Pe=001 #  2>   STEP 3/3: THEN tracker entity "242" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after 1ms)
Fi=002,Fa=000,Pe=000 #  2>   SCENARIO RESULT: Passed after 16ms
{"@timestamp":"2023-10-26T15:15:36.9140070+03:00","level":"Information","messageTemplate":"DebugInfo: EntityIsNotUpdated_WhenPreviousVersionConsumed started","message":"DebugInfo: EntityIsNotUpdated_WhenPreviousVersionConsumed started"}
SCENARIO: EntityIsNotUpdated WhenPreviousVersionConsumed [id: "242"]
  STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "242" consumed...
{"@timestamp":"2023-10-26T15:15:36.9214370+03:00","level":"Information","messageTemplate":"DebugInfo: EntityIsNotUpdated_WhenPreviousVersionConsumed finished","message":"DebugInfo: EntityIsNotUpdated_WhenPreviousVersionConsumed finished"}
  STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "242" consumed (Passed after 9ms)
  STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country"...
  STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after 3ms)
  STEP 3/3: THEN tracker entity "242" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country"...
  STEP 3/3: THEN tracker entity "242" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after 1ms)
  SCENARIO RESULT: Passed after 16ms

Despite the line “ SCENARIO RESULT: Passed after 16ms”, test failed with error "System.Text.EncoderFallbackException : Unable to translate Unicode character \uDD85 at index 942 to specified code page."

On previous versions of Allure.NUnit (2.10.0-preview.1 and 2.9.5-preview.1) the output looked like this

Dodo.Tracker.ComponentTests.BDD.Scenarios.Consumers.WhenConsumeDataCatalogEntities<Country,Int32>.EntityIsUpdated_WhenTheSameVersionConsumed(auto<Int32>)

Fi=003,Fa=000,Pe=001 #  4> SCENARIO: EntityIsUpdated WhenTheSameVersionConsumed [id: "176"]
Fi=003,Fa=000,Pe=001 #  4>   STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "176" consumed...
Fi=003,Fa=000,Pe=001 #  4>   STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "176" consumed (Passed after 5ms)
Fi=003,Fa=000,Pe=001 #  4>   STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country"...
Fi=003,Fa=000,Pe=001 #  4>   STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after 3ms)
Fi=003,Fa=000,Pe=001 #  4>   STEP 3/3: THEN tracker entity "176" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country"...
Fi=003,Fa=000,Pe=001 #  4>   STEP 3/3: THEN tracker entity "176" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after <1ms)
Fi=004,Fa=000,Pe=000 #  4>   SCENARIO RESULT: Passed after 12ms
{"@timestamp":"2023-10-26T15:02:10.7115450+03:00","level":"Information","messageTemplate":"DebugInfo: EntityIsUpdated_WhenTheSameVersionConsumed started","message":"DebugInfo: EntityIsUpdated_WhenTheSameVersionConsumed started"}
SCENARIO: EntityIsUpdated WhenTheSameVersionConsumed [id: "176"]
  STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "176" consumed...
{"@timestamp":"2023-10-26T15:02:10.7172410+03:00","level":"Information","messageTemplate":"DebugInfo: EntityIsUpdated_WhenTheSameVersionConsumed finished","message":"DebugInfo: EntityIsUpdated_WhenTheSameVersionConsumed finished"}
  STEP 1/3: GIVEN datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" id for entity "176" consumed (Passed after 5ms)
  STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country"...
  STEP 2/3: WHEN consume datacatalog event "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after 3ms)
  STEP 3/3: THEN tracker entity "176" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country"...
  STEP 3/3: THEN tracker entity "176" equvivalent to "Dodo.DataCatalog.Contracts.Country.v1.Country" (Passed after <1ms)
  SCENARIO RESULT: Passed after 12ms

Please tell us about your environment:

• Test framework: NUnit 3.13.3
• Allure adaptor: Allure.NUnit 2.10.0

Other information

I have attached two ***-result.json files from the allure-result folder.
5043da32e43b460290ac4bfdea306745-result.json - result without problems, obtained using the previous version of Allure.NUnit (2.9.5-preview.1)

e4921941c6c74e4ea2bcff9c4e354ab1-result.json- result with problem, obtained using the Allure.NUnit 2.10.0. In this file json breaks at the "step" tag

[delatrie]

Hello, @AlexZhirkov1 ! Thanks for the report. I'll try to reproduce it. Would be much of a help if you provide me with a reproducible case though (the code of a test or a repo).

[AlexZhirkov1]

Hello @delatrie.
We've managed to dig down to the root cause of this issue. You can find the (semi-)reproducible example @eranikid/allure-repro:

$ git clone [email protected]:eranikid/allure-repro.git && cd allure-repro && dotnet test

You might have to repeat the dotnet test part several times as it is random-based and only fails around 7 out of 10 times.

The actual issue is really just what it says in the exception message in the opening post - allureObject contains unrepresentable utf16. We'be been using bchavez/Bogus as a fake data generator for quite some time, and it literally states this in xmldoc for Random.String() method. Previous Allure.NUnit versions were somehow ignoring invalid/unrepresentable strings in the output, which you can verify by changing Allure.NUnit version in repro repository back down to 2.9.5-preview.1, but starting with 2.10.0 Allure.NUnit can't fails to handle such situation.

We'll just probably switch the fake data generation method, and leave the decision of whether or not you'd like this fixed in Allure.NUnit up to you.