Skip to content

Add Assert.Scope() for soft assertions#7355

Merged
Evangelink merged 26 commits intomainfrom
dev/amauryleve/soft-assertion
Apr 20, 2026
Merged

Add Assert.Scope() for soft assertions#7355
Evangelink merged 26 commits intomainfrom
dev/amauryleve/soft-assertion

Conversation

@Evangelink
Copy link
Copy Markdown
Member

Fixes #571

Comment thread src/TestFramework/TestFramework/Assertions/Assert.cs
Comment thread src/TestFramework/TestFramework/Assertions/Assert.cs Outdated
Comment thread src/TestFramework/TestFramework/Assertions/Assert.Scope.cs
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
# Conflicts:
#	src/TestFramework/TestFramework/Assertions/Assert.cs
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
@Evangelink Evangelink marked this pull request as ready for review February 9, 2026 13:39
Comment thread docs/RFCs/011-Soft-Assertions-Nullability-Design.md
Comment thread docs/RFCs/011-Soft-Assertions-Nullability-Design.md Outdated
Comment thread docs/RFCs/011-Soft-Assertions-Nullability-Design.md Outdated
Comment thread src/TestFramework/TestFramework/Assertions/Assert.cs Outdated
@Evangelink Evangelink marked this pull request as draft February 10, 2026 10:20
@Evangelink Evangelink marked this pull request as ready for review February 12, 2026 12:01
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements soft assertions through a new Assert.Scope() API that allows multiple assertion failures to be collected and reported together, addressing issue #571. When using Assert.Scope(), assertions failures are collected instead of thrown immediately, and all collected failures are reported when the scope is disposed.

Changes:

  • Adds Assert.Scope() experimental API for soft assertion collection
  • Refactors assertion failure reporting: introduces ReportAssertFailed for soft assertions while keeping ThrowAssertFailed for hard assertions (Assert.Fail, Assert.Inconclusive)
  • Updates all assertion methods to use ReportAssertFailed except for hard assertions and parameter validation
  • Includes comprehensive RFC documentation explaining design decisions around nullability annotations and postconditions

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/TestFramework/TestFramework/Assertions/AssertScope.cs Core implementation of assertion scope using AsyncLocal and ConcurrentQueue for thread-safe failure collection
src/TestFramework/TestFramework/Assertions/Assert.Scope.cs Public API for Assert.Scope() marked as experimental
src/TestFramework/TestFramework/Assertions/Assert.cs Adds ReportAssertFailed method with soft assertion support and refactors message formatting
src/TestFramework/TestFramework/Assertions/Assert.*.cs Updates all assertion classes (AreEqual, IsTrue, IsNull, etc.) to use ReportAssertFailed
src/TestFramework/TestFramework/Assertions/StringAssert.cs Updates string assertions to use ReportAssertFailed
src/TestFramework/TestFramework/Assertions/CollectionAssert.cs Updates collection assertions to use ReportAssertFailed
src/TestFramework/TestFramework/Resources/FrameworkMessages.resx Adds new messages for scope failures, removes AssertThatFailedFormat, updates AssertionFailed format
src/TestFramework/TestFramework/Resources/xlf/*.xlf Updates all localization files with new strings marked as needing translation
src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt Adds Assert.Scope() to public API
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs Comprehensive test coverage for Assert.Scope() functionality
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.That.cs Updates test expectations for Assert.That message format changes
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ThrowsExceptionTests.cs Renames test to match ReportAssertFailed
docs/RFCs/011-Soft-Assertions-Nullability-Design.md Detailed RFC documenting design decisions, nullability annotation handling, and trade-offs

Copy link
Copy Markdown
Member

@Youssef1313 Youssef1313 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. But I would love to see a full E2E test

Copilot AI review requested due to automatic review settings February 20, 2026 13:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 41 out of 41 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/TestFramework/TestFramework/Assertions/Assert.cs:235

  • CheckParameterNotNull now throws via CreateAssertFailedException directly. This bypasses LaunchDebuggerIfNeeded() and also loses the [StackTraceHidden] behavior provided by ThrowAssertFailed, so null-parameter failures will behave differently from other assertion failures. Consider routing this through ThrowAssertFailed(assertionName, finalMessage) (or otherwise ensuring debugger-launch + stacktrace-hiding parity).
    internal static void CheckParameterNotNull([NotNull] object? param, string assertionName, string parameterName)
    {
        if (param is null)
        {
            string finalMessage = string.Format(CultureInfo.CurrentCulture, FrameworkMessages.NullParameterToAssert, parameterName);
            throw CreateAssertFailedException(assertionName, finalMessage);
        }

Comment thread docs/RFCs/011-Soft-Assertions-Nullability-Design.md Outdated
Comment thread src/TestFramework/TestFramework/Assertions/Assert.cs
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
Comment thread src/TestFramework/TestFramework/Assertions/AssertScope.cs Outdated
…e preservation

Address review feedback: capture ExceptionDispatchInfo at the point errors are
added to the scope (in AddError) rather than at Dispose time. The queue now
stores ExceptionDispatchInfo instead of raw AssertFailedException, so:
- Single error: Throw() preserves the original assertion-site stack trace.
- Multiple errors: SourceException is extracted from each ExceptionDispatchInfo
  via Array.ConvertAll and passed to AggregateException, retaining all original
  stack traces.
Copilot AI review requested due to automatic review settings April 20, 2026 10:16
…e preservation

Address review feedback: capture ExceptionDispatchInfo at the point errors are
added to the scope (in AddError) rather than at Dispose time. The queue now
stores ExceptionDispatchInfo instead of raw AssertFailedException, so:
- Single error: Throw() preserves the original assertion-site stack trace.
- Multiple errors: SourceException is extracted from each ExceptionDispatchInfo
  via Array.ConvertAll and passed to AggregateException, retaining all original
  stack traces.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 41/41 changed files
  • Comments generated: 2

Comment thread src/TestFramework/TestFramework/Assertions/Assert.cs
Comment thread docs/RFCs/011-Soft-Assertions-Nullability-Design.md
@Evangelink Evangelink merged commit c0e4728 into main Apr 20, 2026
11 checks passed
@Evangelink Evangelink deleted the dev/amauryleve/soft-assertion branch April 20, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Is there any way to soft assert in MSTEST just like verify assert in TESTNG

4 participants