Add MSTEST0070 analyzer validating [Condition] member references#9076
Open
Evangelink wants to merge 1 commit into
Open
Add MSTEST0070 analyzer validating [Condition] member references#9076Evangelink wants to merge 1 commit into
Evangelink wants to merge 1 commit into
Conversation
Adds `ConditionShouldBeValidAnalyzer` (MSTEST0070) which catches at build time the same errors that `ConditionAttribute.IsConditionMet` throws at runtime, so typos and refactors don't silently break test gating. Rules emitted (one diagnostic per member name in the attribute): - `MemberNotFoundRule` -- the named member doesn't exist on the type - `MemberNotPublicRule` -- member is not `public` - `MemberNotStaticRule` -- member is not `static` - `MemberWrongKindRule` -- member is an event/nested type/etc. - `MemberWrongReturnTypeRule` -- property/field/method doesn't return `bool` - `MethodHasParametersRule` -- referenced method has parameters - `PropertyNotReadableRule` -- property has no public getter The analyzer fires on any symbol carrying `[Condition]` (test class or test method) and validates every member name passed to the attribute, including the `params string[]` overload and the `ConditionMode`-prefixed overloads. Inherited public static members are recognized. Resource strings localized via the standard `UpdateXlf` flow. Listed in `AnalyzerReleases.Unshipped.md` as MSTEST0070, Usage, Warning. Adds 18 analyzer tests covering all rules, multi-attribute stacking, inherited members, `ConditionMode.Exclude`, and the additional-members overload. All 18 tests pass on net8.0. Follow-up to #9071 (per design review). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
🧪 Test quality grade — PR #9076
This advisory comment was generated automatically. Grades are heuristic
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #9071 (per design review).
Summary
Adds
MSTEST0070 — ConditionShouldBeValidAnalyzer, which catches at build time the same errors thatConditionAttribute.IsConditionMetthrows at runtime, so typos and refactors don't silently break test gating.This is the analyzer counterpart called out as a follow-up in #9071 and addresses the user feedback "add an analyzer to detect if the member targets (or type detected) is invalid".
Diagnostics
One diagnostic is produced per invalid member name on each
[Condition]attribute.MemberNotFoundRuleMemberNotPublicRulepublic.MemberNotStaticRulestatic.MemberWrongKindRuleMemberWrongReturnTypeRulebool.MethodHasParametersRulePropertyNotReadableRuleAll seven diagnostics share id
MSTEST0070with different message formats, matching howMSTEST0018(DynamicData) is structured.Example
Coverage
AttributeUsage(Class | Method)).(Type, string)ctor, theparams string[]overload, and theConditionMode-prefixed overloads.[Condition]attributes are each validated independently.BaseTypechain), matchingBindingFlags.Public | Static | FlattenHierarchy.property → field → method), so if a property and method share a name only the property is validated — consistent with what would actually be invoked.Tests
18 tests in
test/UnitTests/MSTest.Analyzers.UnitTests/ConditionShouldBeValidAnalyzerTests.cs:[Condition]attributes on one method (each validated),ConditionMode.Excludector overload, attribute on class targetAll 18 pass on
net8.0.Files touched
src/Analyzers/MSTest.Analyzers/ConditionShouldBeValidAnalyzer.cstest/UnitTests/MSTest.Analyzers.UnitTests/ConditionShouldBeValidAnalyzerTests.cssrc/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs(addedConditionShouldBeValidRuleId = "MSTEST0070")src/Analyzers/MSTest.Analyzers/Helpers/WellKnownTypeNames.cs(addedConditionAttributeentry)src/Analyzers/MSTest.Analyzers/Resources.resx+ 13.xlffiles (regenerated viadotnet msbuild /t:UpdateXlf)src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Unshipped.mdNotes for reviewers
Warning, enabled by default — matchesMSTEST0018(DynamicData) which solves the same class of problem. Happy to downgrade toInfoif you'd prefer a softer rollout.https://learn.microsoft.com/dotnet/core/testing/mstest-analyzers/mstest0070will be authored as a separatedotnet/docsPR after this merges.DynamicDataShouldBeValidAnalyzer.