[DRAFT] Add Debug CI runs #3589

paulmedynski · 2025-08-29T17:32:23Z

Description

We want to start building and running tests in Debug configuration for some of our pipeline runs. Here is how the pipelines will change:

All pushes to a PR will trigger a run in Debug mode.
CI runs in Release mode are triggered on merges to main. These are batched up to avoid more than one run at a time.
Scheduled runs in Debug and Release mode are setup for nightly (Release) and weekly (Debug).

Unfortunately, Azure Pipelines do not permit dynamic top-level parameters to pipelines, so we can't choose Debug vs Release at trigger time. As a result, the CI-SqlClient and CI-SqlClient-Package pipelines will be used for the batch and scheduled triggers. Two new pipelines, PR-SqlClient and PR-SqlClient-Package will be created to service the PR triggered runs.

However, I can't make these changes yet since Azure Pipelines doen't permit creating new pipelines from YAML files on a branch. The new definition files must be merged to main before the pipelines can be created. So, for now, CI-SqlClient and CI-SqlClient-Package will be updated to use Debug mode so PR triggered tests can run. Once this PR is merged to main, a followup PR will include the shuffling around of YAML files and Azure Pipeline creation/updating.

Changes include:

Changed default build configuration from Release to Debug for CI-SqlClient[-Package] pipelines.
Plumbed 'builConfiguration' template parameter down through all of the pipeline YML files.
Removed the pipeline variable $(Configuration) in favour of buildConfiguration parameter.
Added environment variable override for unit test config file.

Testing

This PR will testt the CI-SqlClient and CI-SqlClient-Package pipelines in Debug mode.
After merging I will manually test the CI batch and scheculed runs.

paulmedynski

Adding commentary for reviewers.

paulmedynski · 2025-08-29T19:16:32Z

eng/pipelines/common/templates/jobs/build-signed-package-job.yml

    parameters:
      publishSymbols: ${{ parameters['PublishSymbols'] }}
      symbolsArtifactName: mds_symbols_$(System.TeamProject)_$(Build.Repository.Name)_$(Build.SourceBranchName)_$(NuGetPackageVersion)_$(System.TimelineId)
+      buildConfiguration: Release


This job is only used for official release builds, so we hardcode Release here.

paulmedynski · 2025-08-29T19:16:58Z

eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml

    default: $(Platform)

-  - name: configuration
+  - name: buildConfiguration


I standardized on this parameter name, values, and default everywhere.

paulmedynski · 2025-08-29T19:18:06Z

eng/pipelines/common/templates/steps/build-all-configurations-signed-dlls-step.yml

    inputs:
      solution: '**/build.proj'
-      configuration: '${{parameters.Configuration }}'
+      configuration: '${{parameters.buildConfiguration }}'


The MSBuild@1 task has a configuration parameter for the build configuration, which we sometimes use.

paulmedynski · 2025-08-29T19:19:11Z

eng/pipelines/common/templates/steps/build-and-run-tests-netcore-step.yml

    solution: build.proj
    msbuildArchitecture: x64
-    msbuildArguments: '-p:Configuration=${{parameters.configuration }} -t:BuildAKVNetCore -p:ReferenceType=${{parameters.referenceType }} -p:TestMicrosoftDataSqlClientVersion=${{parameters.NugetPackageVersion }}'
+    msbuildArguments: '-p:Configuration=${{parameters.buildConfiguration }} -t:BuildAKVNetCore -p:ReferenceType=${{parameters.referenceType }} -p:TestMicrosoftDataSqlClientVersion=${{parameters.NugetPackageVersion }}'


And sometimes for MSBuild@1 we manually set the build configuration. Shrug!

paulmedynski · 2025-08-29T19:21:00Z

eng/pipelines/libraries/common-variables.yml

    # AuthAKVName
    # AuthSignCertName

-  - name: Configuration


This variable is no longer needed.

paulmedynski · 2025-08-29T19:21:43Z

eng/pipelines/stages/stress-tests-ci-stage.yml

    displayName: Build Configuration
    type: string
-    default: Release
+    default: Debug


When we enable the stress tests, we can decide which build configuration to use. For now, they default to Debug like everything else.

paulmedynski · 2025-08-29T19:26:19Z

eng/pipelines/dotnet-sqlclient-ci-release-pipeline.yml

Unfortunately, ADO won't let me create a new pipeline pointing to this file beause it doesn't exist on main yet. I have opened a bug, but I doubt it will be addressed. So, we will have to get this merged to GitHub main before I can setup a pipeline to test it.

paulmedynski · 2025-08-29T19:31:54Z

eng/pipelines/dotnet-sqlclient-ci-release-pipeline.yml

+#
+#   - Commits to GitHub main
+#   - Commits to ADO internal/main
+#   - Weekdays at 01:00 UTC on GitHub main


Note that I changed the day and time of these schedules. The old values were Tue-Sat 00:00, and Fri 04:00 respectively.

paulmedynski · 2025-08-29T19:33:02Z

src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/Config.cs

        public bool IsJsonSupported = false;
        public static Config Load(string configPath = @"config.json")
        {
+            configPath = Environment.GetEnvironmentVariable("MDS_TEST_CONFIG") ?? configPath;


This lets me have a single config file that I can use for any workspace, rather than having to edit config.json on every Git branch/worktree.

eng/pipelines/common/templates/steps/publish-test-results-step.yml

- Changed default build configuration from Release to Debug for CI-SqlClient[-Package] pipelines. - Plumbed 'builConfiguration' template parameter down through all of the pipeline YML files. - Removed the pipeline variable $(Configuration) in favour of buildConfiguration parameter. - Added environment variable override for unit test config file. - Removed triggers and schedules from existing CI pipelines. - Added new pipeline for triggers and schedules that uses Release configuration. - Commented out some failing assertions. - Inhibited the ConnectionPoolTestDebug tests from running since they are flaky. - Commented out more flaky tests. - Doubling the test job timeout for Debug builds. - Fixed a test that incorrectly consumes an exception when it shouldn't. - Added diagnostics to Enclave tests that are failing to help diagnose the issues. - Commented out a failing Debug.Assert(). - Expanded error detection in one test. - Commented out another failing Debug.Assert(). - Fixed exception type check to look for the actual exceptions being thrown. - Updated xUnit config to show complete strings and collections when different. - Updated CI pipelines to be triggered by pushes to main and a schedule only. - Added buildConfiguration all the way down through the test stages/jobs/steps. - Added pipeline URLs to the new PR pipeline YAML files. - Commented out failing test chunk.

Copilot

Pull Request Overview

This PR refactors the pipeline configuration system to standardize parameter naming and adds temporary workarounds for failing assertions. The main changes include:

Renaming configuration/Configuration to buildConfiguration across all pipeline files for consistency
Renaming testsTimeout to testJobTimeout and changing its type from string to number
Adding parameter value constraints (values: [Debug, Release]) to enforce valid build configurations
Commenting out failing Debug assertions and tests with TODO(GH-3604) markers
Adding environment variable support for test configuration paths
Minor formatting and documentation improvements to pipeline files

Reviewed Changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
xunit.runner.json	Adds xUnit configuration options to control output length and live output display
Config.cs	Adds support for MDS_TEST_CONFIG environment variable to override config path
DataStreamTest.cs	Comments out 4 failing IOException assertions with TODO markers
ConnectionPoolTest.Debug.cs	Changes class from static to abstract to prevent xUnit from running tests
ApiShould.cs	Fixes exception handling, adds else clause, comments out failing enclave test section
VirtualSecureModeEnclaveProvider.cs	Comments out failing Debug.Assert with enhanced error message
TdsParser.cs	Comments out 2 SNI error message validation assertions
SqlCommand.Reader.cs	Comments out null check assertion for cached async reader
AzureAttestationBasedEnclaveProvider.cs	Comments out offset validation assertion
AlwaysEncryptedHelperClasses.cs	Comments out uninitialized entry validation assertion
DbConnectionInternal.cs	Removes debug-only activation/deactivation counter tracking code
Pipeline YAML files (15 files)	Standardizes parameter naming from configuration/Configuration to buildConfiguration, adds value constraints, renames testsTimeout to testJobTimeout, improves documentation and formatting

Copilot · 2025-11-07T19:41:38Z

src/Microsoft.Data.SqlClient/tests/ManualTests/AlwaysEncrypted/ApiShould.cs

+                    if (ex is SqlException or InvalidOperationException)
+                    {
+                        Assert.Equal("Operation cancelled by user.", ex.Message);
+                    }


The assertion now accepts both SqlException and InvalidOperationException but asserts the same message for both. InvalidOperationException may have a different message than 'Operation cancelled by user.' Consider validating the message per exception type or documenting why both exceptions share this message.

Copilot · 2025-11-07T19:41:38Z

src/Microsoft.Data.SqlClient/tests/ManualTests/AlwaysEncrypted/ApiShould.cs

+                            else
+                            {
+                                throw;
+                            }


The else block with a bare throw statement is redundant. When the if condition is false and there are no other statements in the else block except throw, the exception will propagate naturally without the else clause. Consider removing the else block for cleaner code.

Copilot · 2025-11-07T19:41:38Z

eng/pipelines/common/templates/steps/generate-nuget-package-step.yml

+  - name: buildConfiguration
    type: string
-    default: '$(Configuration)'
+    default: Debug


The default value 'Debug' is inconsistent with other pipeline files where 'Release' is typically the default for official builds (see dotnet-sqlclient-ci-project-reference-pipeline.yml line 96). Consider using 'Release' as the default to align with production build practices.

Suggested change

default: Debug

default: Release

Copilot · 2025-11-07T19:41:39Z

eng/pipelines/dotnet-sqlclient-ci-project-reference-pipeline.yml

+#
+#   - Only the ADO repo has an 'internal/main' branch.
+#
+# Changes are batched together to ensure that the pipline never runs


Typo in word 'pipeline' - misspelled as 'pipline'.

Suggested change

# Changes are batched together to ensure that the pipline never runs

# Changes are batched together to ensure that the pipeline never runs

Copilot · 2025-11-07T19:41:39Z

eng/pipelines/dotnet-sqlclient-ci-package-reference-pipeline.yml

+#
+#   - Only the ADO repo has an 'internal/main' branch.
+#
+# Changes are batched together to ensure that the pipline never runs


Typo in word 'pipeline' - misspelled as 'pipline'.

Suggested change

# Changes are batched together to ensure that the pipline never runs

# Changes are batched together to ensure that the pipeline never runs

paulmedynski added this to the 7.0-preview1 milestone Aug 29, 2025

paulmedynski added the Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems. label Aug 29, 2025

paulmedynski commented Aug 29, 2025

View reviewed changes

edwardneal reviewed Aug 29, 2025

View reviewed changes

eng/pipelines/common/templates/steps/publish-test-results-step.yml Show resolved Hide resolved

paulmedynski force-pushed the dev/paul/ci-debug branch 2 times, most recently from 4494e6f to a8fc182 Compare September 5, 2025 12:34

paulmedynski modified the milestones: 7.0-preview1, 7.0-preview2 Sep 5, 2025

paulmedynski force-pushed the dev/paul/ci-debug branch 2 times, most recently from 112949c to 8f36c71 Compare September 15, 2025 10:47

paulmedynski force-pushed the dev/paul/ci-debug branch 2 times, most recently from 4b25cc6 to 1ebb2d2 Compare September 30, 2025 13:40

paulmedynski force-pushed the dev/paul/ci-debug branch from 1ebb2d2 to 7b95619 Compare October 2, 2025 10:56

paulmedynski mentioned this pull request Oct 4, 2025

Add PR Debug pipelines #3659

Merged

paulmedynski force-pushed the dev/paul/ci-debug branch 3 times, most recently from b503a65 to 3bb1701 Compare October 6, 2025 13:25

paulmedynski removed this from the 7.0-preview2 milestone Oct 8, 2025

paulmedynski force-pushed the dev/paul/ci-debug branch from 403d0d9 to 9da40ad Compare October 16, 2025 20:15

Copilot AI review requested due to automatic review settings November 7, 2025 19:36

paulmedynski force-pushed the dev/paul/ci-debug branch from 9da40ad to 23c261c Compare November 7, 2025 19:36

Copilot AI reviewed Nov 7, 2025

View reviewed changes

	# Changes are batched together to ensure that the pipline never runs
	# Changes are batched together to ensure that the pipeline never runs

[DRAFT] Add Debug CI runs #3589

Are you sure you want to change the base?

[DRAFT] Add Debug CI runs #3589

Uh oh!

Conversation

paulmedynski commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Uh oh!

paulmedynski left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

paulmedynski commented Aug 29, 2025 •

edited

Loading