Allow Code Quality only analysis #3064

mbg · 2025-08-28T16:19:02Z

This is a first stab at enabling Code Quality analysis to be able to run on its own, while also maintaining the ability for it to run in addition to a Code Scanning analysis.

Fundamentally, this means that there are two separate ways in which Code Quality analysis can run:

The existing "add-on" implementation, where the database is initialised for Code Scanning, and we inject the code-quality suite into the run-queries call. The Code Quality SARIF is then generated by an extra call to interpret-results.
The new "standalone" implementation, where the database is initialised for Code Quality by disabling default queries and configuring the database with code-quality queries.

43d9bc8 and 5d95d46 implement the bulk of the changes needed for the "add-on" implementation that result from needing to check whether Code Quality is enabled in addition to or instead of Code Scanning.

1746aed implements the bulk of the changes needed for the "standalone" implementation. Concretely, this checks whether Code Quality is enabled on its own, and then mutates the configuration as necessary.

Probably best reviewed commit-by-commit.

Risk assessment

For internal use only. Please select the risk level of this change:

High risk: Changes are not fully under feature flags, have limited visibility and/or cannot be tested outside of production.

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Consider adding a changelog entry for this change.
Confirm the readme and docs have been updated if necessary.

src/analyze.ts

Copilot

Pull Request Overview

This PR adds support for Code Quality only analysis by refactoring how analysis configurations are handled and enabling standalone Code Quality database initialization. Previously, Code Quality could only run as an "add-on" to Code Scanning analysis.

Key changes:

Consolidates analysis configuration constants into a new analyses.ts module
Enables Code Quality to run independently without Code Scanning
Deprecates the quality-queries input in favor of the analysis-kinds input
Adds helper utilities for working with different analysis configurations

Reviewed Changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/analyses.ts	New module defining analysis configurations and constants
src/config-utils.ts	Major logic changes to support standalone Code Quality analysis initialization
src/util.ts	Added `isDefined` utility function
src/start-proxy.ts	Refactored to use centralized `isDefined` function
src/init-action.ts	Added deprecation warning for `quality-queries` input
src/analyze.ts	Updated query execution logic to handle both standalone and add-on Code Quality
src/upload-*.ts	Updated to use new analysis configuration objects
pr-checks/checks/*.yml	Updated test configurations to use `analysis-kinds` instead of `quality-queries`
init/action.yml	Marked `quality-queries` input as deprecated

src/analyses.ts

src/config-utils.ts

Copilot · 2025-09-01T15:20:20Z

src/config-utils.ts

+    config.originalUserInput["disable-default-queries"] = true;
+    config.originalUserInput.queries = queries;
+    config.originalUserInput["query-filters"] = [];


Direct mutation of config.originalUserInput is inconsistent with the pattern used elsewhere in the codebase. Consider using cloneObject or a similar approach to avoid mutating the original input, as this could cause unexpected side effects if the config object is used elsewhere.

Suggested change

config.originalUserInput["disable-default-queries"] = true;

config.originalUserInput.queries = queries;

config.originalUserInput["query-filters"] = [];

const originalUserInputClone = cloneObject(config.originalUserInput);

originalUserInputClone["disable-default-queries"] = true;

originalUserInputClone.queries = queries;

originalUserInputClone["query-filters"] = [];

config.originalUserInput = originalUserInputClone;

src/analyze.ts

src/config-utils.ts

…bled

… input

…ised with

…e documentation

henrymercer · 2025-09-01T15:57:10Z

src/config-utils.ts

+
+  // If Code Scanning analysis is disabled, then we initialise the database for Code Quality.
+  // That entails disabling the default queries and only running quality queries.
+  if (!isCodeScanningEnabled(config)) {


I like that we're looking at things that won't work proactively, but would consider changing the condition here. It works since the analysis kinds are only code scanning and code quality, but I think in general the logic we want is more like config.analysisKinds.length === 1 && isCodeQualityEnabled(config).

henrymercer · 2025-09-01T16:00:38Z

src/config-utils.ts

+      logger.warning(
+        "Query customizations will be ignored, because only `code-quality` analysis is enabled.",
+      );


Should we actually error in these cases? It's easy to miss Actions warnings, and users could think they're analysing something that doesn't actually work. That also avoids us the slight confusion of then having to override config.originalUserInput.

I wasn't sure about this, but decided to go with a warning because in Default Setup we provide inputs to queries and config. We can probably change that there for when Code Quality is enabled, but Code Scanning isn't, but I didn't want to make this an error just yet in case it causes us problems during the transition there.

I appreciate wanting to make the transition smooth, but I have some concerns about modifying config.originalUserInput, particularly since we use this to generate telemetry.

My understanding is today we have analyses with a quality-queries input and without an analysis-kinds input. This means we'll run code scanning, and therefore shouldn't hit this case. Then the transition can move to an explicit analysis-kinds input with the config and other customization inputs removed.

Is there anything I'm missing here?

I think there are two separate points here:

Whether we can swap over from using quality-queries to analysis-kinds and remove the queries input in CQ-only mode in one go. (Probably "yes" and this being a warning is just overly defensive.)

That modifying config.originalUserInput is an issue for telemetry.

Regarding 2., I agree that this isn't a very nice approach. I'll have a look at refactoring this tomorrow so that originalUserInput can remain as-is and we set up the configuration for a CQ-only database differently.

henrymercer · 2025-09-01T16:04:46Z

src/upload-lib.ts

@@ -943,7 +913,7 @@ function handleProcessingResultForUnsuccessfulExecution(

 export function validateUniqueCategory(
  sarif: SarifFile,
-  sentinelPrefix: string = CodeScanningTarget.sentinelPrefix,
+  sentinelPrefix: string = analyses.CodeScanning.sentinelPrefix,


Do we need a default value here? Removing it might decrease the potential for errors.

henrymercer · 2025-09-01T16:30:22Z

src/config-utils.ts

+ *
+ * @returns Returns `CodeScanning` if `AnalysisKind.CodeScanning` is enabled; otherwise `CodeQuality`.
+ */
+export function getDbAnalysisConfig(config: Config): AnalysisConfig {


This seems like a detail of how saving SARIF results currently works, in particular the fact that when both modes are on, we currently produce a code scanning file, then filter that to produce a code quality file. I don't really think it's a property of the database, since we'll ask the database to analyse both code scanning and code quality results in that situation.

I'd consider removing this in favour of some more explicit logic concerning the SARIF extension in analyze, unless there are more situations where this notion of "primary consumer" matters.

This seems like a detail of how saving SARIF results currently works, in particular the fact that when both modes are on, we currently produce a code scanning file, then filter that to produce a code quality file. I don't really think it's a property of the database, since we'll ask the database to analyse both code scanning and code quality results in that situation.

I don't think this is true, since in CQ-only mode the database is specifically initialised with code-quality queries. What getDbAnalysisConfig tells us is whether the database is initialised with a Code Scanning configuration (in CS only or CS+CQ mode) or a Code Quality configuration (in CQ only mode).

It's true though that this is currently only relevant when we are generating the "primary" SARIF file.

I don't think this is true, since in CQ-only mode the database is specifically initialised with code-quality queries. What getDbAnalysisConfig tells us is whether the database is initialised with a Code Scanning configuration (in CS only or CS+CQ mode) or a Code Quality configuration (in CQ only mode).

I'm unsure how much meaning this has in practice. For instance, if I really wanted to, I could create a code scanning configuration that runs the code quality queries only by disabling the default queries and running a custom query pack. I'd expect the database to be identical to a database with analysis-kinds: code-quality. The only difference I can see is the name of the SARIF file and the upload API.

github-advanced-security bot found potential problems Aug 28, 2025

View reviewed changes

src/analyze.ts Fixed Show fixed Hide fixed

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch 4 times, most recently from e28ecb0 to 603a572 Compare September 1, 2025 15:17

mbg marked this pull request as ready for review September 1, 2025 15:17

mbg requested a review from a team as a code owner September 1, 2025 15:17

mbg requested review from Copilot and henrymercer September 1, 2025 15:17

Copilot AI reviewed Sep 1, 2025

View reviewed changes

mbg added 20 commits September 1, 2025 16:46

Mark quality-queries input as deprecated

563ab34

Update checks to use analysis-kinds instead of quality-queries

40ee250

Add utility function to check if code scanning is enabled

816eb47

Move UploadTarget definitions to analyses.ts

7f85815

Rename UploadTarget to AnalysisConfig

3937ce4

Add kind property to AnalysisConfig and documentation

1899612

Only specify queries for run-queries if both analysis kinds are ena…

aaa1740

…bled

Handle different permutations of analysis kinds in analyze action

5807209

Use hard-coded list of quality queries, rather than quality-queries…

4eac995

… input

Add note for getPerQueryAlertCounts

9f3fef2

Stop propagating quality-queries input

34804ae

Add sarifExtension field to AnalysisConfig

9515500

Add and use helpers for determining analysis config the DB is initial…

29bb585

…ised with

Add test with analysisKindsInput: code-quality

9228379

Remove augmentationProperties variable from initConfig and improv…

6a8665f

…e documentation

Move isDefined from start-proxy to util

d0e7a6f

Store computed Code Scanning config in variable

553e678

Override query configuration for Code Quality only analyses

690c548

Fix addSarifExtension and add tests

0e8cce2

Check if Code Scanning is enabled before uploading Code Scanning SARIF

a99145b

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch from 5b9c46f to a99145b Compare September 1, 2025 15:47

henrymercer reviewed Sep 1, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow Code Quality only analysis #3064

Allow Code Quality only analysis #3064

mbg commented Aug 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Sep 1, 2025

Uh oh!

Uh oh!

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

mbg Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

mbg Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

mbg Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

Uh oh!

-    config.originalUserInput["disable-default-queries"] = true;
-    config.originalUserInput.queries = queries;
-    config.originalUserInput["query-filters"] = [];
+    const originalUserInputClone = cloneObject(config.originalUserInput);
+    originalUserInputClone["disable-default-queries"] = true;
+    originalUserInputClone.queries = queries;
+    originalUserInputClone["query-filters"] = [];
+    config.originalUserInput = originalUserInputClone;

Allow Code Quality only analysis #3064

Are you sure you want to change the base?

Allow Code Quality only analysis #3064

Conversation

mbg commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Risk assessment

Merge / deployment checklist

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mbg commented Aug 28, 2025 •

edited

Loading