Add Per-feature Flag Struct for New Features

This commit adds the per-feature flag struct and documentations for
new features in TektonCD pipeline.

part of: TEP-0138

Author: JeromeJu

api_compatibility_policy.md

@@ -90,7 +90,22 @@ For more information on support windows, see the [deprecations table](./docs/dep
 
 ## Feature Gates
 
-Stability levels of feature gates are independent from CRD apiVersions. Features enabled by API fields at different levels of stability can be enabled using the flag `enable-api-fields`:
+Stability levels of feature gates are independent from CRD apiVersions. 
+
+[TEP0138](https://github.com/tektoncd/community/blob/main/teps/0138-decouple-api-and-feature-versioning.md) has introduced per-feature flags for new API-driven features and the migration plan for `enable-api-fields`. Please refer to the table below for the API-driven features validation transition:
+
+| Releases               | Global flag `enable-api-fields`    | Per-feature flag                   |
+| ---------------------- | --------------------------------   | ---------------------------------- |
+| Prior to v0.53.0       | All alpha/beta API-driven features |                                    |
+| After v0.53.0        | [Existing alpha/beta API-driven features](https://github.com/tektoncd/community/blob/02418c0d39578a6a42f9d2d30caea8060dd89385/teps/0138-decouple-api-and-feature-versioning.md#sunset-enable-api-fields-after-existing-features-stabilize) prior to v0.53.0 | New alpha/beta API-driven features introduced after v0.53.0 |
+| All [alpha/beta API-driven features in v0.53.0](https://github.com/tektoncd/community/blob/02418c0d39578a6a42f9d2d30caea8060dd89385/teps/0138-decouple-api-and-feature-versioning.md#sunset-enable-api-fields-after-existing-features-stabilize) become stable or are removed | Sunset ~~`enable-api-fields`~~ | All alpha/beta API-driven features |
+
+
+_Note that behavioural(non-API-driven) flags will retain their original usage._
+
+With per-feature flags, cluster operators are able to enable a single new feature with their dedicated feature flags. For instructions on how to add a per-feature flag, please check the [developer feature versioning guide](./docs/developers/feature-versioning.md#per-feature-flag).
+
+Note that the `enable-api-fields` feature flag will continue to validate all features that were[beta](https://github.com/tektoncd/pipeline/blob/release-v0.52.x/docs/additional-configs.md#beta-features) and [alpha](https://github.com/tektoncd/pipeline/blob/release-v0.52.x/docs/additional-configs.md#alpha-features) prior to [v0.53.0](https://github.com/tektoncd/pipeline/tree/release-v0.53.x):
 
 * `stable` - This value indicates that only fields of the highest stability level are enabled; i.e. `alpha` and `beta` fields are not enabled.
 

docs/developers/feature-versioning.md

@@ -1,15 +1,50 @@
 # Feature Versioning
 
-The stability levels of features (feature versioning) are independent of CRD API versions (API versioning).
+The stability levels of features (feature versioning) are independent of CRD [API versioning](./api-versioning.md).
 
 ## Adding feature gates for API-driven features
-API-driven features are features that are enabled via a specific field in pipeline API. They comply to the [feature gates](../../api_compatibility_policy.md#feature-gates) and the [feature graduation process](../../api_compatibility_policy.md#feature-graduation-process) specified in the [API compatibility policy](../../api_compatibility_policy.md). For example, [remote tasks](https://github.com/tektoncd/pipeline/blob/454bfd340d102f16f4f2838cf4487198537e3cfa/docs/taskruns.md#remote-tasks) is an API-driven feature.
+API-driven features are features that are accessed via a specific field in pipeline API. They comply to the [feature gates](../../api_compatibility_policy.md#feature-gates) and the [feature graduation process](../../api_compatibility_policy.md#feature-graduation-process) specified in the [API compatibility policy](../../api_compatibility_policy.md). For example, [remote tasks](https://github.com/tektoncd/pipeline/blob/454bfd340d102f16f4f2838cf4487198537e3cfa/docs/taskruns.md#remote-tasks) is an API-driven feature.
+
+## Adding feature gated API fields for API-driven features
+### Per-feature flag
+
+All new features added after [v0.53.0](https://github.com/tektoncd/pipeline/releases/tag/v0.53.0) will be enabled by their dedicated feature flags. To introduce a new per-feature flag, we will proceed as follows:
+- Add default values to the new per-feature flag for the new API-driven feature following the `PerFeatureFlag` struct in [feature_flags.go](./../../pkg/apis/config/feature_flags.go).
+- Write unit tests to verify the new feature flag and update all test cases that require the configMap setting, such as those related to provenance propagation.
+- To add integration tests:
+    - First, add the tests to `pull-tekton-pipeline-alpha-integration-test` by enabling the newly-introduced per-feature flag at [alpha test Prow environment](./../../test/e2e-tests-kind-prow-alpha.env).
+    - When the flag is promoted to `beta` stability level, change the test to use [beta Prow environment setup](./../../test/e2e-tests-kind-prow-beta.env).
+    - To add additional CI tests for combinations of feature flags, add tests for all alpha feature flags being turned on, with one alpha feature turned off at a time.
+- Add the tested new per-feature flag key value to the [the pipeline configMap](./../../config/config-feature-flags.yaml).
+- Update documentations for the new alpha feature at [alpha-stability-level](./../additional-configs.md#alpha-features).
+
+#### Example of adding a new Per-feature flag
+1. Add the default value following the Per-Feature flag struct
+```golang
+const enableExampleNewFeatureKey = 'example-new-feature'
+
+var DefaultExampleNewFeatre = PerFeatureFlag {
+        Name:      enableExampleNewFeatureKey,
+        Stability: AlphaAPIFields,
+        Enabled:   DefaultAlphaFeatureEnabled,
+}
+```
+2. Add unit tests with the newly-introduced yamls `feature-flags-example-new-feature` and `feature-flags-invalid-example-new-feature` according to the existing testing framework.
+
+3. For integration tests, add `example-new-feature: true` to [alpha test Prow environment](./../../test/e2e-tests-kind-prow-alpha.env).
+
+4. Add `example-new-feature: false` to [the pipeline configMap](./../../config/config-feature-flags.yaml) with a release note.
+
+5. Update documentations for the new alpha feature at [alpha-stability-level](./../additional-configs.md#alpha-features).
+
 ### `enable-api-fields`
 
-We've introduced a feature-flag called `enable-api-fields` to the
+Prior to [v0.53.0](https://github.com/tektoncd/pipeline/tree/release-v0.53.x),  we have had the global feature flag `enable-api-fields` in
 [config-feature-flags.yaml file](../../config/config-feature-flags.yaml)
 deployed as part of our releases.
 
+_Note that the `enable-api-fields` flag will has been deprecated since [v0.53.0](https://github.com/tektoncd/pipeline/tree/release-v0.53.x) and we will transition to use [Per-feature flags](#per-feature-flag) instead._
+
 This field can be configured either to be `alpha`, `beta`, or `stable`. This field is
 documented as part of our
 [install docs](../install.md#customizing-the-pipelines-controller-behavior).

pkg/apis/config/feature_flags.go

@@ -27,12 +27,18 @@ import (
 )
 
 const (
-	// StableAPIFields is the value used for "enable-api-fields" when only stable APIs should be usable.
+	// StableAPIFields is the value used for API-driven features of stable stability level.
 	StableAPIFields = "stable"
-	// AlphaAPIFields is the value used for "enable-api-fields" when alpha APIs should be usable as well.
+	// AlphaAPIFields is the value used for API-driven features of alpha stability level.
 	AlphaAPIFields = "alpha"
-	// BetaAPIFields is the value used for "enable-api-fields" when beta APIs should be usable as well.
+	// BetaAPIFields is the value used for API-driven features of beta stability level.
 	BetaAPIFields = "beta"
+	// Features of "alpha" stability level are disabled by default
+	DefaultAlphaFeatureEnabled = false
+	// Features of "beta" stability level are disabled by default
+	DefaultBetaFeatureEnabled = false
+	// Features of "stable" stability level are enabled by default
+	DefaultStableFeatureEnabled = true
 	// FailNoMatchPolicy is the value used for "trusted-resources-verification-no-match-policy" to fail TaskRun or PipelineRun
 	// when no matching policies are found
 	FailNoMatchPolicy = "fail"
@@ -366,3 +372,18 @@ func GetVerificationNoMatchPolicy(ctx context.Context) string {
 func IsSpireEnabled(ctx context.Context) bool {
 	return FromContextOrDefaults(ctx).FeatureFlags.EnforceNonfalsifiability == EnforceNonfalsifiabilityWithSpire
 }
+
+// TODO(#7285): Patch the default values of new features that were added after
+// `enable-api-fields` was no longer used.
+type PerFeatureFlag struct {
+	// Name of the feature flag
+	Name string
+	// Stability level of the feature, one of StableAPIFields, BetaAPIFields or AlphaAPIFields
+	Stability string
+	// Enabled is whether the feature is turned on
+	Enabled bool
+	// Deprecated indicates whether the feature is deprecated
+	// +optional
+	//nolint:gocritic
+	Deprecated bool
+}