ConfigMap as a ValueSource in Param in TaskRuns and PipelineRuns

[mostafaCamel]

This proposal is to be able to reference ConfigMap as a value source for a Param in TaskRun or PipelineRun . This is to support Kubernetes native options (ConfigMap) as value source along with direct value passed to TaskRun and PipelineRun.

Note: this proposal is basically picking up upon the closed (unmerged) proposal.

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign vincent-pli after the PR has been reviewed.
You can assign the PR to them by writing /assign @vincent-pli in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• teps/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mostafaCamel]

For whatever it's worth, I've made the code change in my fork (I have yet to add documentation and tests)
mostafaCamel/pipeline@0a3f516

[waveywaves]

added the TEP as an agenda item to be discussed in the upcoming meeting

[mostafaCamel]

Thanks Vibhav! I am new to the project so let me know if I need to/can attend

[tekton-robot]

@mostafaCamel: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[waveywaves]

@mostafaCamel you are welcome to join the working group calls ! Check the github.com/tektoncd/community repo for more info 🙂

[waveywaves]

should we add support for StepActions as well ?

[vdemeester]

/assign

[waveywaves]

/assign

[mostafaCamel]

While working on the draft tektoncd/pipeline#8642 , I realized the new feature is not working for inline parameters. This is due to the following function createParamSpecFromParam which expects a Param to have a Type during validation (which is not the case for parameters with value sources).

What I could do is to change the behavior of this function so that it defaults to Type string when populating a ParamSpec from a param with ValueSource and to tell the users that params with value sources will always be treated as strings.

Another option is not to allow inline parameters if value source is used (in order to dynamically determine the type of Param in the first reconcile

https://github.com/tektoncd/pipeline/blob/781c2800649e24e1c098daaad95eba4016b86def/pkg/apis/pipeline/v1/taskrun_validation.go#L200

[mostafaCamel]

Looking again at the code of the function createParamSpecFromParam , it seems that these freshly-created ParamSpecs are only used for validation and are not appended to the ParamSpecs array of TaskSpec/PipelineSpec . Maybe what we can do is to:

• skip the creation of ParamSpecs out of Param if Param still has non-resolved valuesource.
  • so skip the validation of this ParamSpec
• After the first reconcile run, the Param value will be resolved
• In subsequent validations, the ParamSpec will be created and validated.

This will not affect the CRD content as these ParamSpecs are never appended to PipelineSpec/TaskSpec anyways. However, we're getting exposed to getting validation errors on these "fresh" ParamSpecs after the TaskRun/PipelineRun creation. These are the validations done

				errs = errs.Also(ValidateParameterTypes(ctx, paramSpec))
				errs = errs.Also(ValidateParameterVariables(ctx, pt.TaskSpec.Steps, paramSpec))
				errs = errs.Also(ValidateUsageOfDeclaredParameters(ctx, pt.TaskSpec.Steps, paramSpec))
[...]
		errs = errs.Also(ValidatePipelineParameterVariables(ctx, ps.PipelineSpec.Tasks, paramSpec))
		errs = errs.Also(validatePipelineTaskParameterUsage(ps.PipelineSpec.Tasks, paramSpec))

https://github.com/tektoncd/pipeline/blob/23b3deefcea58e469e5775c4b39f6657dc80b538/pkg/apis/pipeline/v1/pipelinerun_validation.go#L218

Another thing worth noting is how this paramspec is being created in createParamSpecFromParam

        value := p.Value
	pSpec := ParamSpec{
		Name:    p.Name,
		Default: &value,
		Type:    p.Value.Type,
	}

paramSpec.Default is a pointer while param.Value is not a pointer. So paramSpec.Default will be a non-null field with an empty value instead of null/empty field

[mostafaCamel]

During the working group meeting today, it was suggested we can support inline parameters by using the Type field. I checked and this field is only in ParamSpec and ParamValue , not in Param
https://github.com/tektoncd/pipeline/blob/990917dffc997071df470be66aed30a10338a5e0/pkg/apis/pipeline/v1/param_types.go#L191-L196

What I can do is to add an optional Type field inside the ValueSource struct. Then:

• during createParamSpecFromParam in validation (for inline parameters) , the ParamSpec type will be populated based on the ValueSource type (and will default to string if ValueSource.Type is not defined by the user).
• during the value fetching from the configmap and the creation of a new ParamValue instance, the Type in this ParamValue will be validated against (i.e. fail if different than) the Type (if defined) in ValueSource