Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions config/300-crds/300-pipeline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ spec:
x-kubernetes-preserve-unknown-fields: true
timeout:
description: |-
Time after which the TaskRun times out. Defaults to 1 hour.
Duration after which the TaskRun times out. Defaults to 1 hour.
Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
type: string
when:
Expand Down Expand Up @@ -794,7 +794,7 @@ spec:
x-kubernetes-preserve-unknown-fields: true
timeout:
description: |-
Time after which the TaskRun times out. Defaults to 1 hour.
Duration after which the TaskRun times out. Defaults to 1 hour.
Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
type: string
when:
Expand Down Expand Up @@ -1126,7 +1126,7 @@ spec:
x-kubernetes-preserve-unknown-fields: true
timeout:
description: |-
Time after which the TaskRun times out. Defaults to 1 hour.
Duration after which the TaskRun times out. Defaults to 1 hour.
Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
type: string
when:
Expand Down Expand Up @@ -1465,7 +1465,7 @@ spec:
x-kubernetes-preserve-unknown-fields: true
timeout:
description: |-
Time after which the TaskRun times out. Defaults to 1 hour.
Duration after which the TaskRun times out. Defaults to 1 hour.
Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
type: string
when:
Expand Down
12 changes: 12 additions & 0 deletions config/300-crds/300-pipelinerun.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1419,6 +1419,11 @@ spec:
x-kubernetes-preserve-unknown-fields: true
taskServiceAccountName:
type: string
timeout:
description: |-
Duration after which the TaskRun times out.
Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
type: string
x-kubernetes-list-type: atomic
timeout:
description: |-
Expand Down Expand Up @@ -3901,6 +3906,13 @@ spec:
description: The name of the Step to override.
type: string
x-kubernetes-list-type: atomic
timeout:
description: |-
Duration after which the TaskRun times out. Overrides the timeout specified
on the Task's spec if specified. Takes lower precedence to PipelineRun's
`spec.timeouts.tasks`
Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
type: string
x-kubernetes-list-type: atomic
taskRunTemplate:
description: TaskRunTemplate represent template of taskrun
Expand Down
15 changes: 15 additions & 0 deletions docs/pipeline-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -3322,6 +3322,21 @@ Kubernetes core/v1.ResourceRequirements
<p>Compute resources to use for this TaskRun</p>
</td>
</tr>
<tr>
<td>
<code>timeout</code><br/>
<em>
<a href="https://godoc.org/k8s.io/apimachinery/pkg/apis/meta/v1#Duration">
Kubernetes meta/v1.Duration
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>Time after which the TaskRun times out.
Refer Go&rsquo;s ParseDuration documentation for expected format: <a href="https://golang.org/pkg/time/#ParseDuration">https://golang.org/pkg/time/#ParseDuration</a></p>
</td>
</tr>
</tbody>
</table>
<h3 id="tekton.dev/v1.PipelineTaskRunTemplate">PipelineTaskRunTemplate
Expand Down
37 changes: 36 additions & 1 deletion docs/pipelineruns.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please also update the Configuring a Failure Timeout section? There are quite a few ways to set various timeout settings and it can be quite difficult to find the setting you need. It would be good to consolidate them in the future, but consolidating the documentation would be very helpful

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good idea, not sure if the reorganization of the section fits in this PR, I am creating an issue out of this instead for now so that a contributor can pick it up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, thanks for creating the followup issue as well! What you've added to that section looks good for the scope of this PR 🙏🏼

Original file line number Diff line number Diff line change
Expand Up @@ -951,6 +951,7 @@ spec:
podTemplate:
nodeSelector:
disktype: ssd
timeout: "1h30m"
```
{{% /tab %}}

Expand All @@ -968,12 +969,16 @@ spec:
taskPodTemplate:
nodeSelector:
disktype: ssd
timeout: "1h30m"
```
{{% /tab %}}
{{< /tabs >}}

If used with this `Pipeline`, `build-task` will use the task specific `PodTemplate` (where `nodeSelector` has `disktype` equal to `ssd`)
along with `securityContext` from the `pipelineRun.spec.podTemplate`.
along with `securityContext` from the `pipelineRun.spec.podTemplate`. The task will also have a specific timeout of 1 hour and 30 minutes. This overrides any existing timeout already defined by the pipelineTask as well, though the specified `pipelineRun.spec.timeouts.tasks` will still take precedence.

For more details on timeout overrides, precedence rules, validation, and practical examples, see [Overriding Individual Task Timeouts](#overriding-individual-task-timeouts) in the failure timeout section.

`PipelineTaskRunSpec` may also contain `StepSpecs` and `SidecarSpecs`; see
[Overriding `Task` `Steps` and `Sidecars`](./taskruns.md#overriding-task-steps-and-sidecars) for more information.

Expand Down Expand Up @@ -1414,6 +1419,36 @@ The global default timeout is set to 60 minutes when you first install Tekton. Y
a different global default timeout value using the `default-timeout-minutes` field in
[`config/config-defaults.yaml`](./../config/config-defaults.yaml).

#### Overriding Individual Task Timeouts

You can use `taskRunSpecs` to override individual task timeouts at runtime without modifying the Pipeline definition.

**Timeout Precedence (highest to lowest):**
1. `taskRunSpecs[].timeout` - runtime override per task
2. `pipeline.spec.tasks[].timeout` - Pipeline spec timeout
3. `timeouts.tasks` or `timeouts.pipeline` - PipelineRun constraints
4. Global default timeout

```yaml
apiVersion: tekton.dev/v1
kind: PipelineRun
spec:
timeouts:
pipeline: "10m" # 3. PipelineRun constraint
pipelineSpec:
tasks:
- name: task-a
timeout: "8m" # 2. Pipeline spec timeout
taskSpec: { ... }
- name: task-b
taskSpec: { ... } # 4. Uses global default (60m)
taskRunSpecs:
- pipelineTaskName: task-a
timeout: "5m" # 1. Highest priority - overrides 8m Pipeline timeout
```

**Note:** `taskRunSpecs` timeouts cannot exceed pipeline-level constraints and will fail validation if they do.

Example timeouts usages are as follows:

Combination 1: Set the timeout for the entire `pipeline` and reserve a portion of it for `tasks`.
Expand Down
2 changes: 2 additions & 0 deletions docs/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -1206,6 +1206,8 @@ format. For example, valid values are `1h30m`, `1h`, `1m`, and `60s`.
For example, if the `PipelineRun` sets `timeouts.pipeline = 1h` and the `Pipeline` sets `tasks[0].timeout = 3h`, the task will still timeout after `1h`.
See [`PipelineRun - Configuring a failure timeout`](pipelineruns.md#configuring-a-failure-timeout) for details.

**Note:** Task timeouts specified in the Pipeline can be overridden at runtime using the [`taskRunSpecs`](pipelineruns.md#overriding-individual-task-timeouts) field in the PipelineRun. This provides flexibility to adjust timeouts for specific execution contexts without modifying the Pipeline definition.

In the example below, the `build-the-image` `Task` is configured to time out after 90 seconds:

```yaml
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: sleep-task
spec:
steps:
- name: sleep-long
image: alpine:latest
script: |
#!/bin/sh
sleep 10
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: verify-task
spec:
steps:
- name: verify-completion
image: alpine:latest
script: |
#!/bin/sh
echo "- Pipeline task timeout: 60s (defined in pipeline spec)"
echo "- TaskRunSpecs override: 20s (defined in pipelinerun spec)"
echo "- Actual task duration: 10s (within 20s override timeout)"
---
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: taskrun-timeout-override
spec:
tasks:
- name: long-running-task
taskRef:
name: sleep-task
kind: Task
timeout: "60s" # pipeline task timeout (will be overridden to 20s)
- name: verify-timeout
taskRef:
name: verify-task
kind: Task
runAfter: ["long-running-task"]
---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: taskrun-timeout-override
spec:
pipelineRef:
name: taskrun-timeout-override
taskRunSpecs:
- pipelineTaskName: long-running-task
timeout: "20s" # 20s timeout (can't actually test with a timeout lesser than 10s else the task fails)
10 changes: 8 additions & 2 deletions pkg/apis/pipeline/v1/openapi_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1/pipeline_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ type PipelineTask struct {
// +listType=atomic
Workspaces []WorkspacePipelineTaskBinding `json:"workspaces,omitempty"`

// Time after which the TaskRun times out. Defaults to 1 hour.
// Duration after which the TaskRun times out. Defaults to 1 hour.
// Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
// +optional
Timeout *metav1.Duration `json:"timeout,omitempty"`
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1/pipeline_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1415,7 +1415,7 @@ func TestPipelineSpec_Validate_Failure(t *testing.T) {
if err == nil {
t.Errorf("PipelineSpec.Validate() did not return error for invalid pipelineSpec")
}
if d := cmp.Diff(tt.expectedError.Error(), err.Error(), cmpopts.IgnoreUnexported(apis.FieldError{})); d != "" {
if d := cmp.Diff(tt.expectedError.Error(), err.Error()); d != "" {
t.Errorf("PipelineSpec.Validate() errors diff %s", diff.PrintWantGot(d))
}
})
Expand Down
8 changes: 8 additions & 0 deletions pkg/apis/pipeline/v1/pipelinerun_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,13 @@ type PipelineTaskRunSpec struct {

// Compute resources to use for this TaskRun
ComputeResources *corev1.ResourceRequirements `json:"computeResources,omitempty"`

// Duration after which the TaskRun times out. Overrides the timeout specified
// on the Task's spec if specified. Takes lower precedence to PipelineRun's
// `spec.timeouts.tasks`
// Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
// +optional
Timeout *metav1.Duration `json:"timeout,omitempty"`
}

// GetTaskRunSpec returns the task specific spec for a given
Expand All @@ -678,6 +685,7 @@ func (pr *PipelineRun) GetTaskRunSpec(pipelineTaskName string) PipelineTaskRunSp
s.SidecarSpecs = task.SidecarSpecs
s.Metadata = task.Metadata
s.ComputeResources = task.ComputeResources
s.Timeout = task.Timeout
}
}
return s
Expand Down
76 changes: 72 additions & 4 deletions pkg/apis/pipeline/v1/pipelinerun_validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,10 @@ func (ps *PipelineRunSpec) Validate(ctx context.Context) (errs *apis.FieldError)
}
}

// Validate individual TaskRunSpecs with timeout context
for idx, trs := range ps.TaskRunSpecs {
errs = errs.Also(validateTaskRunSpec(ctx, trs, ps.Timeouts).ViaIndex(idx).ViaField("taskRunSpecs"))
}
errs = errs.Also(validateSpecStatus(ps.Status))

if ps.Workspaces != nil {
Expand All @@ -117,9 +121,6 @@ func (ps *PipelineRunSpec) Validate(ctx context.Context) (errs *apis.FieldError)
wsNames[ws.Name] = idx
}
}
for idx, trs := range ps.TaskRunSpecs {
errs = errs.Also(validateTaskRunSpec(ctx, trs).ViaIndex(idx).ViaField("taskRunSpecs"))
}

if ps.TaskRunTemplate.PodTemplate != nil {
errs = errs.Also(validatePodTemplateEnv(ctx, *ps.TaskRunTemplate.PodTemplate).ViaField("taskRunTemplate"))
Expand Down Expand Up @@ -313,7 +314,7 @@ func (ps *PipelineRunSpec) validatePipelineTimeout(timeout time.Duration, errorM
return errs
}

func validateTaskRunSpec(ctx context.Context, trs PipelineTaskRunSpec) (errs *apis.FieldError) {
func validateTaskRunSpec(ctx context.Context, trs PipelineTaskRunSpec, pipelineTimeouts *TimeoutFields) (errs *apis.FieldError) {
if trs.StepSpecs != nil {
errs = errs.Also(config.ValidateEnabledAPIFields(ctx, "stepSpecs", config.BetaAPIFields).ViaField("stepSpecs"))
errs = errs.Also(validateStepSpecs(trs.StepSpecs).ViaField("stepSpecs"))
Expand All @@ -329,5 +330,72 @@ func validateTaskRunSpec(ctx context.Context, trs PipelineTaskRunSpec) (errs *ap
if trs.PodTemplate != nil {
errs = errs.Also(validatePodTemplateEnv(ctx, *trs.PodTemplate))
}

errs = errs.Also(validateTaskRunSpecTimeout(ctx, trs.Timeout, pipelineTimeouts))

return errs
}

// validateTaskRunSpecTimeout validates a TaskRunSpec's timeout against pipeline timeouts.
// This function works in isolation and doesn't rely on previous validation steps.
func validateTaskRunSpecTimeout(ctx context.Context, timeout *metav1.Duration, pipelineTimeouts *TimeoutFields) *apis.FieldError {
if timeout == nil {
return nil
}

cfg := config.FromContextOrDefaults(ctx)

// Validate basic timeout (negative values)
if _, err := validateTimeout(timeout, cfg.Defaults.DefaultTimeoutMinutes); err != nil {
return err
}

// Find applicable timeout limit: Tasks -> Pipeline -> Default (60min)
var maxTimeout *metav1.Duration
var timeoutSource string

switch {
case pipelineTimeouts != nil && pipelineTimeouts.Tasks != nil:
if validatedTimeout, err := validateTimeout(pipelineTimeouts.Tasks, cfg.Defaults.DefaultTimeoutMinutes); err != nil {
// Return error if Tasks timeout is invalid (prevents silent failures)
return err
} else {
maxTimeout = validatedTimeout
timeoutSource = "pipeline tasks duration"
}
case pipelineTimeouts != nil && pipelineTimeouts.Pipeline != nil:
if validatedTimeout, err := validateTimeout(pipelineTimeouts.Pipeline, cfg.Defaults.DefaultTimeoutMinutes); err != nil {
// Return error if Pipeline timeout is invalid (prevents silent failures)
return err
} else {
maxTimeout = validatedTimeout
timeoutSource = "pipeline duration"
}
default:
maxTimeout = &metav1.Duration{Duration: time.Duration(cfg.Defaults.DefaultTimeoutMinutes) * time.Minute}
timeoutSource = "default pipeline duration"
}

// always check against max timeout
if maxTimeout != nil && maxTimeout.Duration != config.NoTimeoutDuration {
if taskRunTimeout, _ := validateTimeout(timeout, cfg.Defaults.DefaultTimeoutMinutes); taskRunTimeout.Duration > maxTimeout.Duration { // We know this won't error from above
return apis.ErrInvalidValue(
fmt.Sprintf("%s should be <= %s %s", taskRunTimeout.Duration, timeoutSource, maxTimeout.Duration),
"timeout")
}
}

return nil
}

// validateTimeout validates a timeout field and returns the validated timeout with defaults applied.
// If timeout is nil, returns default timeout. If timeout is negative, returns an error.
func validateTimeout(timeout *metav1.Duration, defaultTimeoutMinutes int) (*metav1.Duration, *apis.FieldError) {
if timeout == nil {
return &metav1.Duration{Duration: time.Duration(defaultTimeoutMinutes) * time.Minute}, nil
}
if timeout.Duration < 0 {
return nil, apis.ErrInvalidValue(timeout.Duration.String()+" should be >= 0", "timeout")
}
return timeout, nil
}
Loading
Loading