Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions api/v1alpha1/worker_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,82 @@ type TemporalWorkerDeploymentList struct {
Items []TemporalWorkerDeployment `json:"items"`
}

// TemporalWorkerDeploymentPatchSpec defines version-specific overrides for a TemporalWorkerDeployment
type TemporalWorkerDeploymentPatchSpec struct {
// TemporalWorkerDeploymentName is the name of the TemporalWorkerDeployment this patch applies to.
// The patch must be in the same namespace as the target deployment.
TemporalWorkerDeploymentName string `json:"temporalWorkerDeploymentName"`

// VersionID specifies which version this patch applies to
VersionID string `json:"versionID"`

// Replicas overrides the number of desired pods for this specific version.
// If not specified, the version will use the replicas from the main TemporalWorkerDeployment spec.
// +optional
Replicas *int32 `json:"replicas,omitempty"`

// SunsetStrategy overrides how to manage sunsetting this specific version.
// If not specified, the version will use the sunset strategy from the main TemporalWorkerDeployment spec.
// +optional
SunsetStrategy *SunsetStrategy `json:"sunsetStrategy,omitempty"`
}

// PatchStatus indicates the current state of the patch
// +enum
type PatchStatus string

const (
// PatchStatusActive indicates the patch is currently applied to an existing version
PatchStatusActive PatchStatus = "Active"

// PatchStatusOrphaned indicates the referenced version no longer exists
PatchStatusOrphaned PatchStatus = "Orphaned"

// PatchStatusInvalid indicates the patch references a TemporalWorkerDeployment that doesn't exist
PatchStatusInvalid PatchStatus = "Invalid"
)

// TemporalWorkerDeploymentPatchStatus defines the observed state of TemporalWorkerDeploymentPatch
type TemporalWorkerDeploymentPatchStatus struct {
// Status indicates the current state of this patch
Status PatchStatus `json:"status"`

// AppliedAt indicates when this patch was last successfully applied
// +optional
AppliedAt *metav1.Time `json:"appliedAt,omitempty"`

// ObservedGeneration reflects the generation of the most recently observed patch spec
// +optional
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
}

//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
// +kubebuilder:resource:shortName=twdpatch;twd-patch;temporalworkerpatch
//+kubebuilder:printcolumn:name="Target",type="string",JSONPath=".spec.temporalWorkerDeploymentName",description="Target TemporalWorkerDeployment"
//+kubebuilder:printcolumn:name="Version",type="string",JSONPath=".spec.versionID",description="Target Version ID"
//+kubebuilder:printcolumn:name="Status",type="string",JSONPath=".status.status",description="Patch Status"
//+kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"

// TemporalWorkerDeploymentPatch is the Schema for the temporalworkerdeploymentpatches API
type TemporalWorkerDeploymentPatch struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec TemporalWorkerDeploymentPatchSpec `json:"spec,omitempty"`
Status TemporalWorkerDeploymentPatchStatus `json:"status,omitempty"`
}

//+kubebuilder:object:root=true

// TemporalWorkerDeploymentPatchList contains a list of TemporalWorkerDeploymentPatch
type TemporalWorkerDeploymentPatchList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []TemporalWorkerDeploymentPatch `json:"items"`
}

func init() {
SchemeBuilder.Register(&TemporalWorkerDeployment{}, &TemporalWorkerDeploymentList{})
SchemeBuilder.Register(&TemporalWorkerDeploymentPatch{}, &TemporalWorkerDeploymentPatchList{})
}
103 changes: 103 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

156 changes: 156 additions & 0 deletions docs/version-patches.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Version-Specific Configuration Patches

The Temporal Worker Controller supports version-specific configuration overrides through the `TemporalWorkerDeploymentPatch` custom resource. This allows fine-grained control over individual worker deployment versions without modifying the main `TemporalWorkerDeployment` resource.

## Overview

Version patches enable you to:

- **Scale specific versions independently**: Adjust replica counts for individual versions based on their workload or importance
- **Customize sunset strategies per version**: Apply different cleanup policies to versions based on their lifecycle needs
- **Maintain operational flexibility**: Make per-version adjustments without affecting the overall deployment strategy

## Use Cases

### 1. Scaling Down Deprecated Versions

When a version is deprecated but still processing workflows, you might want to reduce its resource consumption:

```yaml
apiVersion: temporal.io/v1alpha1
kind: TemporalWorkerDeploymentPatch
metadata:
name: scale-down-deprecated-v1
spec:
temporalWorkerDeploymentName: my-worker-deployment
versionID: "my-worker-deployment.v1.0.0"
replicas: 1 # Reduce from default 3 to 1
sunsetStrategy:
scaledownDelay: 30m # Faster scaledown
deleteDelay: 2h # Quicker cleanup
```

### 2. Scaling Up Critical Versions

For versions handling critical workflows that require higher availability:

```yaml
apiVersion: temporal.io/v1alpha1
kind: TemporalWorkerDeploymentPatch
metadata:
name: scale-up-critical-v2
spec:
temporalWorkerDeploymentName: my-worker-deployment
versionID: "my-worker-deployment.v2.1.0"
replicas: 10 # Increase from default 3 to 10
sunsetStrategy:
scaledownDelay: 24h # Keep running longer
deleteDelay: 7d # Preserve for a week
```

## Patch Status

The controller automatically updates patch status to indicate whether patches are successfully applied:

### Status Types

- **Active**: The patch is applied to an existing version
- **Orphaned**: The referenced version no longer exists
- **Invalid**: The referenced TemporalWorkerDeployment doesn't exist

### Status Fields

```yaml
status:
status: Active
appliedAt: "2024-01-15T10:30:00Z"
observedGeneration: 1
```

## Controller Behavior

### Reconciliation

1. **Patch Discovery**: The controller watches for changes to `TemporalWorkerDeploymentPatch` resources
2. **Status Updates**: Patch statuses are updated during each reconciliation loop
3. **Continuous Application**: During every reconciliation, patches are applied to compute effective configuration for all operations
4. **Event Propagation**: Changes to patches trigger reconciliation of the target `TemporalWorkerDeployment`

### How Patches Are Applied

Patches are applied **continuously during each reconciliation loop**, affecting:

- **Scaling Operations**: Existing deployments are scaled immediately when patch replica counts differ from current state
- **Sunset Decisions**: Patch sunset strategies are used to determine when to scale down and delete deployments
- **New Deployment Creation**: When new deployments are created, they use the effective configuration including patches

The controller actively monitors the difference between current deployment state and the effective configuration (including patches) and takes action to reconcile any differences.

### Conflict Resolution

- If multiple patches target the same version and field, the last applied patch wins
- Patches are applied in alphabetical order by name for deterministic behavior
- Invalid patches are marked as such and do not affect deployment behavior

## Best Practices

### Naming Conventions

Use descriptive names that indicate the purpose and target:

```yaml
metadata:
name: scale-down-v1-deprecated
# or
name: extend-sunset-critical-v2
```

### Lifecycle Management

1. **Create patches proactively** for versions you know will need special handling
2. **Monitor patch statuses** to identify orphaned patches that can be cleaned up
3. **Use labels** to group related patches for easier management:

```yaml
metadata:
labels:
temporal.io/version-class: deprecated
temporal.io/scaling-policy: conservative
```

### Resource Management

- **Clean up orphaned patches** regularly to avoid resource accumulation
- **Co-locate patches** with their target deployments in the same namespace
- **Set appropriate RBAC** to control who can create/modify patches

## Limitations

1. **Namespace Scope**: Patches must be in the same namespace as their target `TemporalWorkerDeployment`
2. **Supported Fields**: Currently only `replicas` and `sunsetStrategy` fields can be overridden
3. **Version Scope**: Patches apply to specific version IDs, not version patterns
4. **Immediate Effect**: Patch changes take effect during the next reconciliation loop (typically within 10 seconds)

## Monitoring

### Kubectl Commands

```bash
# List all patches
kubectl get temporalworkerdeploymentpatches

# Show patch details
kubectl describe twdpatch scale-down-old-version

# Check patch status
kubectl get twdpatch -o custom-columns=NAME:.metadata.name,STATUS:.status.status,VERSION:.spec.versionID
```

### Metrics

The controller provides metrics for:
- Number of active/orphaned/invalid patches
- Patch application success/failure rates
- Time since last patch status update

This enables monitoring and alerting on patch health and effectiveness.
Loading