Skip to content
This repository was archived by the owner on Jun 8, 2022. It is now read-only.

Commit 34cd4ee

Browse files
zzxwillzzxwill
committed
Workload Labeling Proposal
to recommend a simple way - Workload Labeling, to easily establish connections between Workloads, Components, AppConfigurations, and pods. Signed-off-by: zzxwill <[email protected]>
1 parent 9547ba9 commit 34cd4ee

File tree

5 files changed

+295
-0
lines changed

5 files changed

+295
-0
lines changed

‎design/workload-labeling.md

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
# Workload Labeling
2+
3+
* Owner: Jianbo Sun (@wonderflow), Zheng Xi Zhou (@zzxwill)
4+
* Reviewers: Crossplane Maintainers
5+
* Status: Draft
6+
7+
## Background
8+
9+
Per [issue #193](https://github.com/crossplane/oam-kubernetes-runtime/issues/193), there are several sceneries which currently don't have elegant solutions.
10+
11+
1). In ApplicationConfiguration `bookinfo`, OAM users have to specify the pod label in `spec.labels` field of the service trait `my-service`, while OAM Kubernetes
12+
Runtime exactly knows how to set `selector` by the workload.
13+
14+
```yaml
15+
apiVersion: core.oam.dev/v1alpha2
16+
kind: ApplicationConfiguration
17+
metadata:
18+
name: bookinfo
19+
spec:
20+
components:
21+
- componentName: details
22+
traits:
23+
- trait:
24+
apiVersion: v1
25+
kind: Service
26+
metadata:
27+
name: details
28+
spec:
29+
labels:
30+
app: details
31+
```
32+
33+
2). In ApplicationConfiguration `appconfig-example`, OAM users have to copy GVK and name of the workload to `spec.scaleTargetRef`
34+
filed， from where is clearly specified in Component. Again, the runtime exactly knows how to set `scaleTargetRef`.
35+
36+
```yaml
37+
apiVersion: core.oam.dev/v1alpha2
38+
kind: ApplicationConfiguration
39+
metadata:
40+
name: appconfig-example
41+
spec:
42+
components:
43+
- componentName: php-apache
44+
traits:
45+
- trait:
46+
apiVersion: autoscaling/v2beta2
47+
kind: HorizontalPodAutoscaler
48+
metadata:
49+
name: php-apache
50+
spec:
51+
scaleTargetRef:
52+
apiVersion: apps/v1
53+
kind: Deployment
54+
name: php-apache
55+
---
56+
apiVersion: core.oam.dev/v1alpha2
57+
kind: Component
58+
metadata:
59+
name: php-apache
60+
spec:
61+
workload:
62+
apiVersion: apps/v1
63+
kind: Deployment
64+
metadata:
65+
name: php-apache
66+
```
67+
68+
3). Here is a similar situation in which users have to manually set `serviceName` and `servicePort` in trait Ingress.
69+
70+
```yaml
71+
apiVersion: core.oam.dev/v1alpha2
72+
kind: ApplicationConfiguration
73+
metadata:
74+
name: example
75+
spec:
76+
components:
77+
- componentName: web
78+
traits:
79+
- trait:
80+
apiVersion: v1
81+
kind: Service
82+
metadata:
83+
name: web
84+
spec:
85+
selector:
86+
app: web
87+
ports:
88+
- port: 80
89+
- trait:
90+
apiVersion: extensions/v1beta1
91+
kind: Ingress
92+
spec:
93+
rules:
94+
- http:
95+
paths:
96+
- path: /
97+
backend:
98+
serviceName: web
99+
servicePort: 80
100+
```
101+
102+
4). If a trait can easily retrieve the underlying pods, it will be helpful for:
103+
- ingress/service/traffic trait can easily route their traffic to pods with the information of OAM AppConfig/Component.
104+
- log/metrics trait can easily find which pods need to gather logs or metrics by OAM AppConfig.
105+
At present, users have to manually set `spec.template.metadata.label` in workload which can help trait find the targeted
106+
pods.
107+
108+
```yaml
109+
apiVersion: core.oam.dev/v1alpha2
110+
kind: Component
111+
metadata:
112+
name: example-deployment
113+
spec:
114+
workload:
115+
apiVersion: apps/v1
116+
kind: Deployment
117+
metadata:
118+
name: nginx-deployment
119+
labels:
120+
app: nginx
121+
spec:
122+
selector:
123+
matchLabels:
124+
app: nginx
125+
template:
126+
metadata:
127+
labels:
128+
component.oam.dev/name: example-deployment
129+
app: nginx
130+
spec:
131+
containers:
132+
- name: nginx
133+
image: nginx:1.17
134+
ports:
135+
- containerPort: 80
136+
```
137+
138+
In all, to achieve various goals, OAM users have to repeatedly specify information to manually establish connections
139+
between pods, workloads and traits.
140+
141+
## Goals
142+
143+
The goal is to recommend a simple way - Workload Labeling, to easily establish connections between Workloads, Components, AppConfigurations,
144+
and pods.
145+
146+
## Proposal
147+
148+
Step 1: Setting the label `definition.oam.dev/unique-pod` as the unique identifier of pods
149+
150+
We found that if our final target workload resource is pod, we can always assume the pod has unique labels:
151+
152+
- For pods created by K8s Deployment, the `pod-template-hash` label will be automatically added.
153+
154+
```yaml
155+
apiVersion: v1
156+
kind: Pod
157+
metadata:
158+
labels:
159+
pod-template-hash: 748f857667
160+
```
161+
162+
- For pods created by K8s DaemonSet, StatefulSet, [OpenKruise/Cloneset](https://openkruise.io/en-us/docs/cloneset.html), `controller-revision-hash` label will be generated.
163+
```yaml
164+
apiVersion: v1
165+
kind: Pod
166+
metadata:
167+
labels:
168+
controller-revision-hash: b8d6c88f6
169+
```
170+
171+
Declare `definition.oam.dev/unique-pod` in WorkloadDefinition as OAM specified labels, which will always exist in pods.
172+
Then trait automatically fills correlation fields by detecting the unique pod label, for example, generating service, split traffic, etc.
173+
174+
```yaml
175+
apiVersion: core.oam.dev/v1alpha2
176+
kind: WorkloadDefinition
177+
metadata:
178+
name: xxx
179+
labels:
180+
definition.oam.dev/unique-pod: pod-template-hash # optional but recommended, values in [pod-template-hash, controller-revision-hash]
181+
spec:
182+
...
183+
```
184+
185+
Step 2: Utilizing generated workload labels to establish clear relationship between workloads and its component
186+
187+
OAM Kubernetes Runtime now has the ability of [automatically generating these labels for every workload](https://github.com/crossplane/oam-kubernetes-runtime/pull/189) as below.
188+
189+
```yaml
190+
apiVersion: apps/v1
191+
kind: Deployment
192+
metadata:
193+
annotations:
194+
deployment.kubernetes.io/revision: "1"
195+
labels:
196+
app: nginx
197+
app.oam.dev/component: label-component
198+
app.oam.dev/name: label-appconfig
199+
app.oam.dev/revision: label-component-v1
200+
definition.oam.dev/unique-pod: pod-template-hash
201+
```
202+
203+
Label `app.oam.dev/component` marks the Component, `app.oam.dev/name` marks the ApplicationConfiguration, and `app.oam.dev/revision`
204+
marks the revision of the component.
205+
206+
Currently, a trait can find the attached workload by [`spec.workloadRef`](./one-pager-trait-scope-workload-interaction-mechanism.md) field of the Trait. With those labels above, a
207+
trait can quickly find the corresponding component and its revision.
208+
209+
```yaml
210+
spec:
211+
workloadRef:
212+
apiVersion: apps/v1
213+
kind: Deployment
214+
name: label-nginx
215+
```
216+
217+
Moreover, with recommended label `definition.oam.dev/unique-pod` in Component manifest set, we can see it is
218+
generated as one label of the Deployment workload, which can help identify the pods.
219+
220+
## Best practice
221+
222+
- For OAM users
223+
224+
We recommend you to set label `definition.oam.dev/unique-pod` in WorkloadDefinition and Component.
225+
226+
We do NOT recommend you to use Ingress/Service, HPA trait at all. Though based on the mechanism of this proposal,
227+
we can automatically set the required, we encourage users to use official Route and Scale trait.
228+
229+
- For Trait builder
230+
231+
We recommend you to utilize those Workload labels to help identify the Components, ApplicationConfiguration and Pods, to
232+
help implement operation logic.
233+
234+
## Impact to existing system
235+
236+
As the best practice, the proposal won't affect the existing system.

‎examples/workload-labeling/sample_application_config.yaml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
apiVersion: core.oam.dev/v1alpha2
2+
kind: ApplicationConfiguration
3+
metadata:
4+
name: label-appconfig
5+
spec:
6+
components:
7+
- componentName: label-component
8+
traits:
9+
- trait:
10+
apiVersion: core.oam.dev/v1alpha2
11+
kind: ManualScalerTrait
12+
metadata:
13+
name: example-appconfig-trait
14+
spec:
15+
replicaCount: 3

‎examples/workload-labeling/sample_component.yaml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
apiVersion: core.oam.dev/v1alpha2
2+
kind: Component
3+
metadata:
4+
name: label-component
5+
spec:
6+
workload:
7+
apiVersion: apps/v1
8+
kind: Deployment
9+
metadata:
10+
name: label-nginx
11+
labels:
12+
app: nginx
13+
definition.oam.dev/unique-pod: pod-template-hash
14+
spec:
15+
selector:
16+
matchLabels:
17+
app: nginx
18+
template:
19+
metadata:
20+
labels:
21+
app: nginx
22+
spec:
23+
containers:
24+
- name: nginx
25+
image: nginx:1.17
26+
ports:
27+
- containerPort: 80

‎examples/workload-labeling/sample_trait_definition.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
apiVersion: core.oam.dev/v1alpha2
2+
kind: TraitDefinition
3+
metadata:
4+
name: manualscalertraits.core.oam.dev
5+
spec:
6+
workloadRefPath: spec.workloadRef
7+
definitionRef:
8+
name: manualscalertraits.core.oam.dev

‎examples/workload-labeling/sample_workload_definition.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
apiVersion: core.oam.dev/v1alpha2
2+
kind: WorkloadDefinition
3+
metadata:
4+
name: deployments.apps
5+
labels:
6+
definition.oam.dev/unique-pod: pod-template-hash
7+
spec:
8+
definitionRef:
9+
name: deployments.apps

0 commit comments

Comments
 (0)