Skip to content

Commit e265c6b

Browse files
github-actions[bot]github-actions[bot]
committed
Build: (ee2cc1e) Merge pull request #389 from kubevirt/gh_actions/update_metrics_docs
0 parents  commit e265c6b

97 files changed

Lines changed: 7529 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎index.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
## Metrics
2+
3+
- [Metrics](metrics.md)
4+
5+
## Runbooks
6+
7+
- [Runbooks](runbooks_index.md)
8+

‎metrics.md‎

Lines changed: 248 additions & 0 deletions
Large diffs are not rendered by default.

‎runbooks/CDIDataImportCronOutdated.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# CDIDataImportCronOutdated
2+
3+
## Meaning
4+
5+
This alert fires when `DataImportCron` cannot poll or import the latest disk
6+
image versions.
7+
8+
`DataImportCron` polls disk images, checking for the latest versions, and
9+
imports the images into persistent volume claims (PVCs) or VolumeSnapshots. This
10+
process ensures that these sources are updated to the latest version so that
11+
they can be used as reliable clone sources or golden images for virtual machines
12+
(VMs).
13+
14+
For golden images, _latest_ refers to the latest operating system of the
15+
distribution. For other disk images, _latest_ refers to the latest hash of the
16+
image that is available.
17+
18+
**Note:** If the status of a `DataImportCron` PVC is `Pending` because there is no
19+
default storage class, the `CDIDataImportCronOutdated` alert is suppressed and the
20+
`CDINoDefaultStorageClass` alert is triggered.
21+
22+
## Impact
23+
24+
VMs might be created from outdated disk images.
25+
26+
VMs might fail to start because no boot source is available for cloning.
27+
28+
## Diagnosis
29+
30+
1. Check the cluster for a default Kubernetes storage class:
31+
```bash
32+
$ kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubernetes\.io\/is-default-class=="true")].metadata.name}'
33+
```
34+
35+
Check the cluster for a default virtualization storage class:
36+
```bash
37+
$ kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubevirt\.io\/is-default-virt-class=="true")].metadata.name}'
38+
```
39+
40+
The output displays the default (Kubernetes and/or virtualization) storage
41+
class. You must either set a default storage class on the cluster, or ask for
42+
a specific storage class in the `DataImportCron` specification, in order for
43+
the `DataImportCron` to poll and import golden images. If the default
44+
storage class does not exist, the created import DataVolume and PVC will be
45+
in `Pending` phase.
46+
47+
2. List the `DataImportCron` objects that are not up-to-date:
48+
49+
```bash
50+
$ kubectl get dataimportcron -A -o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="UpToDate")].status}{"\t"}{.metadata.namespace}{"/"}{.metadata.name}{"\n"}{end}' | grep False
51+
```
52+
53+
3. If a default storage class is not defined on the cluster, check the
54+
`DataImportCron` specification for a `DataVolume` template storage class:
55+
56+
```bash
57+
$ kubectl -n <namespace> get dataimportcron <dataimportcron> -o jsonpath='{.spec.template.spec.storage.storageClassName}{"\n"}'
58+
```
59+
60+
4. Obtain the name of the `DataVolume` associated with the `DataImportCron`
61+
object:
62+
63+
```bash
64+
$ kubectl -n <namespace> get dataimportcron <dataimportcron> -o jsonpath='{.status.lastImportedPVC.name}{"\n"}'
65+
```
66+
67+
5. Check the `DataVolume` status:
68+
69+
```bash
70+
$ kubectl -n <namespace> get dv <datavolume> -o jsonpath-as-json='{.status}'
71+
```
72+
73+
6. Set the `CDI_NAMESPACE` environment variable:
74+
75+
```bash
76+
$ export CDI_NAMESPACE="$(kubectl get deployment -A -o jsonpath='{.items[?(.metadata.name=="cdi-operator")].metadata.namespace}')"
77+
```
78+
79+
7. Check the `cdi-deployment` log for error messages:
80+
81+
```bash
82+
$ kubectl logs -n $CDI_NAMESPACE deployment/cdi-deployment
83+
```
84+
85+
## Mitigation
86+
87+
1. Set a default storage class, either on the cluster or in the `DataImportCron`
88+
specification, to poll and import golden images. The updated Containerized Data
89+
Importer (CDI) should resolve the issue within a few seconds.
90+
91+
2. If the issue does not resolve itself, or, if you have changed the default
92+
storage class in the cluster, you must delete the existing boot sources
93+
(data volumes or volume snapshots) in the cluster namespace that are configured
94+
with the previous default storage class. The CDI will recreate the data volumes
95+
with the newly configured default storage class.
96+
97+
3. If your cluster is installed in a restricted network environment, disable the
98+
`enableCommonBootImageImport` feature gate in order to opt out of automatic
99+
updates:
100+
101+
```bash
102+
$ kubectl patch hco kubevirt-hyperconverged -n $CDI_NAMESPACE --type json -p '[{"op": "replace", "path": "/spec/featureGates/enableCommonBootImageImport", "value": false}]'
103+
```
104+
105+
<!--DS: If you cannot resolve the issue, log in to the
106+
link:https://access.redhat.com[Customer Portal] and open a support case,
107+
attaching the artifacts gathered during the diagnosis procedure.-->
108+
<!--USstart-->
109+
See the [HCO cluster configuration documentation](https://github.com/kubevirt/hyperconverged-cluster-operator/blob/main/docs/cluster-configuration.md#enablecommonbootimageimport-feature-gate)
110+
for more information.
111+
112+
If you cannot resolve the issue, see the following resources:
113+
114+
- [OKD Help](https://okd.io/docs/community/help/)
115+
- [#virtualization Slack channel](https://kubernetes.slack.com/channels/virtualization)
116+
<!--USend-->

‎runbooks/CDIDataVolumeUnusualRestartCount.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# CDIDataVolumeUnusualRestartCount
2+
3+
## Meaning
4+
5+
This alert fires when a `DataVolume` object restarts more than three times.
6+
7+
## Impact
8+
9+
Data volumes are responsible for importing and creating a virtual machine disk
10+
on a persistent volume claim. If a data volume restarts more than three times,
11+
these operations are unlikely to succeed. You must diagnose and resolve the
12+
issue.
13+
14+
## Diagnosis
15+
16+
1. Find Containerized Data Importer (CDI) pods with more than three restarts:
17+
18+
```bash
19+
$ kubectl get pods --all-namespaces -l app=containerized-data-importer -o=jsonpath='{range .items[?(@.status.containerStatuses[0].restartCount>3)]}{.metadata.name}{"/"}{.metadata.namespace}{"\n"}'
20+
```
21+
22+
2. Obtain the details of the pods:
23+
24+
```bash
25+
$ kubectl -n <namespace> describe pods <pod>
26+
```
27+
28+
3. Check the pod logs for error messages:
29+
30+
```bash
31+
$ kubectl -n <namespace> logs <pod>
32+
```
33+
34+
## Mitigation
35+
36+
Delete the data volume, resolve the issue, and create a new data volume.
37+
38+
<!--DS: If you cannot resolve the issue, log in to the
39+
link:https://access.redhat.com[Customer Portal] and open a support case,
40+
attaching the artifacts gathered during the diagnosis procedure.-->
41+
<!--USstart-->
42+
If you cannot resolve the issue, see the following resources:
43+
44+
- [OKD Help](https://okd.io/docs/community/help/)
45+
- [#virtualization Slack channel](https://kubernetes.slack.com/channels/virtualization)
46+
<!--USend-->

‎runbooks/CDIDefaultStorageClassDegraded.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# CDIDefaultStorageClassDegraded
2+
3+
## Meaning
4+
5+
This alert fires if the default storage class does not support smart cloning
6+
(CSI or snapshot-based) or the ReadWriteMany access mode. The alert does not
7+
fire if at least one default storage class supports these features.
8+
9+
A default virtualization storage class has precedence over a default Kubernetes
10+
storage class for creating a VirtualMachine disk image.
11+
12+
<!--DS: In case of single-node OpenShift, the alert is suppressed if there is a default
13+
storage class that supports smart cloning, but not ReadWriteMany.-->
14+
15+
## Impact
16+
17+
If the default storage class does not support smart cloning, the default cloning
18+
method is host-assisted cloning, which is much less efficient.
19+
20+
If the default storage class does not support ReadWriteMany, virtual machines
21+
(VMs) cannot be live migrated.
22+
23+
## Diagnosis
24+
25+
1. Get the default virtualization storage class by running the following
26+
command:
27+
28+
```bash
29+
$ export CDI_DEFAULT_VIRT_SC="$(kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubernetes\.io\/is-default-class=="true")].metadata.name}')"
30+
```
31+
32+
2. If a default virtualization storage class exists, check that it supports
33+
ReadWriteMany by running the following command:
34+
35+
```bash
36+
$ kubectl get storageprofile $CDI_DEFAULT_VIRT_SC -o jsonpath='{.status.claimPropertySets}' | grep ReadWriteMany
37+
```
38+
39+
3. If there is no default virtualization storage class, get the default
40+
Kubernetes storage class by running the following command:
41+
42+
```bash
43+
$ export CDI_DEFAULT_K8S_SC="$(kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubernetes\.io\/is-default-class=="true")].metadata.name}')"
44+
```
45+
46+
4. If a default Kubernetes storage class exists, check that it supports
47+
ReadWriteMany by running the following command:
48+
49+
```bash
50+
$ kubectl get storageprofile $CDI_DEFAULT_VIRT_SC -o jsonpath='{.status.claimPropertySets}' | grep ReadWriteMany
51+
```
52+
53+
<!--USstart-->
54+
See [doc](https://github.com/kubevirt/containerized-data-importer/blob/main/doc/efficient-cloning.md)
55+
for details about smart clone prerequisites.
56+
<!--USend-->
57+
58+
## Mitigation
59+
60+
Ensure that you have a default (Kubernetes or virtualization) storage class, and
61+
that the default storage class supports smart cloning and ReadWriteMany.
62+
63+
<!--USstart-->
64+
If you cannot resolve the issue, see the following resources:
65+
66+
- [OKD Help](https://okd.io/docs/community/help/)
67+
- [#virtualization Slack channel](https://kubernetes.slack.com/channels/virtualization)
68+
<!--USend-->
69+
70+
<!--DS: If you cannot resolve the issue, log in to the
71+
[Customer Portal](https://access.redhat.com) and open a support case, attaching
72+
the artifacts gathered during the diagnosis procedure.-->

‎runbooks/CDIMultipleDefaultVirtStorageClasses.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# CDIMultipleDefaultVirtStorageClasses
2+
3+
## Meaning
4+
5+
This alert fires when more than one default virtualization storage class exists.
6+
7+
A default virtualization storage class has precedence over a default Kubernetes
8+
storage class for creating a VirtualMachine disk image.
9+
10+
## Impact
11+
12+
If more than one default virtualization storage class exists, a data volume that
13+
requests a default storage class (storage class not explicitly specified),
14+
receives the most recently created one.
15+
16+
## Diagnosis
17+
18+
Obtain a list of default virtualization storage classes by running the following
19+
command:
20+
21+
```bash
22+
$ kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubevirt\.io\/is-default-virt-class=="true")].metadata.name}'
23+
```
24+
25+
## Mitigation
26+
27+
Ensure that only one storage class has the default virtualization storage class
28+
annotation.
29+
30+
<!--DS: If you cannot resolve the issue, log in to the
31+
link:https://access.redhat.com[Customer Portal] and open a support case,
32+
attaching the artifacts gathered during the diagnosis procedure.-->
33+
<!--USstart-->
34+
If you cannot resolve the issue, see the following resources:
35+
36+
- [OKD Help](https://okd.io/docs/community/help/)
37+
- [#virtualization Slack channel](https://kubernetes.slack.com/channels/virtualization)
38+
<!--USend-->

‎runbooks/CDINoDefaultStorageClass.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# CDINoDefaultStorageClass
2+
3+
## Meaning
4+
5+
This alert fires when a data volume is `Pending` because there is no default
6+
storage class.
7+
8+
A default virtualization storage class has precedence over a default Kubernetes
9+
storage class for creating a VirtualMachine disk image.
10+
11+
## Impact
12+
13+
If there is no default Kubernetes storage class and no default virtualization
14+
storage class, a data volume that does not have a specified storage class
15+
remains in a `Pending` phase.
16+
17+
## Diagnosis
18+
19+
1. Check for a default Kubernetes storage class by running the following
20+
command:
21+
22+
```bash
23+
$ kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubernetes\.io\/is-default-class=="true")].metadata.name}'
24+
```
25+
26+
2. Check for a default virtualization storage class by running the following
27+
command:
28+
29+
```bash
30+
$ kubectl get sc -o jsonpath='{.items[?(.metadata.annotations.storageclass\.kubevirt\.io\/is-default-virt-class=="true")].metadata.name}'
31+
```
32+
33+
## Mitigation
34+
35+
Create a default storage class for Kubernetes, virtualization, or both.
36+
37+
A default virtualization storage class has precedence over a default Kubernetes
38+
storage class for creating a virtual machine disk image.
39+
40+
* Create a default Kubernetes storage class by running the following command:
41+
42+
```bash
43+
$ kubectl patch storageclass <storage-class-name> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
44+
```
45+
46+
* Create a default virtualization storage class by running the following
47+
command:
48+
49+
```bash
50+
$ kubectl patch storageclass <storage-class-name> -p '{"metadata": {"annotations":{"storageclass.kubevirt.io/is-default-virt-class":"true"}}}'
51+
```
52+
53+
<!--USstart-->
54+
If you cannot resolve the issue, see the following resources:
55+
56+
- [OKD Help](https://okd.io/docs/community/help/)
57+
- [#virtualization Slack channel](https://kubernetes.slack.com/channels/virtualization)
58+
<!--USend-->
59+
60+
<!--DS: If you cannot resolve the issue, log in to the
61+
[Customer Portal](https://access.redhat.com) and open a support case,
62+
attaching the artifacts gathered during the diagnosis procedure.-->

0 commit comments

Comments
 (0)