add a 3 cirros-vm test for manual recreate (#1626)

* add a 3 cirros-vm test for manual recreate

* update restore yaml

* add some notes

* added second test case

* updated w/ container based disks for vms

* cleanup readme

* clean up cirros pv test,readme,results

* more cleanup

* add screenshot of diff

* fix file name

* Update tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/README.MD

Co-authored-by: Scott Seago <[email protected]>

* Update tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/README.MD

Co-authored-by: Scott Seago <[email protected]>

* retested and updated for latest d.s. builds

* remove old results

---------

Co-authored-by: Scott Seago <[email protected]>

Author: weshayutin

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/README.MD

@@ -0,0 +1,69 @@
+# Various test cases for backing up / restoring vms using selectors
+
+## DataVolume precreated: backup all, restore one at a time
+#### setup:
+```
+alias velero='oc -n openshift-adp exec deployment/velero -c velero -it -- ./velero'
+cd cirros-test-with-dataVolumePV 
+oc create namespace cirros-test-ds-pv
+oc create -f cirros-datasource-datavolume.yaml
+oc create -f cirros-test-1.yaml -f cirros-test-2.yaml -f cirros-test-3.yaml
+velero backup create backup-cirros-test-pv-all --include-namespaces cirros-test-ds-pv --snapshot-move-data=true
+oc delete -f cirros-test-1.yaml -f cirros-test-2.yaml -f cirros-test-3.yaml
+```
+
+#### to restore:
+```
+alias velero='oc -n openshift-adp exec deployment/velero -c velero -it -- ./velero'
+velero restore create cirros-test-pv-1  --from-backup backup-cirros-test-pv-all --selector app=cirros-test-pv-1
+velero restore create cirros-test-pv-2  --from-backup backup-cirros-test-pv-all --selector app=cirros-test-pv-2
+velero restore create cirros-test-pv-3  --from-backup backup-cirros-test-pv-all --selector app=cirros-test-pv-3
+```
+
+**The above use case should all work as expected. \0/**
+See the results in [cirros-test-with-dataVolumePV/RESULTS](cirros-test-with-dataVolumePV/RESULTS)
+
+**To restore the datasource and datavolume**
+You may want to test the restore of the vm's after also deleting the datasource and datavolume.
+Include the `openshift-virtualization-os-images` namespace in the original backup.
+
+```
+spec:
+  defaultVolumesToFsBackup: false
+  storageLocation: velero-sample-1
+  includedNamespaces:
+    - cirros-test-ds-pv
+    - openshift-virtualization-os-images
+  snapshotMoveData: true
+```
+
+* note: restoring by label without the datavolume and datasource previously restored or present will not work unless tagged appropriately.
+
+## DataVolume not precreated, container disk: backup all, restore one at a time
+#### setup:
+```
+alias velero='oc -n openshift-adp exec deployment/velero -c velero -it -- ./velero'
+cd cirros-test-with-dataVolumeContainer
+
+oc create namespace cirros-test-cont
+oc create -f cirros-test-1.yaml -f cirros-test-2.yaml -f cirros-test-3.yaml
+velero backup create backup-cirros-test-cont-all --include-namespaces cirros-test-cont --snapshot-move-data=true
+oc delete -f cirros-test-1.yaml -f cirros-test-2.yaml -f cirros-test-3.yaml
+```
+#### to restore scenario 1: (does NOT work)
+See the results in [cirros-test-with-dataVolumeContainer/RESULTS](cirros-test-with-dataVolumeContainer/RESULTS)
+
+```
+velero restore create cirros-test-cont-1  --from-backup backup-cirros-test-cont-all --selector app=cirros-test-cont-1
+velero restore create cirros-test-cont-2  --from-backup backup-cirros-test-cont-all --selector app=cirros-test-cont-2
+velero restore create cirros-test-cont-3  --from-backup backup-cirros-test-cont-all --selector app=cirros-test-cont-3
+```
+
+#### to restore scenario 2: (works)
+```
+velero restore create cirros-test-1  --from-backup backup-cirros-test-cont-all
+```
+
+**Velero selectors in some cases are not working, The datavolume PV/PVC is not a restore resource**
+
+[visual diff of restore's resources](cirros-test-with-dataVolumeContainer/RESULTS/visual_diff_of_restore.png)

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/cirros-test-with-dataVolumeContainer/RESULTS/cirros-test-cont-1-dv

@@ -0,0 +1,54 @@
+apiVersion: cdi.kubevirt.io/v1beta1
+kind: DataVolume
+metadata:
+  annotations:
+    cdi.kubevirt.io/allowClaimAdoption: "true"
+    cdi.kubevirt.io/storage.deleteAfterCompletion: "false"
+    cdi.kubevirt.io/storage.prePopulated: cirros-test-cont-1-dv
+    cdi.kubevirt.io/storage.usePopulator: "true"
+  creationTimestamp: "2025-02-14T18:02:10Z"
+  generation: 1
+  labels:
+    kubevirt.io/created-by: 6a13cdf9-5248-43c5-af45-173686433723
+    velero.io/backup-name: backup-cirros-test-cont-all
+    velero.io/restore-name: cirros-test-cont-1
+  name: cirros-test-cont-1-dv
+  namespace: cirros-test-cont
+  ownerReferences:
+  - apiVersion: kubevirt.io/v1
+    blockOwnerDeletion: true
+    controller: true
+    kind: VirtualMachine
+    name: cirros-test-cont-1
+    uid: 8df919d1-9798-47f8-b4ce-1ba0859d451c
+  resourceVersion: "24067411"
+  uid: bf4409e2-1992-4d36-bfb9-3c29857947dc
+spec:
+  pvc:
+    accessModes:
+    - ReadWriteOnce
+    resources:
+      requests:
+        storage: 150Mi
+    volumeMode: Block
+  source:
+    registry:
+      pullMethod: node
+      url: docker://quay.io/kubevirt/cirros-container-disk-demo
+status:
+  conditions:
+  - lastHeartbeatTime: "2025-02-14T18:02:10Z"
+    lastTransitionTime: "2025-02-14T18:02:10Z"
+    message: No PVC found
+    reason: NotFound
+    status: Unknown
+    type: Bound
+  - lastHeartbeatTime: "2025-02-14T18:02:10Z"
+    lastTransitionTime: "2025-02-14T18:02:10Z"
+    status: "False"
+    type: Ready
+  - lastHeartbeatTime: "2025-02-14T18:02:10Z"
+    lastTransitionTime: "2025-02-14T18:02:10Z"
+    status: "False"
+    type: Running
+  phase: Pending

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/cirros-test-with-dataVolumeContainer/RESULTS/describe-cirros-test-cont-1-dv

@@ -0,0 +1,56 @@
+Name:         cirros-test-cont-1-dv
+Namespace:    cirros-test-cont
+Labels:       kubevirt.io/created-by=6a13cdf9-5248-43c5-af45-173686433723
+              velero.io/backup-name=backup-cirros-test-cont-all
+              velero.io/restore-name=cirros-test-cont-1
+Annotations:  cdi.kubevirt.io/allowClaimAdoption: true
+              cdi.kubevirt.io/storage.deleteAfterCompletion: false
+              cdi.kubevirt.io/storage.prePopulated: cirros-test-cont-1-dv
+              cdi.kubevirt.io/storage.usePopulator: true
+API Version:  cdi.kubevirt.io/v1beta1
+Kind:         DataVolume
+Metadata:
+  Creation Timestamp:  2025-02-14T18:02:10Z
+  Generation:          1
+  Owner References:
+    API Version:           kubevirt.io/v1
+    Block Owner Deletion:  true
+    Controller:            true
+    Kind:                  VirtualMachine
+    Name:                  cirros-test-cont-1
+    UID:                   8df919d1-9798-47f8-b4ce-1ba0859d451c
+  Resource Version:        24067411
+  UID:                     bf4409e2-1992-4d36-bfb9-3c29857947dc
+Spec:
+  Pvc:
+    Access Modes:
+      ReadWriteOnce
+    Resources:
+      Requests:
+        Storage:  150Mi
+    Volume Mode:  Block
+  Source:
+    Registry:
+      Pull Method:  node
+      URL:          docker://quay.io/kubevirt/cirros-container-disk-demo
+Status:
+  Conditions:
+    Last Heartbeat Time:   2025-02-14T18:02:10Z
+    Last Transition Time:  2025-02-14T18:02:10Z
+    Message:               No PVC found
+    Reason:                NotFound
+    Status:                Unknown
+    Type:                  Bound
+    Last Heartbeat Time:   2025-02-14T18:02:10Z
+    Last Transition Time:  2025-02-14T18:02:10Z
+    Status:                False
+    Type:                  Ready
+    Last Heartbeat Time:   2025-02-14T18:02:10Z
+    Last Transition Time:  2025-02-14T18:02:10Z
+    Status:                False
+    Type:                  Running
+  Phase:                   Pending
+Events:
+  Type    Reason    Age    From                          Message
+  ----    ------    ----   ----                          -------
+  Normal  NotFound  3m50s  datavolume-import-controller  No PVC found

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/cirros-test-with-dataVolumeContainer/RESULTS/describe-vm-cirros-test-cont-1

@@ -0,0 +1,80 @@
+Name:         cirros-test-cont-1
+Namespace:    cirros-test-cont
+Labels:       app=cirros-test-cont-1
+              velero.io/backup-name=backup-cirros-test-cont-all
+              velero.io/restore-name=cirros-test-cont-1
+Annotations:  kubevirt.io/latest-observed-api-version: v1
+              kubevirt.io/storage-observed-api-version: v1
+API Version:  kubevirt.io/v1
+Kind:         VirtualMachine
+Metadata:
+  Creation Timestamp:  2025-02-14T18:02:10Z
+  Finalizers:
+    kubevirt.io/virtualMachineControllerFinalize
+  Generation:        1
+  Resource Version:  24067415
+  UID:               8df919d1-9798-47f8-b4ce-1ba0859d451c
+Spec:
+  Data Volume Templates:
+    Metadata:
+      Annotations:
+        cdi.kubevirt.io/storage.deleteAfterCompletion:  false
+      Creation Timestamp:                               <nil>
+      Name:                                             cirros-test-cont-1-dv
+    Spec:
+      Pvc:
+        Access Modes:
+          ReadWriteOnce
+        Resources:
+          Requests:
+            Storage:  150Mi
+        Volume Mode:  Block
+      Source:
+        Registry:
+          Pull Method:  node
+          URL:          docker://quay.io/kubevirt/cirros-container-disk-demo
+  Running:              true
+  Template:
+    Metadata:
+      Creation Timestamp:  <nil>
+      Name:                cirros-test-cont-1
+    Spec:
+      Architecture:  amd64
+      Domain:
+        Devices:
+          Disks:
+            Disk:
+              Bus:  virtio
+            Name:   volume0
+          Interfaces:
+            Mac Address:  02:87:36:00:00:7e
+            Masquerade:
+            Name:  default
+          Rng:
+        Machine:
+          Type:  q35
+        Resources:
+          Requests:
+            Memory:  256M
+      Networks:
+        Name:  default
+        Pod:
+      Termination Grace Period Seconds:  0
+      Volumes:
+        Data Volume:
+          Name:  cirros-test-cont-1-dv
+        Name:    volume0
+Status:
+  Conditions:
+    Last Probe Time:       2025-02-14T18:02:10Z
+    Last Transition Time:  2025-02-14T18:02:10Z
+    Message:               VMI does not exist
+    Reason:                VMINotExists
+    Status:                False
+    Type:                  Ready
+  Printable Status:        Provisioning
+  Volume Snapshot Statuses:
+    Enabled:  false
+    Name:     volume0
+    Reason:   PVC not found
+Events:       <none>

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/cirros-test-with-dataVolumeContainer/RESULTS/restore-cirros-test-cont-1

@@ -0,0 +1,53 @@
+Name:         cirros-test-cont-1
+Namespace:    openshift-adp
+Labels:       <none>
+Annotations:  <none>
+
+Phase:                       Completed
+Total items to be restored:  5
+Items restored:              5
+
+Started:    2025-02-14 18:02:09 +0000 UTC
+Completed:  2025-02-14 18:02:10 +0000 UTC
+
+Backup:  backup-cirros-test-cont-all
+
+Namespaces:
+  Included:  all namespaces found in the backup
+  Excluded:  <none>
+
+Resources:
+  Included:        *
+  Excluded:        nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io, csinodes.storage.k8s.io, volumeattachments.storage.k8s.io, backuprepositories.velero.io
+  Cluster-scoped:  auto
+
+Namespace mappings:  <none>
+
+Label selector:  app=cirros-test-cont-1
+
+Or label selector:  <none>
+
+Restore PVs:  auto
+
+CSI Snapshot Restores: <none included>
+
+Existing Resource Policy:   <none>
+ItemOperationTimeout:       4h0m0s
+
+Preserve Service NodePorts:  auto
+
+Uploader config:
+
+
+HooksAttempted:   0
+HooksFailed:      0
+
+Resource List:
+  cdi.kubevirt.io/v1beta1/DataVolume:
+    - cirros-test-cont/cirros-test-cont-1-dv(created)
+  kubevirt.io/v1/VirtualMachine:
+    - cirros-test-cont/cirros-test-cont-1(created)
+  velero.io/v2alpha1/DataUpload:
+    - openshift-adp/backup-cirros-test-cont-all-fvs4v(skipped)
+    - openshift-adp/backup-cirros-test-cont-all-nkdsq(skipped)
+    - openshift-adp/backup-cirros-test-cont-all-tgtb7(skipped)

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/cirros-test-with-dataVolumeContainer/RESULTS/restore-cirros-test-cont-1-get-all

@@ -0,0 +1,5 @@
+NAME                                               PHASE     PROGRESS   RESTARTS   AGE
+datavolume.cdi.kubevirt.io/cirros-test-cont-1-dv   Pending                         107s
+
+NAME                                            AGE    STATUS         READY
+virtualmachine.kubevirt.io/cirros-test-cont-1   108s   Provisioning   False

tests/e2e/sample-applications/virtual-machines/cirros-test/3-vms/cirros-test-with-dataVolumeContainer/RESULTS/visual_diff_of_restore.png

@@ -0,0 +1,58 @@
+apiVersion: kubevirt.io/v1
+kind: VirtualMachine
+metadata:
+  annotations:
+  generation: 1
+  labels:
+    app: cirros-test-cont-1
+  name: cirros-test-cont-1
+  namespace: cirros-test-cont
+spec:
+  dataVolumeTemplates:
+  - metadata:
+      annotations:
+        cdi.kubevirt.io/storage.deleteAfterCompletion: "false"
+      creationTimestamp: null
+      name: cirros-test-cont-1-dv
+    spec:
+      pvc:
+        accessModes:
+        - ReadWriteOnce
+        resources:
+          requests:
+            storage: 150Mi
+        volumeMode: Block
+      source:
+        registry:
+          pullMethod: node
+          url: docker://quay.io/kubevirt/cirros-container-disk-demo
+  running: true
+  template:
+    metadata:
+      creationTimestamp: null
+      name: cirros-test-cont-1
+    spec:
+      architecture: amd64
+      domain:
+        devices:
+          disks:
+          - disk:
+              bus: virtio
+            name: volume0
+          interfaces:
+          - masquerade: {}
+            name: default
+          rng: {}
+        machine:
+          type: q35
+        resources:
+          requests:
+            memory: 256M
+      networks:
+      - name: default
+        pod: {}
+      terminationGracePeriodSeconds: 0
+      volumes:
+      - dataVolume:
+          name: cirros-test-cont-1-dv
+        name: volume0