[BUG] Cannot detect root partitions in /dev/disk/azure if OS disk is NVMe

[yuxisun1217]

Describe the bug: A clear and concise description of what the bug is.
For the VM sizes that use NVMe as OS disk(e.g Standard_E2bds_v5 and enable NVMe), the 66-azure-storage.rules cannot detect root partitions. No 'root' links under /dev/disk/azure:

[root@walae2bdsv5nvme ~]# ll /dev/disk/azure
total 0
lrwxrwxrwx. 1 root root  9 Jul 25 07:06 resource -> ../../sda
lrwxrwxrwx. 1 root root 10 Jul 25 07:06 resource-part1 -> ../../sda1

For non-NVMe OS disk which is correct:

# ll /dev/disk/azure
total 0
lrwxrwxrwx. 1 root root  9 Jul 25 07:46 resource -> ../../sdb
lrwxrwxrwx. 1 root root 10 Jul 25 07:46 resource-part1 -> ../../sdb1
lrwxrwxrwx. 1 root root  9 Jul 25 07:46 root -> ../../sda
lrwxrwxrwx. 1 root root 10 Jul 25 07:46 root-part1 -> ../../sda1
lrwxrwxrwx. 1 root root 10 Jul 25 07:46 root-part2 -> ../../sda2
lrwxrwxrwx. 1 root root 10 Jul 25 07:46 root-part3 -> ../../sda3
lrwxrwxrwx. 1 root root 10 Jul 25 07:46 root-part4 -> ../../sda4

Distro and WALinuxAgent details (please complete the following information):
WALinuxAgent-2.7.0.6 running on rhel 9.2
Python: 3.9.16
Goal state agent: 2.9.1.1

Additional context
OS disk is NVMe.

Disk /dev/nvme0n1: 64 GiB, 68719476736 bytes, 134217728 sectors
Disk model: MSFT NVMe Accelerator v1.0              
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disklabel type: gpt
Disk identifier: 004BF26F-565D-44EF-9604-658287E29686

Device           Start       End   Sectors  Size Type
/dev/nvme0n1p1    2048    411647    409600  200M EFI System
/dev/nvme0n1p2  411648   1435647   1024000  500M Linux filesystem
/dev/nvme0n1p3 1435648   1437695      2048    1M BIOS boot
/dev/nvme0n1p4 1437696 134215679 132777984 63.3G Linux LVM

[narrieta]

@anhvoms can you take a look?

[ani-sinha]

@anhvoms any progress on this ticket?

[andyzhangx]

apparently nvme data disk cannot be detected in /dev/disk/azure either

[andyzhangx]

is anyone working on waagent udev rules support on nvme disk? this is a issue blocking customers using v6 sku VM today, you could provision Standard_E2ads_v6 Ubuntu VM to have a try (need whitelist first)

[yuxisun1217]

Find this rule in MSFT document: https://learn.microsoft.com/en-us/azure/virtual-machines/nvme-linux#421-manual-download-of-udev-file
Rule file: https://github.com/Azure/SAP-on-Azure-Scripts-and-Utilities/blob/main/NVMe-Preflight-Check/88-azure-nvme-data-disk.rules

But may need to test it.

[addyess]

bump! I'm seeing this too. It's quite annoying!
@narrieta any thoughts here?

[addyess]

@yuxisun1217 i tried out this udev rule file and didn't have much success

checkout this device which is lun0

root@the-vm:/usr/lib/udev/rules.d# readlink /sys/class/block/nvme0n2/../device
../../../c05b:00:00.0
root@the-vm:/usr/lib/udev/rules.d# readlink /sys/class/block/nvme0n2/device
../../nvme0

the udev rule plays some games trying to bake the device id into the lun id

ENV{DEVTYPE}=="partition", PROGRAM="/bin/sh -c 'readlink /sys/class/block/%k/../device|cut -d: -f4'", ENV{fabric_name}="data/by-lun/$result"
ENV{DEVTYPE}=="disk", PROGRAM="/bin/sh -c 'readlink /sys/class/block/%k/device|cut -d: -f4'", ENV{fabric_name}="data/by-lun/$result"

those cut don't work out the way as intended...

did you have any success with anything else?