HelenRAID

[mcimerman]

HelenRAID

Notes

Part of a bachelor's thesis at MFF UK.

Even though this project is not finished yet, I would like to get some
feedback before thesis submission (17.7.2025).

This is just a draft. Which means that I will gradually add planned
features and comment code.

For a quick introduction and start, please look at project HelenRAID_presentation.pdf.

Try it

There is create_file_bd_disks.bdsh at uspace/app/hrctl which creates 4
100MB virtual disks you can use for testing. When the image is created it is
copied to /w/cfg/create_file_bd_disks.bdsh.

You can create a sample RAID 1 volume from /w/cfg/sample_hrconfig_file_bd.sif
like this:

batch /w/cfg/create_file_bd_disks.bdsh

hrctl -c -f /w/cfg/sample_hrconfig_file_bd.sif

Print its state:

hrctl -s

Add hotspare:

hrctl --modify devices/hr1 --hotspare disk4

Check the state if you like.

Then fail an extent (give the index [0, 2]);

hrctl --modify devices/hr1 --fail index

Check the state again, a rebuild should be in progress.

---

There is also sample_hrconfig_pci.sif for using QEMU disks.

---

For some I/O testing there is bdwrite tool in uspace/app/bdwrite,
which writes cyclic blocks (filled A-Z) to specified block devices.
Run qemu with some disks attached, create a volume and run:

bdwrite devices/volume_name -o offset -c block_count

Then check for example with hexdump -C hr_disk1.img the contents.
Or blkdump in HelenOS.

---

When using the attached Qemu disks, you can try creating a volume,
restarting Qemu with smaller number of original disks and automatically
assembling with hrctl -a.

Caveats

will be dealt with

• REBUILD position is not currently saved
• Simulating extent fault currently doesn't erase the superblock
  • So when automatic re-assembly is used, it can pick the invalidated
    extent, even though there is a newly rebuilt extent (former hotspare)

will not be dealt with

• because of putting volumes in new raid location service category,
  automatic disassembly and assembly of stacked level volumes is not
  supported
  • the effects of hrctl -d hrctl -a are undefined when
    there are disks containing stacked levels connected to the system

To be added

• more code comments
• REBUILD progress saving and proper metadata on-disk counter handling
• Parallel RAID 5
• Linux md metadata format support

Tree

High-level tree:

uspace/app/hrctl        - client
uspace/lib/device/hr    - client-server API, and common types and defines
uspace/srv/bd/hr        - server

uspace/srv/bd/hr tree:

├── fge.c                        - fibril group executor
├── fge.h
├── hr.c                         - server main
├── io.c                         - I/O workers
├── io.h
├── meson.build
├── metadata                     - specific metadata dir
│   ├── foreign                  - non-native metadata formats
│   │   ├── geom
│   │   │   ├── g_mirror.h
│   │   │   ├── g_stripe.h
│   │   │   ├── hr_g_mirror.c    - mapping to HelenRAID metadata functions
│   │   │   ├── hr_g_stripe.c    - mapping to HelenRAID metadata functions
│   │   │   └── sys_endian.h
│   │   └── softraid
│   │       ├── hr_softraid.c    - mapping to HelenRAID metadata functions  
│   │       ├── softraid.c
│   │       └── softraidvar.h
│   ├── native.c                 - native metadata format code
│   └── native.h
├── raid0.c                      - striping code
├── raid1.c                      - mirroring code
├── raid5.c                      - parity code (OUTDATED, parallel code WIP)
├── superblock.c                 - generic metadata code
├── superblock.h
├── util.c                       - utility and helper code
├── util.h
└── var.h                        - internal volume types and defines