Cannot use format.com to format a drive with zfs

[aureliencouderc]

We cannot use format.com to format a drive with zfs, like format.com U: /FS:ZFS

[lundman]

You can not format of type zfs, that is zpool create or zfs create.

[sskras]

@aureliencouderc, do you expect the U: to be an ordinary volume created by the usual means on Windows, or do you expect it to reside on a ZFS pool created using zpool.exe? The format.com usage scenario is unclear here.

[guenther-alka]

Windows format can use disks or partitions of a fixed size and format as *fat, ntfs or ReFS.
Disk manager can then manually resize the volume size.

This is not how ZFS works. In ZFS you use storage virtualisation. This means that you create a ZFS pool with zpool create. Such a pool offers capacity for ZFS datasets (filesystems, snaps and zvols). A ZFS filesystem has no given size like a partition but can dynamically grow up to whole poolsize. If you expand a pool e.g. by adding more vdevs, filesystems can grow up to new poolsize without any settings. You can limit usage with quotas and guarantee capacity with reservations.

Whole different concept. This is why you must use zpool and zfs, not format or fdisk.

[sskras]

This is not how ZFS works. In ZFS you use storage virtualisation.

Yes, but...

Whole different concept. This is why you must use zpool and zfs, not format or fdisk.

... invoked format.com can be hooked to create the pool and the dataset on a given Windows volume/partition transparently when supplied with /FS:ZFS on the command line.

At least with WinBTFS it's possible since v0.9 (2017-03-05) only if with some caveats like maharmstone/btrfs#665.

[sskras]

This seems related – quoting Err Msg: This Drive Cannot Be Checked with this Version of UNTFS.DLL (239838):

the code that actually performs the verification when running Chkdsk.exe or Autochk.exe resides in utility .dll files such as Untfs.dll and Ufat.dll

And the libraries are really there:

C:\Users\saukrs> dir C:\Windows\System32\u*.dll | findstr/i "ntfs refs fat udf btrfs"
2024-03-16  00:17           714 432 ubtrfs.dll
2021-08-11  21:46           111 104 uexfat.dll
2021-08-11  21:46           152 064 ufat.dll
2021-10-14  21:01           595 968 untfs.dll
2023-07-20  17:57           808 448 uReFS.dll
2019-12-07  12:08           568 320 uReFSv1.dll
2021-08-11  21:46           173 056 uudf.dll

I guess WinBTRFS implemented the glue-interface there: maharmstone/btrfs@c637bc1

The whole source ubtrfs code around that commit:
https://github.com/maharmstone/btrfs/tree/c637bc1/src/ubtrfs

Its' current form:
https://github.com/maharmstone/btrfs/tree/HEAD/src/ubtrfs

Seems like the mechanism is called FMIFS officially. Even seemingly unused by format.com, it gives a clearer picture of how things are organized under the Windows hood.

So uzfs.dll would require only two functions to be exported:

NTSTATUS WINAPI ChkdskEx();
BOOL __stdcall FormatEx();

Of which only FormatEx() needs to be implemented, eg. by doing what currently zpool create ... does for a single partition.

Now if quick formatting wouldn't be enough, and Windows would demand to implement full formatting (which checking for bad media), it might be feasible to add zpool scrub ..., maybe combine it with a reimplemention of badblocks style scanning. Maybe by threwing all such code into ChkdskEx().

(For those interested)

[sskras]

The bigger issue (at least to me) seems to be addressing a partition which wasn't neither formatted into a volume, nor assigned a drive letter. Eg:

C:\Users\saukrs> wmic volume get Capacity,DeviceId,DriveLetter,FileSystem,Label
Capacity      DeviceID                                           DriveLetter  FileSystem  Label
255405891584  \\?\Volume{3b7e9709-77dc-4d88-ab05-e923b26d95a3}\  C:           NTFS        pagrindinis
529526784     \\?\Volume{878041e1-48c2-484b-bcc1-507b0ff19c4c}\               NTFS
202518818816  \\?\Volume{c27eb7f0-3284-43ef-9f12-c3405d3bcd64}\  F:           NTFS        NTFS Volume
              \\?\Volume{d9c70424-4b75-4019-a347-fee3f39d5ed1}\
100663296     \\?\Volume{5feefb5c-468a-4da8-9fcc-0dfba19dcab8}\               FAT32
41527803904   \\?\Volume{0869fa25-bd66-7074-6498-7f4d08d64db0}\  B:           Btrfs       A Btrfs Vol

It's this volume:

C:\Users\saukrs> mountvol \\?\Volume{d9c70424-4b75-4019-a347-fee3f39d5ed1}\ /L
    \\?\Volume{d9c70424-4b75-4019-a347-fee3f39d5ed1}\

On w10 format.com needes either of these three to identify the volume:

• drive letter (followed by a colon)
• mount point
• volume name

C:\Users\saukrs> format /?
Formats a disk for use with Windows.

FORMAT volume [/FS:file-system] [/V:label] [/Q] [/L[:state]] [/A:size] [/C] [/I:state] [/X] [/P:passes] [/S:state]
FORMAT volume [/V:label] [/Q] [/F:size] [/P:passes]
FORMAT volume [/V:label] [/Q] [/T:tracks /N:sectors] [/P:passes]
FORMAT volume [/V:label] [/Q] [/P:passes]
FORMAT volume [/Q]

  volume          Specifies the drive letter (followed by a colon),
                  mount point, or volume name.
  /FS:filesystem  Specifies the type of the file system (FAT, FAT32, exFAT,
                  NTFS, UDF, ReFS).
  [...]

But since SetVolumeMountPoint() uses GUIDs to address the volume, format.com seems to accept the volume GUID-name too. Just only if you remove the trailing backslash:

C:\Users\saukrs> format \\?\Volume{d9c70424-4b75-4019-a347-fee3f39d5ed1} /FS:BTRFS
Access Denied as you do not have sufficient privileges or
the disk may be locked by another process.
You have to invoke this utility running in elevated mode
and make sure the disk is unlocked.
Cannot open volume for direct access.

C:\Users\saukrs> gsudo format \\?\Volume{d9c70424-4b75-4019-a347-fee3f39d5ed1} /FS:BTRFS
The type of the file system is RAW.
The new file system is BTRFS.

WARNING, ALL DATA ON NON-REMOVABLE DISK
DRIVE \\?\Volume{d9c70424-4b75-4019-a347-fee3f39d5ed1} WILL BE LOST!
Proceed with Format (Y/N)? n

So I think it's pretty doable.

[lundman]

So if you used format it would... make a pool? With no options? Or some options? But no redundancy?

[sskras]

Right, it would create only a single partition based pool – the only way it could work (that I imagine).
Your questions are very valid. Waiting for @aureliencouderc to step in and describe the idea in more detail. :)

Thinking of that more, we could reuse some format.com CLI switches, should they be passable to uzfs.dll:

• for naming the pool + the dataset:

    /V:label        Specifies the volume label.
  

• for enabling the compression:

    /C              NTFS only: Files created on the new volume will be compressed
                    by default.
  

• for leaving metadata redundancy by default:
  

    /D              UDF 2.50 only: Metadata will be duplicated.
  

  Just leave the default redundant_metadata = all intact in this case. Do redundant_metadata = most or so when the switch is missing.
  
  
• for overriding the default recordsize – value of the logical / data block size:

    /A:size         Overrides the default allocation unit size. Default settings
                    are strongly recommended for general use.
                    ReFS supports 4096, 64K.
                    NTFS supports 512, 1024, 2048, 4096, 8192, 16K, 32K, 64K,
                    128K, 256K, 512K, 1M, 2M.
                    FAT supports 512, 1024, 2048, 4096, 8192, 16K, 32K, 64K,
                    (128K, 256K for sector size > 512 bytes).
                    FAT32 supports 512, 1024, 2048, 4096, 8192, 16K, 32K, 64K,
                    (128K, 256K for sector size > 512 bytes).
                    exFAT supports 512, 1024, 2048, 4096, 8192, 16K, 32K, 64K,
                    128K, 256K, 512K, 1M, 2M, 4M, 8M, 16M, 32M.
  

• for initializing empty blocks in the pool:

    /P:count        Zero every sector on the volume.  After that, the volume
                    will be overwritten "count" times using a different
                    random number each time.  If "count" is zero, no additional
                    overwrites are made after zeroing every sector.  This switch
                    is ignored when /Q is specified.
  

  Just like zpool initialize ... with zfs_initialize_value = 0 set for the first iteration, and later with a random value.
  
  
• for enforcing a direct I/O:

    /DAX[:state]    NTFS Only: Enable direct access storage (DAX) mode for this
                    volume.  In DAX mode, the volume is accessed via the memory
                    bus, boosting IO performance.  A volume can be formatted
                    with DAX mode only if the hardware is DAX capable.
                    State can specify "enable" or "disable".  /DAX is considered
                    as /DAX:enable.
  

  ... in the vein of zpool create -O direct=always ...
  
  
• for storing multiple copies of the data:

    /I:state        ReFS only: Specifies whether integrity should be enabled on
                    the new volume. "state" is either "enable" or "disable"
                    Integrity is enabled on storage that supports data redundancy
                    by default.
  

  Eg. zpool create -O copies=2 ....

Seems like there is no way to enable encryption on Windows volumes from the format.com.

Although I doubt all of the aforementioned switches would get signaled to uzfs.dll, the basic three ones should be, I believe: /V:LABEL, /C, and /A:SIZE.

Just my 2 eurocents.