btrfs-progs: docs: cumulative update

- formatting, typo fixes, corrections
- cross references
- enhancements
- revised and colored status page

Signed-off-by: David Sterba <dsterba@suse.com>

Author: kdave

Documentation/Common-features.rst

@@ -16,8 +16,13 @@ statx
         *stat*
 
 fallocate modes
-        the *fallocate* syscall allows to manipulate file extents like punching
-        holes, preallocation or zeroing a range
+        the *fallocate* syscall allows to manipulate file extents
+
+        Supported modes:
+
+        - keep size (FALLOC_FL_KEEP_SIZE)
+        - hole punching (FALLOC_FL_PUNCH_HOLE)
+        - zero range (FALLOC_FL_ZERO_RANGE)
 
 FIEMAP
         an ioctl that enumerates file extents, related tool is :command:`filefrag`
@@ -39,7 +44,7 @@ xattr, acl
 
 cross-rename
         mode of *renameat2* syscall that can atomically swap 2 directory
-        entries (files/directories/subvolumes)
+        entries (files/directories/subvolumes/symlinks) within the same filesystem
 
 
 File attributes, XFLAGS

Documentation/Custom-ioctls.rst

@@ -19,3 +19,6 @@ a command if available:
   which is also exported in :file:`/sys/fs/btrfs/`
 
 - query/set a subset of features on a mounted filesystem
+
+Programming documentaion of the ioctls is in the manual page
+:doc:`btrfs-ioctl(2)<btrfs-ioctl>`.

Documentation/Feature-by-version.rst

@@ -71,7 +71,7 @@ features see [[Status]] page.
 
         .. note::
            Disabled since 3.14 (and backported to some stable kernel versions)
-           due to problems. Will be enabled in the future.
+           due to problems. Has been completely removed in 5.6.
 
 3.9 - lightweight send
         A mode of *send* that does not add the actual file data to the stream

Documentation/Interoperability.rst

@@ -3,18 +3,28 @@
 Interoperability
 ================
 
+.. _interop-nfs:
+
 NFS
 ---
 
+.. _interop-samba:
+
 Samba
 -----
 
+.. _interop-cgroups:
+
 cgroups
 -------
 
+.. _interop-fsverity:
+
 fsverity
 --------
 
+.. _interop-idmapped:
+
 idmapped mounts
 ---------------
 
@@ -27,5 +37,7 @@ overlayfs
 SELinux
 -------
 
+.. _interop-io-uring:
+
 io_uring
 --------

Documentation/Source-repositories.rst

@@ -108,6 +108,6 @@ patch page to the command:
 
 .. code-block:: bash
 
-        $ wget -O - '<nowiki>https://patchwork.kernel.org/patch/123456/mbox</nowiki>' | git am -
+        $ wget -O - 'https://patchwork.kernel.org/patch/123456/mbox' | git am -
 
 You may want to add *--reject*, or decide otherwise what to do with the patch.

Documentation/Status.rst

@@ -13,7 +13,7 @@ in meeting your performance expectations for your specific workload.
 Combination of features can vary in performance, the table does not
 cover all possibilities.
 
-**The table is based on the latest released linux kernel: 6.2**
+**The table is based on the latest released linux kernel: 6.3**
 
 The columns for each feature reflect the status of the implementation
 in following ways:
@@ -30,187 +30,199 @@ in following ways:
 -  **Unstable**: do not use for other then testing purposes, known
    severe problems, missing implementation of some core parts
 
+.. role:: statusok
+.. role:: statusmok
+.. role:: statusunst
+
 .. list-table::
    :header-rows: 1
 
    * - Feature
      - Stability
      - Performance
      - Notes
-   * - discard (synchronous)
-     - OK
+   * - :doc:`discard (synchronous)<Trim>`
+     - :statusok:`OK`
      -
      - mounted with `-o discard` (has performance implications), also see `fstrim`
-   * - discard (asynchronous)
-     - OK
+   * - :doc:`discard (asynchronous)<Trim>`
+     - :statusok:`OK`
      -
-     - mounted with `-o discard=async` (improved performance)"
+     - mounted with `-o discard=async` (improved performance)
    * - Autodefrag
-     - OK
+     - :statusok:`OK`
      -
      -
-   * - Defrag
-     - mostly OK
+   * - :doc:`Defrag<Defragmentation>`
+     - :statusmok:`mostly OK`
      -
      - extents get unshared (see below)
-   * - Compression
-     - OK (4.14)
+   * - :doc:`Compression<Compression>`
+     - :statusok:`OK`
      -
      -
-   * - Out-of-band dedupe
-     - OK
-     - mostly OK
+   * - :doc:`Out-of-band dedupe<Deduplication>`
+     - :statusok:`OK`
+     - :statusmok:`mostly OK`
      - (reflink), heavily referenced extents have a noticeable performance hit (see below)
-   * - File range cloning
-     - OK
-     - mostly OK
+   * - :doc:`File range cloning<Reflink>`
+     - :statusok:`OK`
+     - :statusmok:`mostly OK`
      - (reflink), heavily referenced extents have a noticeable performance hit (see below)
-   * - More checksumming algorithms
-     - OK
+   * - :doc:`More checksumming algorithms<Checksumming>`
+     - :statusok:`OK`
      - OK
      -
-   * - Auto-repair
-     - OK
+   * - :doc:`Auto-repair<Auto-repair>`
+     - :statusok:`OK`
      - OK
      - automatically repair from a correct spare copy if possible (DUP, RAID1, RAID10, RAID56)
-   * - Scrub
-     - OK
+   * - :doc:`Scrub<Scrub>`
+     - :statusok:`OK`
      - OK
      -
    * - Scrub + RAID56
-     - mostly OK
+     - :statusmok:`mostly OK`
      - mostly OK
      -
    * - nodatacow
-     - OK
+     - :statusok:`OK`
      - OK
      -
-   * - Device replace
-     - mostly OK
+   * - :doc:`Device replace<Volume-management>`
+     - :statusmok:`mostly OK`
      - mostly OK
      - (see below)
    * - Degraded mount
-     - OK (4.14)
+     - :statusok:`OK`
      - n/a
      -
-   * - Single (block group profile)
-     - OK
+   * - :ref:`Single (block group profile)<mkfs-section-profiles>`
+     - :statusok:`OK`
      - OK
      -
-   * - DUP (block group profile)
-     - OK
+   * - :ref:`DUP (block group profile)<mkfs-section-profiles>`
+     - :statusok:`OK`
      - OK
      -
-   * - RAID0
-     - OK
+   * - :ref:`RAID0<mkfs-section-profiles>`
+     - :statusok:`OK`
      - OK
      -
-   * - RAID1
-     - OK
+   * - :ref:`RAID1<mkfs-section-profiles>`
+     - :statusok:`OK`
      - mostly OK
      - reading from mirrors in parallel can be optimized further (see below)
-   * - RAID1C3
-     - OK
+   * - :ref:`RAID1C3<mkfs-section-profiles>`
+     - :statusok:`OK`
      - mostly OK
      - reading from mirrors in parallel can be optimized further (see below)
-   * - RAID1C4
-     - OK
+   * - :ref:`RAID1C4<mkfs-section-profiles>`
+     - :statusok:`OK`
      - mostly OK
      - reading from mirrors in parallel can be optimized further (see below)
-   * - RAID10
-     - OK
+   * - :ref:`RAID10<mkfs-section-profiles>`
+     - :statusok:`OK`
      - mostly OK
      - reading from mirrors in parallel can be optimized further (see below)
-   * - RAID56
-     - unstable
+   * - :ref:`RAID56<mkfs-section-profiles>`
+     - :statusunst:`unstable`
      - n/a
      - (see below)
    * - Mixed block groups
-     - OK
+     - :statusok:`OK`
      - OK
      -
-   * - Filesystem resize
-     - OK
+   * - :doc:`Filesystem resize<Resize>`
+     - :statusok:`OK`
      - OK
      - shrink, grow
-   * - Balance
-     - OK
+   * - :doc:`Balance<Balance>`
+     - :statusok:`OK`
      - OK
      - balance + qgroups can be slow when there are many snapshots
    * - Offline UUID change
-     - OK
+     - :statusok:`OK`
      - OK
      -
    * - Metadata UUID change
-     - OK
+     - :statusok:`OK`
      - OK
      -
-   * - Subvolumes, snapshots
-     - OK
+   * - :doc:`Subvolumes, snapshots<Subvolumes>`
+     - :statusok:`OK`
      - OK
      -
-   * - Send
-     - OK
+   * - :doc:`Send<Send-receive>`
+     - :statusok:`OK`
      - OK
      -
-   * - Receive
-     - OK
+   * - :doc:`Receive<Send-receive>`
+     - :statusok:`OK`
      - OK
      -
-   * - Seeding
-     - OK
+   * - :doc:`Seeding<Seeding-device>`
+     - :statusok:`OK`
      - OK
      -
-   * - Quotas, qgroups
-     - mostly OK
+   * - :doc:`Quotas, qgroups<Qgroups>`
+     - :statusmok:`mostly OK`
      - mostly OK
      - qgroups with many snapshots slows down balance
-   * - Swapfile
-     - OK
+   * - :doc:`Swapfile<Swapfile>`
+     - :statusok:`OK`
      - n/a
      - with some limitations
-   * - NFS
-     - OK
+   * - :ref:`NFS<interop-nfs>`
+     - :statusok:`OK`
      - OK
      -
-   * - cgroups
-     - OK
+   * - :ref:`cgroups<interop-cgroups>`
+     - :statusok:`OK`
      - OK
      - IO controller
-   * - Samba
-     - OK
+   * - :ref:`Samba<interop-samba>`
+     - :statusok:`OK`
      - OK
      - compression, server-side copies, snapshots
-   * - io_uring
-     - OK
+   * - :ref:`io_uring<interop-io-uring>`
+     - :statusok:`OK`
      - OK
      -
-   * - fsverity
-     - OK
+   * - :ref:`fsverity<interop-fsverity>`
+     - :statusok:`OK`
      - OK
      -
-   * - idmapped mount
-     - OK
+   * - :ref:`idmapped mount<interop-idmapped>`
+     - :statusok:`OK`
      - OK
      -
-   * - Free space tree
-     - OK (4.9)
+   * - :ref:`Free space tree<mkfs-feature-free-space-tree>`
+     - :statusok:`OK`
      -
      -
-   * - no-holes
-     - OK
-     - OK
+   * - Block group tree
+     - :statusok:`OK`
      -
-   * - skinny-metadata
-     - OK
+     -
+   * - :ref:`no-holes<mkfs-feature-no-holes>`
+     - :statusok:`OK`
      - OK
      -
-   * - extended-refs
+   * - :ref:`skinny-metadata<mkfs-feature-skinny-metadata>`
+     - :statusok:`OK`
      - OK
+     -
+   * - :ref:`extended-refs<mkfs-feature-extended-refs>`
+     - :statusok:`OK`
      - OK
      -
-   * - zoned mode
+   * - :doc:`Subpage block size<Subpage>`
+     - :statusmok:`mostly OK`
      - mostly OK
+     -
+   * - :doc:`Zoned mode<Zoned-mode>`
+     - :statusmok:`mostly OK`
      - mostly OK
      - there are known bugs, use only for testing
 
@@ -223,6 +235,70 @@ Please open an issue if:
 -  a reference could be enhanced by an actual link to documentation
    (wiki, manual pages)
 
+Subpage block size
+------------------
+
+Most commonly used page sizes are 4KiB, 16KiB and 64KiB. All combinations with
+a 4KiB sector size filesystems are supported. Some features are not compatible
+with subpage or require another feature to work:
+
+.. list-table::
+   :header-rows: 1
+
+   * - Feature
+     - Status
+     - Notes
+   * - Inline files
+     - unsupported
+     - The max_inline mount option value is ignored
+   * - Free space cache v1
+     - unsupported
+     - Free space tree is mandatory
+   * - Compression
+     - partial support
+     - Only page-aligned ranges can be compressed
+
+
+Zoned mode
+----------
+
+.. list-table::
+   :header-rows: 1
+
+   * - Feature
+     - Status
+     - Notes
+   * - Boot
+     - incompatible
+     - The blocks where partition table is stored is used for super block
+   * - Mixed block groups
+     - incompatible
+     - Interleaving data and metadata would lead to out of order write
+   * - NODATACOW
+     - incompatible
+     - In-place overwrite
+   * - fallocate
+     - incompatible
+     - Preallocation of blocks would require an out of order write
+   * - Free space cache v1
+     - incompatible
+     - Cache data are updated in a NODATACOW-way
+   * - Free space tree
+     - supported
+     -
+   * - fstrim
+     - not implemented
+     - This would require free space v1
+   * - single profile
+     - supported
+     - Both data and metadata
+   * - DUP profile
+     - partial support
+     - Only for metadata
+   * - RAID (all)
+     - not implemented
+     - This requires raid-stripe-tree feature which is still work in progress
+
 
 Details that do not fit the table
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -275,10 +351,10 @@ continue to be mountable and usable by newer kernels.
 The core of the on-disk format that comprises building blocks of the
 filesystem:
 
--  layout of the main data structures, eg. superblock, b-tree nodes,
+-  layout of the main data structures, e.g. superblock, b-tree nodes,
    b-tree keys, block headers
 -  the COW mechanism, based on the original design of Ohad Rodeh's paper
-   "Shadowing and clones"
+   "B-trees, Shadowing and Clones" (http://sylab-srv.cs.fiu.edu/lib/exe/fetch.php?media=paperclub:shadow_btree.pdf)
 
 Newly introduced features build on top of the above and could add
 specific structures. If a backward compatibility is not possible to

Documentation/Subpage.rst

@@ -10,54 +10,39 @@ pages, like 64KiB on 64bit ARM or PowerPC. This means filesystems created
 with 64KiB sector size cannot be mounted on a system with 4KiB page size.
 
 While with subpage support, systems with 64KiB page size can create (still needs
-"-s 4k" option for :command:`mkfs.btrfs`) and mount filesystems with 4KiB sectorsize,
-allowing us to push 4KiB sectorsize as default sectorsize for all platforms in the
-near future.
+"-s 4k" option for :command:`mkfs.btrfs`) and mount filesystems with 4KiB sectorsize.
 
 Requirements, limitations
 -------------------------
 
 The initial subpage support has been added in v5.15, although it's still
-considered as experimental at the time of writing (v5.18), most features are
-already working without problems.
+considered as experimental, most features are already working without problems.
 
 End users can mount filesystems with 4KiB sectorsize and do their usual
 workload, while should not notice any obvious change, as long as the initial
 mount succeeded (there are cases a mount will be rejected though).
 
 The following features has some limitations for subpage:
 
-- RAID56 support
-  This support is already queued for v5.19 cycle.
-  Any fs with RAID56 chunks will be rejected at mount time for now.
+- Supported page sizes: 4KiB, 8KiB, 16KiB, 32KiB, 64KiB
 
-- Support for page size other than 64KiB
-  The support for other page sizes (16KiB, 32KiB and more) are already queued
-  for v5.19 cycle.
-  Initially the subpage support is only for 64KiB support, but the design makes
-  it pretty easy to enable support for other page sizes.
+- Supported filesystem sector sizes on a given host are exported in
+  :file:`/sys/fs/btrfs/features/supported_sectorsizes`
 
-- No inline extent creation
-  This is an artificial limit, to prevent mixed inline and regular extents.
+- No inline extents
 
-  It's possible to create mixed inline and regular extents even with
-  non-subpage mount for certain corner cases, it's way easier to create such
-  mixed extents for subpage.
+  This is an artificial limitation, to prevent mixed inline and regular extents.
 
   Thus max_inline mount option will be silently ignored for subpage mounts,
   and it always acts as "max_inline=0".
 
-- Compression write is limited to page aligned ranges
-  Compression write for subpage is introduced in v5.16, with the limitation
-  that only page aligned range can be compressed.
-  This limitation is due to how btrfs handles delayed allocation.
+- Compression writes are limited to page aligned ranges
+
+  Compression write for subpage has been introduced in v5.16, with the
+  limitation that only page aligned range can be compressed.  This limitation
+  is due to how btrfs handles delayed allocation.
 
 - No support for v1 space cache
-  V1 space cache is considered deprecated, and we're defaulting to v2 cache
-  in btrfs-progs already.
-  The old v1 cache has quite some hard coded page size usage, and consider it
-  is already deprecated, we force v2 cache for subpage.
-
-- Slightly higher memory usage for scrub
-  This is due to how we allocate pages for scrub, and will be fixed in the coming
-  releases soon.
+
+  The old v1 cache has quite some hard coded page size usage, and considering
+  it already deprecated, we force v2 cache for subpage.

Documentation/_static/style.css

@@ -1,3 +1,21 @@
 .wy-nav-content {
     max-width: 1200px !important;
 }
+
+.green {
+	color: green;
+}
+
+.statusok {
+	color: green;
+}
+
+.statusmok {
+	/* color: orange; */
+	/* color: #ffae42; */
+	color: darkorange;
+}
+
+.statusunst {
+	color: red;
+}

Documentation/btrfs-device.rst

@@ -14,7 +14,7 @@ The :command:`btrfs device` command group is used to manage devices of the btrfs
 DEVICE MANAGEMENT
 -----------------
 
-.. include ch-volume-management-intro.rst
+.. include:: ch-volume-management-intro.rst
 
 SUBCOMMAND
 ----------

Documentation/btrfs-ioctl.rst

@@ -1,4 +1,4 @@
-btrfs-ioctl(3)
+btrfs-ioctl(2)
 ==============
 
 NAME

Documentation/ch-flexibility.rst

@@ -4,12 +4,16 @@ or enabling some features on-the-fly.
 
 * **dynamic inode creation** -- there's no fixed space or tables for tracking
   inodes so the number of inodes that can be created is bounded by the metadata
-  space and it's utilization
+  space and its utilization
 
 * **block group profile change on-the-fly** -- the block group profiles can be
   changed on a mounted filesystem by running the balance operation and
-  specifying the conversion filters
+  specifying the conversion filters (see :doc:`balance<Balance>`.)
 
 * **resize** -- the space occupied by the filesystem on each device can be
   resized up (grow) or down (shrink) as long as the amount of data can be still
   contained on the device
+
+* **device management** -- devices can be added, removed or replaced without
+  requiring recreating the filesystem (mkfs), new redundancy options available
+  on more devices can be also utilized by rebalancing

Documentation/ch-zoned-intro.rst

@@ -17,15 +17,16 @@ to set various zone sizes.
 Requirements, limitations
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
-* all devices must have the same zone size
-* maximum zone size is 8GiB
-* minimum zone size is 4MiB
-* mixing zoned and non-zoned devices is possible, the zone writes are emulated,
-  but this is namely for testing
-* the super block is handled in a special way and is at different locations than on a non-zoned filesystem:
-   * primary: 0B (and the next two zones)
-   * secondary: 512GiB (and the next two zones)
-   * tertiary: 4TiB (4096GiB, and the next two zones)
+*  all devices must have the same zone size
+*  maximum zone size is 8GiB
+*  minimum zone size is 4MiB
+*  mixing zoned and non-zoned devices is possible, the zone writes are emulated,
+   but this is namely for testing
+*  the super block is handled in a special way and is at different locations than on a non-zoned filesystem:
+
+   *  primary: 0B (and the next two zones)
+   *  secondary: 512GiB (and the next two zones)
+   *  tertiary: 4TiB (4096GiB, and the next two zones)
 
 Incompatible features
 ^^^^^^^^^^^^^^^^^^^^^

Documentation/mkfs.btrfs.rst

@@ -241,6 +241,8 @@ mixed-bg
 
         mixed data and metadata block groups, also set by option *--mixed*
 
+.. _mkfs-feature-extended-refs:
+
 extref
         (default since btrfs-progs 3.12, kernel support since 3.7)
 
@@ -254,11 +256,15 @@ raid56
         extended format for RAID5/6, also enabled if RAID5 or RAID6 block groups
         are selected
 
+.. _mkfs-feature-skinny-metadata:
+
 skinny-metadata
         (default since btrfs-progs 3.18, kernel support since 3.10)
 
         reduced-size metadata for extent references, saves a few percent of metadata
 
+.. _mkfs-feature-no-holes:
+
 no-holes
         (default since btrfs-progs 5.15, kernel support since 3.14)
 
@@ -278,6 +284,8 @@ quota
         Enable quota support (qgroups). The qgroup accounting will be consistent,
         can be used together with *--rootdir*.  See also :doc:`btrfs-quota(8)<btrfs-quota>`.
 
+.. _mkfs-feature-free-space-tree:
+
 free-space-tree
         (default since btrfs-progs 5.15, kernel support since 4.5)
 
@@ -289,6 +297,8 @@ block-group-tree
 
         Enable the block group tree to greatly reduce mount time for large filesystems.
 
+.. _mkfs-section-profiles:
+
 BLOCK GROUPS, CHUNKS, RAID
 --------------------------