Skip to content

Commit 5ae653a

Browse files
fzakariafzakaria
committed
Addressed PR feedback
* moved spec to it's own file * included spec into the markdown * mentioned sorted order of directory entries * moved nix-archive.md to it's own directory
1 parent 2612868 commit 5ae653a

File tree

5 files changed

+61
-60
lines changed

5 files changed

+61
-60
lines changed

‎doc/manual/source/SUMMARY.md.in‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@
122122
- [Derivation](protocols/json/derivation.md)
123123
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
124124
- [Store Path Specification](protocols/store-path.md)
125-
- [Nix Archive (NAR) Format](protocols/nix-archive.md)
125+
- [Nix Archive (NAR) Format](protocols/nix-archive/index.md)
126126
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
127127
- [C API](c-api.md)
128128
- [Glossary](glossary.md)

‎doc/manual/source/protocols/nix-archive/index.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Nix Archive (NAR) format
2+
3+
This is the complete specification of the [Nix Archive] format.
4+
The Nix Archive format closely follows the abstract specification of a [file system object] tree,
5+
because it is designed to serialize exactly that data structure.
6+
7+
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#nix-archive
8+
[file system object]: @docroot@/store/file-system-object.md
9+
10+
The format of this specification is close to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form), with the exception of the `str(..)` function / parameterized rule, which length-prefixes and pads strings.
11+
This makes the resulting binary format easier to parse.
12+
13+
Regular users do *not* need to know this information.
14+
But for those interested in exactly how Nix works, e.g. if they are reimplementing it, this information can be useful.
15+
16+
```ebnf
17+
nar = str("nix-archive-1"), nar-obj;
18+
19+
nar-obj = str("("), nar-obj-inner, str(")");
20+
21+
nar-obj-inner
22+
= str("type"), str("regular") regular
23+
| str("type"), str("symlink") symlink
24+
| str("type"), str("directory") directory
25+
;
26+
27+
regular = [ str("executable") ], str("contents"), str(contents);
28+
29+
symlink = str("target"), str(target);
30+
31+
(* side condition: directory entries must be ordered by their names *)
32+
directory = { directory-entry };
33+
34+
directory-entry = str("entry"), str("("), str("name"), str(name), str("node"), nar-obj, str(")");
35+
```
36+
37+
The `str` function / parameterized rule is defined as follows:
38+
39+
- `str(s)` = `int(|s|), pad(s);`
40+
41+
- `int(n)` = the 64-bit little endian representation of the number `n`
42+
43+
- `pad(s)` = the byte sequence `s`, padded with 0s to a multiple of 8 byte
44+
45+
## Kaitai Struct Specification
46+
47+
The Nix Archive (NAR) format is also formally described using [Kaitai Struct](https://kaitai.io/), an Interface Description Language (IDL) for defining binary data structures.
48+
49+
> Kaitai Struct provides a language-agnostic, machine-readable specification that can be compiled into parsers for various programming languages (e.g., C++, Python, Java, Rust).
50+
51+
```yaml
52+
{{#include nar.ksy}}
53+
```
54+
55+
The source of the spec can be found [here](https://github.com/nixos/nix/blob/master/src/nix-manual/source/protocols/nix-archive/nar.ksy). Contributions and improvements to the spec are welcomed.

‎doc/manual/source/protocols/nix-archive.md‎ renamed to ‎doc/manual/source/protocols/nix-archive/nar.ksy‎

Lines changed: 3 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,3 @@
1-
# Nix Archive (NAR) format
2-
3-
This is the complete specification of the [Nix Archive] format.
4-
The Nix Archive format closely follows the abstract specification of a [file system object] tree,
5-
because it is designed to serialize exactly that data structure.
6-
7-
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#nix-archive
8-
[file system object]: @docroot@/store/file-system-object.md
9-
10-
The format of this specification is close to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form), with the exception of the `str(..)` function / parameterized rule, which length-prefixes and pads strings.
11-
This makes the resulting binary format easier to parse.
12-
13-
Regular users do *not* need to know this information.
14-
But for those interested in exactly how Nix works, e.g. if they are reimplementing it, this information can be useful.
15-
16-
```ebnf
17-
nar = str("nix-archive-1"), nar-obj;
18-
19-
nar-obj = str("("), nar-obj-inner, str(")");
20-
21-
nar-obj-inner
22-
= str("type"), str("regular") regular
23-
| str("type"), str("symlink") symlink
24-
| str("type"), str("directory") directory
25-
;
26-
27-
regular = [ str("executable") ], str("contents"), str(contents);
28-
29-
symlink = str("target"), str(target);
30-
31-
(* side condition: directory entries must be ordered by their names *)
32-
directory = { directory-entry };
33-
34-
directory-entry = str("entry"), str("("), str("name"), str(name), str("node"), nar-obj, str(")");
35-
```
36-
37-
The `str` function / parameterized rule is defined as follows:
38-
39-
- `str(s)` = `int(|s|), pad(s);`
40-
41-
- `int(n)` = the 64-bit little endian representation of the number `n`
42-
43-
- `pad(s)` = the byte sequence `s`, padded with 0s to a multiple of 8 byte
44-
45-
## Kaitai Struct Specification
46-
47-
The Nix Archive (NAR) format is also formally described using [Kaitai Struct](https://kaitai.io/), an Interface Description Language (IDL) for defining binary data structures.
48-
49-
> Kaitai Struct provides a language-agnostic, machine-readable specification that can be compiled into parsers for various programming languages (e.g., C++, Python, Java, Rust).
50-
51-
```yaml
521
meta:
532
id: nix_nar
543
title: Nix Archive (NAR)
@@ -115,7 +64,7 @@ types:
11564
doc: "Must be ')', a token ending the node definition."
11665

11766
type_directory:
118-
doc: "A directory node, containing a list of entries."
67+
doc: "A directory node, containing a list of entries. Entries must be ordered by their names."
11968
seq:
12069
- id: entries
12170
type: dir_entry
@@ -196,7 +145,7 @@ types:
196145
seq:
197146
- id: len_contents
198147
type: u8
199-
# This relies on the property of instances that they are lazily evaluated and cached.
148+
# # This relies on the property of instances that they are lazily evaluated and cached.
200149
- size: 0
201150
if: nar_offset < 0
202151
- id: contents
@@ -217,7 +166,4 @@ types:
217166
expr: _.body == 'target'
218167
- id: target_val
219168
type: padded_str
220-
doc: "The destination path of the symlink."
221-
```
222-
223-
The source of the spec can be found [here](https://github.com/fzakaria/nix-nar-kaitai-spec/blob/main/NAR.ksy). Contributions and improvements to the spec are welcomed.
169+
doc: "The destination path of the symlink."

‎doc/manual/source/store/file-system-object/content-address.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ be many different serialisations.
4646
For these reasons, Nix has its very own archive format—the Nix Archive (NAR) format,
4747
which is carefully designed to avoid the problems described above.
4848

49-
The exact specification of the Nix Archive format is in [specified here](../../protocols/nix-archive.md).
49+
The exact specification of the Nix Archive format is in [specified here](../../protocols/nix-archive/index.md).
5050

5151
## Content addressing File System Objects beyond a single serialisation pass
5252

‎src/nix/nar.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ R""(
88
# File format
99

1010
For the definition of the Nix Archive file format, see
11-
[within the protocols chapter](@docroot@/protocols/nix-archive.md)
11+
[within the protocols chapter](@docroot@/protocols/nix-archive/index.md)
1212
of the manual.
1313

1414
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive

0 commit comments

Comments
 (0)