forked from hercules-390/html
-
Notifications
You must be signed in to change notification settings - Fork 3
/
Copy pathcckddasd.html
1774 lines (1530 loc) · 73 KB
/
cckddasd.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 3.0//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><TITLE>
Hercules: Compressed Dasd Emulation
</TITLE>
<LINK REL=STYLESHEET TYPE="text/css" HREF="hercules.css">
<link rel="shortcut icon" href="images/favicon.ico" />
<link rel="icon" href="images/favicon.ico" />
</HEAD><BODY BGCOLOR="#ffffcc" TEXT="#000000" LINK="#0000A0" VLINK="#008040" ALINK="#000000">
<h1>Compressed Dasd Emulation</h1>
<hr noshade>
<h2>Contents</h2>
<ul>
<li><a href="#introduction"> Introduction </a>
<li><a href="#shadowfiles"> Shadow Files </a>
<ul>
<li><a href="#advantage"> The Advantage of Shadow Files </a>
<li><a href="#sfcommands"> Shadow File Commands </a>
</ul>
<li><a href="#filestructure"> Compressed DASD File Structure </a>
<li><a href="#howitworks"> How It Works </a>
<li><a href="#cckdcommand"> Runtime Tuning Options </a>
<li><a href="#utilities"> Offline Utilities </a>
</ul>
<a name="introduction">
<hr noshade>
<h2>Introduction</h2>
<p>
Using compressed DASD files can significantly reduce the host system
file space required for emulated DASD files and possibly provide a
performance gain as well because less physical I/O occurs.
<p>
Using compressed DASD files also has other advantages too, such as
allowing you to use <a href="#shadowfiles">shadow files</a>.
<p>
Both <b>CKD</b> (Count-Key-Data) and <b>FBA</b> (Fixed-Block-Architecture)
emulation files can be compressed.
<p>
In regular (or uncompressed) files, each CKD track or FBA block occupies
a specific spot in the emulation file. The offset of the track or block
in the file can be directly calculated knowing the track or block number
and the maximum size of the track or block. In compressed files, each
track image or group of blocks may be compressed by
<a href="http://www.zlib.net/"><b>zlib</b></a> or
<a href="http://www.bzip.org/"><b>bzip2</b></a>, and only
occupies the space neccessary for the compressed data. The offset of a compressed
track or block is obtained by performing a two-table lookup. The lookup
tables themselves reside in the emulation file.
<p>
Because FBA blocks are 512 bytes in length, and that being a rather small
number, FBA blocks are thus grouped into <b>block groups</b>. Each block
group contains 120 FBA blocks (60K).
<p>
Whenever a track or block group is written to a compressed file,
it is written either to an existing free space within the file or at
the end of the file, the lookup tables are then updated, and then the
space the track or block group previously occupied is freed. The location
of a track or block group in the file can change many times.
<p>
In the event of a failure (for example, Hercules crash,
operating system crash or power failure), the compressed emulation file
on the host's physical disk may be out of sync if the host operating
system defers physical writes to the file system containing the emulation
file. However, several methods have evolved to greatly reduce the chance
of this occurring as well as the amount of data loss that can occur after
these kinds of events, so the chance of actual data loss is relatively small.
<p>
A compressed dasd image file may occupy only 20% of the disk space required
by an uncompressed file. In other words, you may be able to have 5 times
more emulated volumes on your system by using compressed DASD files than by
using uncompressed dasds. Compressed files are slightly more sensitive to
failures and data corruption however, but due to their design and careful
implementation, the chance of this actually occuring is relatively small.
<p>
Compressed DASD images are created via the
<a href="hercload.html#dasdinit">dasdinit64</a> utility.
<p>
<a name="shadowfiles"></a>
<hr noshade>
<p><h2>Shadow Files</h2>
<p>
Shadow files are only supported for compressed dasd images.
<p>
A compressed CKD or FBA dasd can have more than one physical file.
These additional files are called <em>shadow files</em>.
Shadow files are designed to be used as a kind of <i>"snapshot"</i>, where
a new shadow file can be created on demand.
An emulated dasd is represented by a <em>base</em> file and 0 or more
shadow files. All files are opened <em>read-only</em>
except for the <em>current</em> file, which is opened <em>read-write</em>.
<p>
Shadow files are specified by the <b>sf=</b><i>shadow-file-name</i> parameter
on the device statement for the compressed DASD device.
<p class="note">
Please note that the specified shadow filename does <u>not</u> have to
actually exist. The <i>shadow-file-name</i> operand of the <b><i>sf=</i></b>
parameter is simply a <b>filename <i><u>template</u></i></b> that will
be used to name the shadow file whenever a shadow file is to be created.
Shadow files do not actually get created until you specifically create
them via the <code>sf+xxxx</code> (or <code>sf+*</code>) command, or
automatically <b>(<a href="#autoshadow"><code>*</code></a>)</b> by marking
either the base dasd image file or an existing shadow file as read-only.
Please refer to the discussion of the <b>sf</b> command in the next section
further below for more information.
<p>
The <b>shadow file name</b> should have spot where the shadow file number will be set.
This is either the character immediately preceding the filename extension
or the last character of the file name itself if there is no extension.
<p>
For example, the following device statement:<br>
<pre>
0001 2311 AAAAAA.model-x.ext sf=AAAAAA_Shadow_0.model-x.ext
</pre>
will cause a shadow file with the following name to be created:
<pre>
AAAAAA_Shadow_0.model-<b><i>1</i></b>.ext
</pre>
whereas the following <i>slightly different device statement</i>:
<pre>
0002 2311 BBBBBB.model-x.ext sf=BBBBBB.model-x_Shadow_0.ext
</pre>
will cause a shadow file with the following name to be created:
<pre>
BBBBBB.model-x_Shadow_<b><i>1</i></b>.ext
</pre>
<p>
<u>Notice the placement of the shadow file number</u> in the above two examples:
it is always immediately before the <i>last</i> period of the filename.
In earlier versions of Hercules the shadow file number was
erroneously placed immediately before the <i>first</i> period of the
filename instead of the last period. In newer versions of Hyperion
this has been corrected to place the number immediately before the
<i>last</i> period, as it was originally intended.
<p>
<a name="advantage">
<p><h3>The Advantage of Shadow Files</h3>
<p>
A shadow file contains all the changes made to the emulated dasd
since it was created, until the next shadow file is created. The moment
of the shadow file's creation can be thought of as a <em>"snapshot"</em>
of the current emulated dasd as it existed at that point in time, because
if the shadow file is later removed, then the emulated dasd reverts back
to the state it was in at the moment the snapshot was taken.
<p>
Using shadow files, you can keep the base file on a read-only device
such as cdrom, or change the base file attributes to read-only,
ensuring that this file can never be corrupted.
<p>
With the use of shadow files you can reconfigure your guest
without worrying about whether the new settings might render
your guest unusable. Simply create a new shadow file <i><u>before</u></i>
you do your reconfiguration. If something goes wrong, simply exit Hercules
and then delete that set of shadow files. You'll be right back to where
you were <i>before</i> you made your changes! Then you can simply try again.
Once you are satisfied that your changes are good, you can then do a
"<code>sf-*</code>" backwards merge (see further below) to "commit" your changes.
<p>
There can be up to 8 shadow files in use at any time for an
emulated dasd device. The base file is designated file<b>[0]</b> and
the shadow files are file<b>[1]</b> to file<b>[8]</b>.
The <em>highest</em> numbered file in use at a given time is the <em>current</em>
file, where all writes will occur. Track reads start with the <em>current</em>
file and proceeds downward until a file is found that actually contains the track
image.
<p>
<a name="sfcommands">
<p><h3>Shadow File Commands</h3>
<p>
Hercules console commands are provided to add a new shadow file, remove
the current shadow file (with or without backward merge), compress the
current shadow file, and display the shadow file status and statistics.
<p>
<p class="note">
<i>
<b>Please Note</b> that, except for the <b>sfd</b> command, none
of the below commands should ever be run immediately before, during,
or after a guest has been IPLed without Hercules having been shutdown
both before the commands are issued and then again afterwards!
</i>
<p class="note">
<i>
<b>Note:</b> It is not advisable to merge shadow files back into base images.
When merging shadow files, it is only recommended to merge one set of shadow
files back into the previous set of shadow files. When you merge a shadow
file back into the base image, you might see some error/warning message being
issued as a result, which should be considered an unpreventable side effect
of such a merge, and are completely benign. No file damage has actually occurred.
</i>
<p>
<blockquote>
<table width=90%>
<tr><td colspan="4"> </td></tr>
<tr><td align="left"><b>sfd</b></td>
<td align="left" colspan="2">unit</td>
<td align="left">Display shadow file status and statistics</td>
</tr>
<tr><td align="left"><b>sfc</b></td>
<td align="left" colspan="2">unit</td>
<td align="left">Compress the current file</td>
</tr>
<tr><td align="left"><b>sf+</b></td>
<td align="left" colspan="2">unit</td>
<td align="left">Create a new shadow file <b>(<a href="#autoshadow"><code>*</code></a>)</b></td>
</tr>
<tr><td align="left" valign="top"><b>sf-</b></td>
<td align="left" valign="top">unit</td>
<td align="left" valign="top"<b><u>merge</u> </b><br>nomerge <br>force </td>
<td align="left" valign="top"><p>Remove a shadow file.<p>If <em>merge</em> is
specified or defaulted, then the contents of the current
file is merged into the previous file, the current file
is removed, and the previous file becomes the current file.
The previous file must be able to be opened read-write.
If <em>nomerge</em> is specified then the contents of the
current file is discarded and the previous file becomes the
current file.
<p>
However, if the previous file is read-only,
then a new shadow file is created (re-added) and that becomes
the current file.
The <em>force</em> option is required when doing a merge to
the base file and the base file is read-only because the
'ro' option was specified on the configuration file device statement.
Note that it is not advisable to merge shadow files into base images.
Shadow files should only be merged into previous shadow file sets
or discarded altogether. The whole purpose of shadow files is to
avoid modifying base images. Refer to the <a href="#advantage">
Advantage of Shadow Files</a> section further above.
</td>
</tr>
<tr><td colspan="4"> </td></tr>
<tr><td align="left" valign="top"><b>sfk</b></td>
<td align="left" valign="top">unit</td>
<td align="left" valign="top"<i>level</i></td>
<td align="left" valign="top">Perform the <b>chkdsk</b> function on the current file.<br>
Level is a number from -1 ... 4, with the default level being <b>2</b>:<p>
-1 devhdr, cdevhdr, L1 table. <br>
0 devhdr, cdevhdr, L1 table, L2 tables. <br>
1 devhdr, cdevhdr, L1 table, L2 tables, free spaces. <br>
<b>2</b> devhdr, cdevhdr, L1 table, L2 tables, free spaces, trkhdrs. <i>(default)</i> <br>
3 devhdr, cdevhdr, L1 table, L2 tables, free spaces, trkimgs. <br>
4 devhdr, cdevhdr, build everything else from recovery.
</td>
</tr>
</table>
<p>
<u>Note</u>: You can use '<b><code>*</code></b>' in place of 'unit'
in any of the above commands to apply the command to <i>all</i> compressed dasd.
(<code>sf+*</code>,<code>sf-*</code>,<code>sfc*</code>,<code>sfk*</code>,<code>sfd*</code>)
<p>
<a name="autoshadow"></a>
<b>(<code>*</code>)</b> In addition to the <b>sf+</b> command, shadow files can also be created
<i><u>automatically</u></i> whenever the base dasd image and all of its existing shadow files
(if any) are marked as read-only. When Hercules opens a compressed dasd image file with a <code>
sf=</code> option, it automatically looks for and opens whatever existing shadow files
there may be, with the highest numbered shadow file being the one where all writes
will occur. If the highest numbered shadow file is also marked as a read-only file too,
then a new shadow file will be automatically created and used as the current shadow file
where all writes will occur.
</blockquote>
<!-- ---------------------------------------------------------------------------------- -->
<a name="filestructure"></a>
<hr noshade>
<p><h2>Compressed DASD File Structure</h2>
<p>
A compressed DASD file has 6 types of spaces, a <em><u>device header</u></em>,
a <em><u>compressed device header</u></em>, a <em><u>primary lookup table</u></em>,
<em>secondary lookup tables</em>, track or block group <em>images</em>,
and <em>free spaces</em>. The first 3 types (<i>device header, compressed device
header, primary lookup table</i>) only occur once, in order, at the very beginning
of the file. The rest of the file is occupied by the other 3 space types.
<p>
<p class="note">
<i>
<b>It is also important to note</b> that, except for the actual track or block
group images space types, all <u>numeric</u> fields in each of the above space
types are always kept in <u><b>LITTLE</b></u> endian format and are automatically
converted to host format as needed before being used. This is the complete opposite
of the way emulated guest storage is maintained (which is always in big endian format).
</i>
<p>
<!-- ---------------------------------------------------------------------------------- -->
The very first 512 bytes of a compressed DASD file is the <b>device header</b>.
The device header contains an "eye-catcher" that identifies the file type
(32-bit or 64-bit CKD or FBA and base or shadow). The original CCKD design
used 32-bit file offset values thus limiting file sizes to only 4GB or less.
<p>
The newer 64-bit design however, uses 64-bit offset values, thereby allowing
compressed dasd image files to grow up to the theoretical maximum of 18,000,000
TB (18,000,000,000 GB). The <i>actual</i> maximum file size however, is limited
by both the operating system and the format of whatever file system you are using.
On Windows, using NTFS for example, the maximum file size is limited to only 16
TB (16,000 GB).
<p>
<blockquote><blockquote>
<table>
<tr><th> Device-id <th> Dasd image type </tr>
<tr><td> <td> </tr>
<tr><td> CKD_P370 <td> Normal <i>(uncompressed)</i> CKD dasd image </tr>
<tr><td> CKD_C370 <td> Compressed CKD dasd image </tr>
<tr><td> CKD_S370 <td> Compressed CKD dasd Shadow file </tr>
<tr><td> <center>n/a*</center><td> Normal <i>(uncompressed)</i> FBA<b>**</b> dasd image </tr>
<tr><td> FBA_C370 <td> Compressed FBA dasd image </tr>
<tr><td> FBA_S370 <td> Compressed FBA dasd Shadow file </tr>
<tr><td> <td> </tr>
<tr><td> CKD_P064 <td> Normal <i>(uncompressed)</i> CKD64 dasd image </tr>
<tr><td> CKD_C064 <td> Compressed CKD64 dasd image </tr>
<tr><td> CKD_S064 <td> Compressed CKD64 dasd Shadow file </tr>
<tr><td> <center>n/a*</center><td> Normal <i>(uncompressed)</i> FBA64<b>**</b> dasd image </tr>
<tr><td> FBA_C064 <td> Compressed FBA64 dasd image </tr>
<tr><td> FBA_S064 <td> Compressed FBA64 dasd Shadow file </tr>
</table>
</blockquote>
<i>(*) Normal (uncompressed) FBA/FBA64 dasd image files do not have device headers.
The first 512 bytes of a normal (uncompressed) FBA/FBA64 dasd image file
is the actual first sector of the emulated FBA dasd device itself.</i>
<p>
<i>(**) Normal (uncompressed) FBA64 dasd image files
are exactly identical to normal (uncompressed) FBA dasd image files.
There is absolutely no distinction between the two types. Their formats
are exactly identical.</i>
</blockquote>
<p><br>
The device type and file size information is specified in this header
and, except for the eye-catcher (or device-id), this header is identical
for both compressed CCKD/CCKD64 and compressed CFBA/CFBA64 images as well as for
uncompressed CKD/CKD64 images. As mentioned just above however, it is <i>not</i>
used for uncompressed FBA/FBA64 images.
<p>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CKD_DEVHDR</th></tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>"<b>CKD_C370</b>" (device-id or "eye-catcher")</font></td>
<td align="center" colspan="4"><font size=-1>dh_heads</font></td>
<td align="center" colspan="4"><font size=-1>dh_trksize</font></td>
</tr>
<tr><td align="center" colspan="1"><font size=-1>dh_devtyp</font></td>
<td align="center" colspan="1"><font size=-1>dh_fileseq</font></td>
<td align="center" colspan="2"><font size=-1>dh_highcyl</font></td>
<td align="center" colspan="12"><font size=-1>dh_serial</font></td>
</tr>
<tr>
<td align="center" colspan="16" valign="middle">
<br><br><font size=-1>reserved</font><br><br><br></td>
</tr>
</table>
<p>
The <i>dh_heads</i>, <i>dh_trksize</i> and <i>dh_highcyl</i> values, being numeric, are always kept in little endian format.</i>
<p>
<!-- ---------------------------------------------------------------------------------- -->
<a name="cdevhdr">
<br><br>
The next 512 bytes contains the <b>compressed device header</b>.
<p>
The compressed device header contains file usage information,
such as the amount of free space in the file.
<p>
Note that even though this control block is labeled "CCKD_DEVHDR"
(or "CCKD64_DEVHDR" for 64-bit images), it actually applies to
(is used for) compressed FBA images (CFBA/CFBA64) too, as well as
for compressed CKD images (CCKD/CCKD64).
<p>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD_DEVHDR</th></tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="3"><font size=-1>cdh_vrm</font></td>
<td align="center" colspan="1"><font size=-1>cdh_opts</font></td>
<td align="center" colspan="4"><font size=-1>num_L1tab</font></td>
<td align="center" colspan="4"><font size=-1>num_L2tab</font></td>
<td align="center" colspan="4"><font size=-1>cdh_size</font></td>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>cdh_used</font></td>
<td align="center" colspan="4"><font size=-1>free_off</font></td>
<td align="center" colspan="4"><font size=-1>free_total</font></td>
<td align="center" colspan="4"><font size=-1>free_largest</font></td>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>free_num</font></td>
<td align="center" colspan="4"><font size=-1>free_imbed</font></td>
<td align="center" colspan="4"><font size=-1>cdh_cyls</font></td>
<td align="center" colspan="1"><font size=-1>cdh_nullfmt</font></td>
<td align="center" colspan="1"><font size=-1>cmp_algo</font></td>
<td align="center" colspan="2"><font size=-1>cmp_parm</font></td>
</tr>
<tr>
<td align="center" colspan="16">
<br><br><font size=-1>reserved</font><br><br><br></td>
</tr>
</table>
<br><br>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD64_DEVHDR</th></tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="3"><font size=-1>cdh_vrm</font></td>
<td align="center" colspan="1"><font size=-1>cdh_opts</font></td>
<td align="center" colspan="4"><font size=-1>num_L1tab</font></td>
<td align="center" colspan="4"><font size=-1>num_L2tab</font></td>
<td align="center" colspan="4"><font size=-1>cdh_cyls</font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>cdh_size</font></td>
<td align="center" colspan="8"><font size=-1>cdh_used</font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>free_off</font></td>
<td align="center" colspan="8"><font size=-1>free_total</font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>free_largest</font></td>
<td align="center" colspan="8"><font size=-1>free_num</font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>free_imbed</font></td>
<td align="center" colspan="1"><font size=-1>cdh_nullfmt</font></td>
<td align="center" colspan="1"><font size=-1>cmp_algo</font></td>
<td align="center" colspan="2"><font size=-1>cmp_parm</font></td>
<td align="center" colspan="4"><font size=-1>reserved</font></td>
</tr>
<tr>
<td align="center" colspan="16">
<br><br><font size=-1>reserved</font><br><br><br></td>
</tr>
</table>
<p>
The <i>num_L1tab</i>, <i>num_L2tab</i>, <i>cdh_cyls</i>, <i>cdh_size</i>, <i>cdh_used</i>,
<i>free_off</i>, <i>free_total</i>, <i>free_largest</i>, <i>free_num</i>, <i>free_imbed</i>,
and <i>cmp_parm</i> values, being numeric, are always kept in little endian format.
<p>
<!-- ---------------------------------------------------------------------------------- -->
<br>
After the compressed device header is the <b>primary lookup table</b>
(also called the Level 1 table or <em>L1tab</em>.)
Each 4 byte unsigned entry in the <i>L1tab</i> (8 byte unsigned entry for CCKD64) contains
the file offset of a <em><u>secondary lookup table</u></em> (or <em>level 2 table</em>
or <em>L2tab</em>).
<p>
The track or block group number being accessed divided by 256 gives the index
into the <i>L1tab</i>. That is, each L1tab entry (<i>CCKD_L1ENT</i> or <i>CCKD64_L1ENT</i>)
represents 256 tracks or block groups. The number of entries in the <i>L1tab</i> is dependent
on the size of the emulated device.
<p>
Because each Level 1 table entry (CCKD_L1ENT or CCKD64_L1ENT) is a numeric value, they are
of course always kept in little endian format.
<p>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD_L1TAB</th></tr>
<tr>
<th width="25%" colspan="4">CCKD_L1ENT</th>
<th width="25%" colspan="4">CCKD_L1ENT</th>
<th width="25%" colspan="4">CCKD_L1ENT</th>
<th width="25%" colspan="4">CCKD_L1ENT</th>
</tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>L1<sub>0</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>1</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>2</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>3</sub></font></td>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>L1<sub>4</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>5</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>6</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>7</sub></font></td>
</tr>
<tr>
<td align="center" colspan="16">
<br><center><b>. . .</b></center><br></td>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>L1<sub>n-4</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>n-3</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>n-2</sub></font></td>
<td align="center" colspan="4"><font size=-1>L1<sub>n-1</sub></font></td>
</tr>
</table>
<br><br>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD64_L1TAB</th></tr>
<tr>
<th width="50%" colspan="8">CCKD64_L1ENT</th>
<th width="50%" colspan="8">CCKD64_L1ENT</th>
</tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L1<sub>0</sub></font></td>
<td align="center" colspan="8"><font size=-1>L1<sub>1</sub></font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L1<sub>2</sub></font></td>
<td align="center" colspan="8"><font size=-1>L1<sub>3</sub></font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L1<sub>4</sub></font></td>
<td align="center" colspan="8"><font size=-1>L1<sub>5</sub></font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L1<sub>6</sub></font></td>
<td align="center" colspan="8"><font size=-1>L1<sub>7</sub></font></td>
</tr>
<tr>
<td align="center" colspan="16">
<br><center><b>. . .</b></center><br></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L1<sub>n-4</sub></font></td>
<td align="center" colspan="8"><font size=-1>L1<sub>n-3</sub></font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L1<sub>n-2</sub></font></td>
<td align="center" colspan="8"><font size=-1>L1<sub>n-1</sub></font></td>
</tr>
</table>
<p>
<!-- ---------------------------------------------------------------------------------- -->
<br><br>
Following the primary lookup table (<em>L1tab</em>),
in no particular order, are:
<b>secondary lookup tables</b> (<em>L2tabs</em>),
<em>track or block group images</em>, and
<em>free spaces</em>.
<p>
Each <b>secondary lookup table</b> (or <em>L2tab</em>),
contains 256 eight byte entries (256 sixteen byte entries for CCKD64).
The entry is indexed by the remainder of the track or block group number divided by 256.
<p>
Each L2 entry contains an unsigned 4 byte <i>offset</i> (unsigned 8 byte <i>offset</i> for CCKD64)
to the <b>track or block group image</b> (<i>L2_trkoff</i> = file offset to where the track image
or block group begins),
an unsigned 2 byte track or block group <i>length</i> (the actual amount of data
contained on the track or in the block group)
and an unsigned 2 byte <i>size</i> of the track or block group image (i.e. how much
space is actually available for use at that particular file offset).
<p>
The length is the amount of available space currently being consumed by the
track or block group image, and the size is the amount of space actually available.
The size of course may be greater than the length to prevent short free spaces
from accumulating in the file.
<p>
Because each field in the Level 2 table (<i>L2_trkoff</i>, <i>L2_len</i> and <i>L2_size</i>)
is a numeric value, they are of course always kept in little endian format.
<p>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD_L2TAB</th></tr>
<tr>
<th width="50%" colspan="8">CCKD_L2ENT</th>
<th width="50%" colspan="8">CCKD_L2ENT</th>
</tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>L2_trkoff<sub>0</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>0</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>0</sub></font></td>
<td align="center" colspan="4"><font size=-1>L2_trkoff<sub>1</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>1</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>1</sub></font></td>
</tr>
<tr>
<td align="center" colspan="16"><font size=-1>
<br><center><b>. . .</b></center><br></td>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>L2_trkoff<sub>254</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>254</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>254</sub></font></td>
<td align="center" colspan="4"><font size=-1>L2_trkoff<sub>255</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>255</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>255</sub></font></td>
</tr>
</table>
<br><br>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD64_L2TAB</th></tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<th width="100%" colspan="16">CCKD64_L2ENT</th>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L2_trkoff<sub>0</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>0</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>0</sub></font></td>
<td align="center" colspan="4"><font size=-1> </font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L2_trkoff<sub>1</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>1</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>1</sub></font></td>
<td align="center" colspan="4"><font size=-1> </font></td>
</tr>
<tr>
<td align="center" colspan="16"><font size=-1>
<br><center><b>. . .</b></center><br></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L2_trkoff<sub>254</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>254</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>254</sub></font></td>
<td align="center" colspan="4"><font size=-1> </font></td>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>L2_trkoff<sub>255</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_len<sub>255</sub></font></td>
<td align="center" colspan="2"><font size=-1>L2_size<sub>255</sub></font></td>
<td align="center" colspan="4"><font size=-1> </font></td>
</tr>
</table>
<p>
<!-- ---------------------------------------------------------------------------------- -->
<br><br>
A <b>track image</b> or <b>block group image</b> contains two fields:
a 5-byte <em><u>header</u></em>
and a variable amount of data that may or may not be compressed.
The length field in the secondary lookup table entry (<i>L2_len</i> field shown just above)
includes the length of both the 5-byte header as well as the track or block group data.
<p>
The 5-byte <i>track or block group <u>header</u></i> contains a 1 byte compression indicator
and 4 bytes that identify the track or block group. The format of the identifier
depends on whether the emulated device is CKD or FBA:
<p>
<!-- ---------------------------------------------------------------------------------- -->
<table border=1 width="21.875%">
<tr><th width="100%" colspan="5">CKD Track Header</th></tr>
<tr>
<th width="6.25%">0</th>
<th width="6.25%">1</th>
<th width="6.25%">2</th>
<th width="6.25%">3</th>
<th width="6.25%">4</th>
</tr>
<tr>
<td align="center" colspan="1"><font size=-1>cmp</font></td>
<td align="center" colspan="2"><font size=-1>CC</font></td>
<td align="center" colspan="2"><font size=-1>HH</font></td>
</tr>
<p>
</table>
<p>
For <b>CKD</b>, the 2 byte CC is the cylinder number of the track image
and the HH is the head number. These numbers are stored in <em>big-endian</em>
byte order. When the compression indicator byte is zero the 5 byte header
is identical to the <em>Home Address</em> (or <em>HA</em>) for the track image.
<p>
The data --- which may or may not be compressed --- begins with the <em>R0</em>
count and ends with the <em>end-of-track</em> marker,
which is a count field containing <code>FFFFFFFFFFFFFFFF</code> (eight hex x'FF's).
The Home Address plus the uncompressed track data comprise the track image.
<p>
<!-- ---------------------------------------------------------------------------------- -->
<table border=1 width="21.875%">
<tr><th width="100%" colspan="5">FBA Block Group Header</th></tr>
<tr>
<th width="6.25%">0</th>
<th width="6.25%">1</th>
<th width="6.25%">2</th>
<th width="6.25%">3</th>
<th width="6.25%">4</th>
</tr>
<tr>
<td align="center" colspan="1"><font size=-1>cmp</font></td>
<td align="center" colspan="4"><font size=-1>nnnnnnnn</font></td>
</tr>
</table>
<p>
For <b>FBA</b>, the 4 byte nnnnnnnn is the big-endian <b>group number</b>
-- <i>(<u><b>not</b></u> the block or sector number!)</i> --
of the block group. The sector or block number of the first block in the group
is therefore the block group's <i><u>group number</u></i> times the number of blocks
per group (or 120), yielding the actual sector/block number of the first block in
the group.
The block group data (the individual sectors in the block group) --- which may or
may not be compressed (depending on the <i>cmp</i> flag) --- follows immediately
after the block group header.
<p>
There is of course no end-of-track marker at the end of an FBA block group.
(FBA devices do not have tracks.)
<p>
<b>Please note</b> that, while the <i>FBA Block Group Header</i> is the first part
of the block group, it is <i><u>not</u></i> part of the compressed image.
Only the sectors themselves (which immediately follow the <i>Block Group Header</i>)
are actually compressed. FBA devices do not have Track Headers (Home Addresses).
The <i>FBA Block Group Header</i> should be considered an internal structure
and is never exposed to the guest.
<p>
<!-- ---------------------------------------------------------------------------------- -->
The <b>cmp</b> compression indicator byte contains the value 0, 1 or 2.
Any other value is invalid:
<p>
<table border="1">
<tr><td align="center">0</td><td align="left"> Data is uncompressed</td></tr>
<tr><td align="center">1</td><td align="left"> Data is compressed using zlib</td></tr>
<tr><td align="center">2</td><td align="left"> Data is compressed using bzip2</td></tr>
<tr><td align="center">3...255</td><td> (invalid)</td>
</table>
<p>
<!-- ---------------------------------------------------------------------------------- -->
<br><br>
<b>free space blocks</b> which are <i>not</i> part of the <u>initial free space
chain table</u>, contain a 4 byte <i>offset</i> (8 bytes for CCKD64) to the
<i>next</i> free space, a 4 byte (or 8 byte) <i>length</i> of the free space
(which includes both the offset and length fields) and zero or more bytes
of undefined data.
<p>
The format of free space blocks which <i>are</i> part of the <u>initial free space
chain table</u> (used only during initialization of the internally maintained free
space chain) are formatted <i>slightly differently</i>. (see further below)
<p>
The contents of the actual free space <u>data</u> is entirely unpredictable,
and can be anything from previous track data to previously used secondary
lookup tables, etc. From the programmer's point of view it is meaningless.
It is just unpredictable/undefined file space.
<p>
The <i>fb_offnxt</i> and <i>fb_len</i> values, being numeric, are always kept in little endian format:
<p>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD_FREEBLK</th></tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="50%" colspan="8" align="left"> 8 . . .</th>
</tr>
<tr>
<td align="center" colspan="4"><font size=-1>fb_offnxt</font></td>
<td align="center" colspan="4"><font size=-1>fb_len</font></td>
<td align="center" colspan="8"><font size=-1>data . . .</font></td>
</tr>
<tr>
<td align="center" colspan="16"><font size=-1>
<br><center>data . . .</center><br></td>
</tr>
</table>
<br><br>
<table border=1 width="70%">
<tr><th width="100%" colspan="16">CCKD64_FREEBLK</th></tr>
<tr>
<th width="6.25%">0</th><th width="6.25%">1</th><th width="6.25%">2</th><th width="6.25%">3</th>
<th width="6.25%">4</th><th width="6.25%">5</th><th width="6.25%">6</th><th width="6.25%">7</th>
<th width="6.25%">8</th><th width="6.25%">9</th><th width="6.25%">A</th><th width="6.25%">B</th>
<th width="6.25%">C</th><th width="6.25%">D</th><th width="6.25%">E</th><th width="6.25%">F</th>
</tr>
<tr>
<td align="center" colspan="8"><font size=-1>fb_offnxt</font></td>
<td align="center" colspan="8"><font size=-1>fb_len</font></td>
</tr>
<tr>
<td align="center" colspan="16"><font size=-1>
<br><center>data . . .</center><br></td>
</tr>
</table>
<p>
The compressed device header's <a href=#cdevhdr><i>free_off</i></a> field
contains the file offset to either the first actual free space block (which then
contains the offset to the next free space block, etc), or else the file offset
to a "new" format <u>initial free space chain table</u>, explained further
below.
<p>
The minimum length of a free space is 8 bytes (16 bytes for CCKD64). The free
space chain is ordered by file offset and no two free spaces are adjacent. The
chain is terminated when a free space block has zero offset to the next free space.
<p>
Each free space block in the chain (or the entire "new" format <u>initial free
space chain table</u>) is read when the file is first opened for read-write,
and are written only when the file is closed. The free space chain itself however,
is maintained internally in storage while the file is opened and the system is
running.
<p>
The compressed device header <a href=#cdevhdr><i>free_off</i></a> field
contains either the offset to the first free space block in the chain, or else
to a "new" format <u>initial free space chain table</u>. The compressed device
header <a href=#cdevhdr><i>free_num</i></a> field contains the count of
the total number of free spaces there are in the chain.
<p>
Whenever a dasd image file is closed, a "new" format <u>initial free space chain
table</u> is written out to the first available free space block that is large
enough to hold the entire table (or to the end of the file if a large enough
free space block was not found).
<p>
The "new" format <u>initial free space chain