Rebinning Pipeline Use Case

This page will describe the necessary steps for getting rebinned data via the process defined in ''Phase 2'' of the diagram attached to this page. '''NOTE''': The descriptions that follow will be based on the current plain binary files that are produced from data reduction. Things may change when we eventually move to single crystal !NeXus files from data reduction.

== Reduction to Rotated Data ==

This section handles the first step in the process operating on data files produced by data reduction. Each file contains a list of voxels for each pixel that has non-zero content. To simplify the text a bit, we will focus on the process for a single energy transfer slice from a data set that contains two runs. To begin, we have the following files present. The names and locations of the files are irrelevant, but shows a representative example of our current setup.

CNCS
  - 800
    - 800-mesh
      * CNCS_800_bmesh0000.inb
  - 801
    - 801-mesh
      * CNCS_801_bmesh0000.inb

=== Goiniometer Rotation Application ===

Now each run (800 and 801) must be rotated according to the goiniometer settings. This is achieved by multiplying each voxel by a matrix. The key trick is that each run will have a different matrix, therefore this information will need to be specified on a per run basis. Once the rotation information is included in the file, the angles/matrix parameters for rotation will not have to be formally specified. However, we should retain the ability to override what is in the file for a given run. The current procedure to rotate the data uses the {{{gmeshrot3}}} command on subsequent calls for each run and writes the rotated data to the same file. The procedure is accomplished in the following manner:

cat DIR/CNCS/800/800-mesh/CNCS_800_bmesh0000.inb | gmeshrot3 0 1 0 -10.0 >> OUT/CNCS_800_rbmesh0000.inb
cat DIR/CNCS/801/801-mesh/CNCS_801_bmesh0000.inb | gmeshrot3 0 1 0 -7.0 >> OUT/CNCS_800_rbmesh0000.inb

The invocation of gmeshrot3 requires four parameters: the first three are integers and the fourth is a floating point number. We are currently replacing the call to gmeshrot3 with a new executable: gmeshmat. This command requires nine floating point parameters that specify the rotation matrix in row-major order. The above would then look like:

cat DIR/CNCS/800/800-mesh/CNCS_800_bmesh0000.inb | gmeshmat x1_1 x2_1 x3_1 x4_1 x5_1 x6_1 x7_1 x8_1 x9_1 >> OUT/CNCS_800_rbmesh0000.inb
cat DIR/CNCS/801/801-mesh/CNCS_801_bmesh0000.inb | gmeshmat x1_2 x2_2 x3_2 x4_2 x5_2 x6_2 x7_2 x8_2 x9_2 >> OUT/CNCS_800_rbmesh0000.inb

The output from the rotation step is our simple binary format which is designated by the *.inb extension.

=== UB Matrix Application ===

The UB matrix may be applied as a run-by-run transformation or one transformation for all runs. Its place in the pipeline comes after the goiniometer rotation and if it is invoked, it will look like the command call for rotation ({{{gmeshmat}}}) but with a different set of 9 parameters. So, a possible application with goiniometer rotation for each run and a UB matrix transformation which is the same for all runs might look like the following invocation:

cat DIR/CNCS/800/800-mesh/CNCS_800_bmesh0000.inb | gmeshrot3 0 1 0 -10.0 | gmeshmat x1_1 x2_1 x3_1 x4_1 x5_1 x6_1 x7_1 x8_1 x9_1 >> OUT/CNCS_800_rbmesh0000.inb
cat DIR/CNCS/801/801-mesh/CNCS_801_bmesh0000.inb | gmeshrot3 0 1 0 -7.0 | gmeshmat x1_1 x2_1 x3_1 x4_1 x5_1 x6_1 x7_1 x8_1 x9_1 >> OUT/CNCS_800_rbmesh0000.inb

== Rotated Data to Rebinned Data ==

Once the data has been rotated and transformed by the UB matrix, the rebinning process can now commence. This process happens for each energy slice file that is present after the aforementioned transformations. The required information for the rebinner is the minimum and maximum extents of the three axes and the size (width) of the bins along each axis that are desired. Our axes are currently (Q,,x,,, Q,,y,,, Q,,z,,) and we hope to support (h, k, l) in the near future. However, the energy transfer information can also be used for the rebinning process, so we'll need to handle this at some point. So, the basic premise is that min, max and width information from three axes are required. The rebinning command is invoked in the following manner:

cat OUT/CNCS_800_rbmesh0000.inb | map -n num_threads gmeshrebin3 -f a1_min a1_max a1_width a2_min a2_max a2_width a3_min a3_max a3_width > OUT2/CNCS_800_rebin0000.in

There are two extra options to the gmeshrebin3 command: -b batchsize and -t threshold. The threshold option is one that by default is set to zero (actually 1e-16), but we'll want a user to be able to specify something different. The other option is one that administrators of the code will want tune to a particular system, but we won't expose it directly to the user. The map call is part of the multi-threaded capability of the rebinner. The -n num_threads needs to be specified from outside the execution code. The setup will be tuned by a code administrator and once the number of threads is chosen for a given set of resources, it will not be changed.

The output from the rebinning process is currently ASCII text, hence the *.in ending in the file names.

== Summary ==

The above process will need to be repeated for each energy slice that was generated by data reduction.