update diag_manager documentation

diag_manager/README.md

@@ -1,21 +1,164 @@
-The purpose of this document is to document the differences between the old diag manager and the new (modern) diag manager.
-
-## Contents
-- [1. Diag Table Format](README.md#1-diag-table-format)
-- [2. Scalar Axis](README.md#2-scalar-axis)
-- [3. Average Time Variables](README.md#3-average-time-variables)
-- [4. Subregional Files](README.md#4-subregional-files)
-- [5. Global attributes](README.md#5-global-attributes)
-- [6. Real attributes from diag_field_add_attribute calls](README.md#6-real-attributes-from-diag_field_add_attribute-calls)
-- [7. History files data output "changes"](README.md#7-history-files-data-output-changes)
-
-### 1. Diag Table Format
+Below outlines the differences between the legacy diag_manager and the modern diag_manager. Note, the public interfaces are the same between the legacy and modern diag_manager.
+
+## Table of Contents
+
+- [Module Organization and Naming Convention](#module-organization-and-naming-convention)
+- [Enabling the Modern Diag Manager](#enabling-the-modern-diag-manager)
+- [Diag Table Format](#diag-table-format)
+- [Scalar Axis](#scalar-axis)
+- [Average Time Variables](#average-time-variables)
+- [Subregional Files](#subregional-files)
+  - [`is_subregional` global attribute](#is_subregional-global-attribute)
+  - [Subregional dimension names](#subregional-dimension-names)
+  - [Corner and center diagnostics](#corner-and-center-diagnostics)
+- [Global attributes](#global-attributes)
+  - [Grid type and grid tile](#grid-type-and-grid-tile)
+  - [Associated_files global attribute](#associated_files-global-attribute)
+- [Real attributes from diag_field_add_attribute calls](#real-attributes-from-diag_field_add_attribute-calls)
+- [History files data output "changes"](#history-files-data-output-changes)
+
+
+
+#### Module Organization and Naming Convention
+
+The diag_manager is organized into separate modules, with new modules introduced in the rewrite using the `fms_diag_` prefix convention:
+
+**Legacy Modules (original diag manager):**
+- `diag_manager.F90` (top-level interface)
+- `diag_axis.F90`
+- `diag_data.F90`
+- `diag_grid.F90`
+- `diag_output.F90`
+- `diag_table.F90`
+- `diag_util.F90`
+- `fms_diag_time_reduction.F90`
+- `fms_diag_elem_weight_procs.F90`
+- `fms_diag_fieldbuff_update.F90`
+
+**New Modules (modern diag manager):**
+- `diag_manager.F90` (top-level interface) Same routines as the legacy, but when use_modern_diag is true routines will make calls to fms_diag_object.F90 and friends to perform all operations.
+- `fms_diag_object.F90` - Core diagnostic object implementation
+- `fms_diag_field_object.F90` - Field-specific diagnostic data structures
+- `fms_diag_file_object.F90` - File I/O management
+- `fms_diag_axis_object.F90` - Axis and domain handling
+- `fms_diag_bbox.F90` - Bounding box operations for subregional output
+- `fms_diag_output_buffer.F90` - Output buffering and data management
+- `fms_diag_reduction_methods.F90` - Reduction method implementations
+- `fms_diag_input_buffer.F90` - Input buffer management
+- `fms_diag_yaml.F90` - YAML diagnostic table parsing and handling
+- `fms_diag_time_utils.F90` - Time utility functions
+- `diag_data.F90` - stores all namelist parameters from diag_manager_nml
+
+#### Enabling the Modern Diag Manager
+
+FMS defaults to the legacy diag manager for backwards compatibility. Users must enable the modern diag manager via `use_modern_diag = .true.` in the diag_manager_nml, as seen below. FMS must also be compiled with the `-Duse_yaml` flag to enable and use the option.
+
+```
+&diag_manager_nml
+	use_modern_diag=.true.
+/
+```
+
+### Diag Table Format
 The modern diag manager uses a YAML format instead of the legacy ascii table. A description of the YAML diag table can
 be found [here](diag_yaml_format.md). A formal specification, in the form of a JSON schema, can be found in the
 [gfdl_msd_schemas](https://github.com/NOAA-GFDL/gfdl_msd_schemas) repository on Github.
 
-### 2. Scalar Axis
-The old diag manager was adding a `scalar_axis` dimension of size 1 for scalar variables
+Options can be added in the file section to set the default for each variable in that file. For example, `kind: r8` will set all variables to use r8 kind.
+Please ensure the YAML diag_table is indented correctly. The ordering of the key-value pairs does not matter.
+
+This barebones example creates a single netcdf file (per tile, if using a tiled domain):
+```{yaml}
+title: simple_diag_table
+base_date: 1 1 1 0 0 0
+diag_files:
+- file_name: simple_diagnostics
+  freq: 225 seconds
+  time_units: seconds
+  module: atm_mod
+  kind: r8
+  unlimdim: time
+  varlist:
+  - var_name: var1
+    output_name: variable_one
+	reduction: average
+```
+
+This is a more complex example utilizing subregional output and wildcard filenames:
+```{yaml}
+title: test_diag_manager
+base_date: 2 1 1 0 0 0
+diag_files:
+- file_name: normal
+  freq: 24 days
+  time_units: hours
+  unlimdim: records
+  module: potato_mod
+  kind: r8
+  reduction: min
+  varlist:
+  - module: atm_mod 
+    var_name: sst
+    output_name: sst
+    reduction: average
+    kind: r4
+    write_var: true
+    attributes:
+    - do_sst: .true.
+  sub_region:
+  - grid_type: latlon
+    corner1: -80, 0
+    corner2: -80, 75
+    corner3: -60, 0
+    corner4: -60, 75
+- file_name: normal2
+  freq: -1
+  time_units: hours
+  unlimdim: records
+  write_file: true
+  module: atm_mod 
+  reduction: none
+  kind: r4
+  varlist:
+  - var_name: sstt
+    output_name: sstt
+    long_name: S S T
+  - var_name: sstt2
+    output_name: sstt2
+    long_name: S S T
+    write_var: false
+  sub_region:
+  - grid_type: index
+    tile: 1
+    corner1: 10, 15
+    corner2: 20, 15
+    corner3: 10, 25
+    corner4: 20, 25
+- file_name: normal3
+  freq: -1
+  time_units: hours
+  unlimdim: records
+  write_file: false
+- file_name: wild_card_name%4yr%2mo%2dy%2hr
+  filename_time: end
+  freq: 6 hours
+  time_units: hours
+  unlimdim: time
+  new_file_freq: 6 hours
+  start_time: 2 1 1 0 0 0
+  file_duration: 12 hours
+  module: ocn_mod 
+  reduction: average
+  kind: r4
+  varlist:
+  - var_name: sst
+    output_name: sst
+  global_meta:
+  - is_a_file: true
+```
+
+### Scalar Axis
+The old diag manager adds a `scalar_axis` dimension of size 1 for scalar variables
 
 ```
 dimensions:
@@ -25,9 +168,9 @@ variables:
             	p700:_FillValue = 1.e+20 ;
             	p700:missing_value = 1.e+20 ;
 ```
-The new diag manager will no longer have a dummy scalar axis dimension.
+The new diag manager no longer adds a dummy scalar axis dimension.
 
-### 3. Average Time Variables
+### Average Time Variables
 The old diag manager includes time bounds metadata in a non-standard convention (i.e. `average_T1`, `average_T2`, and `average_DT`)
 1. `average_T1` is the start time for the averaging period (in the same time units as time)
 2. `average_T2` is the end time for the averaging period
@@ -69,34 +212,34 @@ This time_bounds variable is refernced as a variable attribute of time:
 		time:bounds = "time_bnds" ;
 ```
 
-### 4. Subregional Files
+### Subregional Files
 
-#### A. `is_subregional` global attribute:
-Subregional files will have a global NetCDF attribute `is_subregional = True` set for non-global history files. This attribute will be used in PP tools.
+#### `is_subregional` global attribute
+Subregional files will have a global NetCDF attribute `is_subregional = True` set for non-global history files. This attribute will be used by the post-processing tools in FRE.
 
-#### B. Subregional dimension names:
+#### Subregional dimension names
 In some cases, the old diag manager was adding `sub0X` to the dimension names where X is a number greater than 1. This was causing problems in PP tools that were expecting the dimension to have `sub01` in the name. The new diag manager will not have this problem.
 
-#### C. Corner and center diagnostics:
+#### Corner and center diagnostics
 In the old diag manager, if mixing variables that are corner variables, such as velocities={uo,vo,umo,vmo} and center variables, such as tracers={thetao,so,volcello} you sometimes ended up with a different number of variables per file. The extra files had duplicate data for the corner velocities because the two PEs shared the point at the edge. This happened with some grid/layouts/masks/subregion combinations and it caused problems with the combiner. The new diag manager will not have this problem.
 
-### 5. Global attributes
-#### A. Grid type and grid tile:
+### Global attributes
+#### Grid type and grid tile
 The old diag manager was adding the global attributes grid_type = "regular" and grid_tile = "N/A" for all files regardless of what the grid_type and the grid_title actually were. The new diag manager will no longer be doing this as they are not correct and don’t seem to be used.
 
-#### B. Associated_files global attribute:
+#### Associated_files global attribute
 We were unable to reproduce the exact order of the associated_files global attribute, so users may see differences like
 
 ```
 lake_area: 19790101.land_static.nc soil_area: 19790101.land_static.nc land_area: 19790101.land_static.nc <> land_area: 19790101.land_static.nc soil_area: 19790101.land_static.nc lake_area: 19790101.land_static.nc
 ```
 
-### 6. Real attributes from diag_field_add_attribute calls
+### Real attributes from diag_field_add_attribute calls
 When real attributes were added to the file via a diag_field_add_attribute call, the old diag manager is always saving it as NF90_FLOAT regardless of the precision the data was [passed in](https://github.com/NOAA-GFDL/FMS/blob/ebb32649efa395ea14598f74c8d49e74d1408579/diag_manager/diag_manager.F90#L4532-L4543)
 
 The new diag manager is going to write the attribute as it is passed in. This will cause differences when the model component was compiled with r8 as it will write the attribute as r8 instead of r4.
 
-### 7. History files data output "changes"
+### History files data output "changes"
 When the model run time is less than then the output frequency (i.e if the module run time is 2 days and you are writing monthly diagnostics), the old diag manager was writing 9.96921e+36. The new diag manager is not going to write anything for this cases, so if you ncdump the output from the new diag manager, you will get:
 
 ```

diag_manager/diag_axis.F90

@@ -24,6 +24,10 @@
 !!
 !! Users first create axis ID by calling diag_axis_init, then use this axis ID in
 !! register_diag_field.
+!!
+!! @note This file is part of the legacy diag_manager.
+!! If use_modern_diag is enabled, diag_axis_init will route all axis data storage through
+!! the fms_diag_axis_object.F90 module instead.
 
 !> @addtogroup diag_axis_mod
 !> @{

diag_manager/diag_grid.F90

@@ -22,6 +22,10 @@
 !!
 !> @author Seth Underwood seth.underwood@noaa.gov
 !!
+!! @note This module is part of the legacy diag_manager.  If use_modern_diag is enabled, this module will be unused
+!! except for the get_local_indexes procedure, which is still used by the modern diag_manager to determine the local
+!! indexes for regional output in the cube sphere grid.
+!!
 !! <TT>diag_grid_mod</TT> contains useful utilities for dealing
 !!   with, mostly, regional output for grids other than the standard
 !!   lat/lon grid.  This module contains three public procedures <TT>
@@ -38,22 +42,6 @@
 MODULE diag_grid_mod
 use platform_mod
 
-  ! <INFO>
-  !   <FUTURE>
-  !     Multi-tile regional output in the cubed sphere.
-  !   </FUTURE>
-  !   <FUTURE>
-  !     Single grid in the tri-polar grid.
-  !   </FUTURE>
-  !   <FUTURE>
-  !     Multi-tile regional output in the tri-polar grid.
-  !   </FUTURE>
-  !   <FUTURE>
-  !     Regional output using array masking.  This should allow
-  !     regional output to work on any current or future grid.
-  !   </FUTURE>
-  ! </INFO>
-
   USE constants_mod, ONLY: DEG_TO_RAD, RAD_TO_DEG, RADIUS
   USE fms_mod, ONLY: write_version_number, error_mesg, WARNING, FATAL,&
        & mpp_pe