Mesh generator re-factoring

[mo-rickywong]

Are there any linked Issues or Pull Requests?

No response

Brief description

The Cubedsphere/Planar mesh generators code base has evolved organically to such a state that it is becoming increasingly difficult to maintain, let alone supporting user requests for new functionality.

The proposed solution is to implement a pipeline of intermediate processing stages. This would improve code modularity. This benefits ease of maintenance and development by isolating code/data in the OO style. Pipeline stages would also improve flexibility as they could be re-ordered and re-used.

This approach also lends itself to being implemented piecemeal. Each discreet functional unit can distilled and extracted from existing code base individually.

To illustrate, a pipeline to generate a cubesphere mesh might look like this:

graph LR;
    generate["Generate unit cubedsphere mesh"] --> Rotate
    Rotate --> Stretch
    Stretch --> write["Write to NetCDF"]

Loading

A more complex example includes off-line partitioning:

graph LR;
    generate["Generate unit cubedsphere mesh"] --> map
    map["Compute mesh maps"] --> centres
    centres["Calculate cell centres"] --> Rotate
    Rotate --> Stretch
    Stretch --> tee((fork))
    tee --> Partition
    Partition --> multiwrite["Write local to NetCDF"]
    tee --> write["Write global to NetCDF"]

Loading

Further details of the issue.

There are 2 potential approaches to this work.

Consolidate and re-factor live codebase

This would entail amalgamating the current 2 generators in order to have a single code-base to refactor. This should avoid some duplicated effort where the two existing tools have duplicate code. Another advantage of this approach is that it allows us to take advantage of existing system testing.

A disadvantage of this approach is that attempting to refactor an existing code base, particularly a complicated one, can be slow going.

Potential Work Plan

• Combine planar and cubesphere generators into a single executable (rough and ready)
  • Update system tests to use this new executable
  • Documentation
• Hoist mesh mutation out of generator, up to level above. This frees the generator to look more like a pipeline stage.
• Implement pipeline manager, that will construct and execute the pipeline.
• Convert generators into pipeline stages and feed pipeline output into remaining legacy code blob.
• Convert first mutation from blob into pipeline stage.
• Repeat until file write "end state" stage is reached.
• Repeat for additional pipeline paths.

Clean sheet implementation

In this approach, a third tool is created to replace the current two and is developed in parallel, distilling desired functionality from existing code. The advantage here is that such clean-sheet implementations are less slowed by the baggage of existing implementations. The down side is that tests have to be developed in parallel, the existing ones can only be used once the project is complete and full functionality is available.

Potential Work Plan

• Create new program framework
  • Command-line parsing
  • Namelist reading
• Implement pipeline manager, as above
• Implement start (generator) and end (writer) stages. (Minimum pipeline)
• Implement intermediate stages, guided by existing code functionality.
• Repeat until all functionality implemented
• As new configurations are supported, system tests must be developed.