[main] Use rocoto to run real-time RRFS based on main branch

[hu5970]

DESCRIPTION OF CHANGES:

Based on the current main branch:

1. Added the new rocoto workflow generation template and generation scripts (*_nco) to generate XML file
2. Added config files (based on parm/config) for CONUS 3km grid (under ush/sample_confgiures/config_c3)
3. Added function to configure both NA 3km and CONUS 3km grid
4. Updated modulefiles/tasks/wcoss2 to run the system on WCOSS2 with rocoto workflow
   All those changes are for using rocoto workflow to run RRFS. Those changes do not impact any operational code (jobs/scripts/fix/parm/ush) that NCO is working on.

TESTS CONDUCTED:

tested in real-time on WCOSSS2 for both NA 3km and CONUS 3km

[hu5970]

Updated the rocoto workflow to run the ensemble forecast. Now the deterministic cycles, enkf cycles, and ensemble forecast over both NA and CONUS 3km grid should be able to run.

[MatthewPyle-NOAA]

@hu5970 Are the CONUS 3 km fix files updated with the land mask changes made for the NA 3 km?

[ShunLiu-NOAA]

The changes looks good to me. I think we need to a new FIX directory to host the fix files for COUNS domain.

[hu5970]

The CONUS 3km lam does not have the new lake mask yet. Need to work on it soon.

[hu5970]

Lin and I had some discussion on main branch for future RRFSv1. Is it better to have a branch based on main to host the new changes and merge those changes to the main after testing with ECflow?

[MatthewPyle-NOAA]

@hu5970 That approach definitely would be preferable. Thanks!

[hu5970]

Could we have a new branch based on main branch: like main_dev?

[MatthewPyle-NOAA]

We could, but it may get messy to manage if it stays around as a long term branch. But I don't envision many updates for "main", since all development for v1.0 is now being done on rrfs-nco, so maybe that isn't a big concern.

[ShunLiu-NOAA]

Will [rrfs-nco] be used as repository for future update after implementation? Will we sync [main] with [rrfs-nco] in the future?

[hu5970]

No need to make decision of creating the branch for now. Let's find a chance to discuss the current functions of each branch and future of each branch. Here is my thoughts:
rrfs-nco: RRFSv1 operation (NA 3km domain, real-time on WCOSS2 with ECflow)
main: RRFSv1 future development. It includes all rrfs-nco (completely synced) with additional functions for RRFSv1 future development, such as run on both NA3km and COUNS 3km, real-time and retro, on WCOSS2 and Gaea/Ursa.

dev_sci: new functions (JEDI) should merge into main and frozen after main can serve the development purpose.

[MatthewPyle-NOAA]

We will want the final state of rrfs-nco to serve as a base for future development, but the ecflow aspects are NCO-specific, so merging everything into "main" might not be perfectly clean at the github level.