sima0_12_000: Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check by jimmielin · Pull Request #441 · ESCOMP/CAM-SIMA

jimmielin · 2025-10-31T17:22:43Z

Tag name (required for release branches): sima0_12_000
Originator(s): @jimmielin

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

Port tracer_data, a generic reader for chemistry and aerosol data, from CAM to CAM-SIMA. The module is as close to the original tracer_data as possible, only removing CAM-specific features (i.e., chunking and physics buffer), FV dycore polar averaging, structured dycore latitude weighting, and a rmv_file flag that deletes files after being read -- closes CCPPize/move tracer_data to CAM-SIMA #432.
Moves conservative regridding utility src/utils/horizontal_interpolate.F90 to CAM-SIMA (intended to be included in a sparse checkout from CAM in the future) -- currently only used for aircraft inputs, but could be used for all regridded data read from tracer_data in the future.
Add string utilities increment_string, last_non_digit, and get_last_significant_char in support of tracer_data.
Add registry entries for prescribed bulk (BAM) aerosol and prescribed ozone for radiation.
Add tracer_data chemistry and aerosols (BAM) input regression test (Derecho GNU).
Add a check for NaN values in physics check data -- closes NaNs in model state are not detected by physics data differences check #440.
If any NaN values are detected in the model state, the difference count is incremented and the maximum difference will be denoted as NaN.
The number of NaNs will be reported in the output. e.g.,

 ********** Physics Check Data Results **********

 TIMESTEP:            1

 Variable                                          # Diffs  Max Diff   Max Diff loc (rank, col, lev)
 --------                                          -------  --------   -----------------------------
 ocar1                                                  48       NaN   (0,1,25)
 (!) 48 NaN values in variable!
 bcar1                                                  48       NaN   (0,1,25)
 (!) 48 NaN values in variable!
 sslt1                                                  48       NaN   (0,1,25)

Describe any changes made to build system: N/A

Describe any changes made to the namelist: N/A

List any changes to the defaults for the input datasets (e.g. boundary datasets): N/A

List all files eliminated and why: N/A

List all files added and what they do:

A       src/utils/tracer_data.F90
  - new tracer_data utility in CAM-SIMA ported from CAM (will not be moved from CAM)

A       src/utils/horizontal_interpolate.F90
  - move horizontal_interpolate conservative regridder for chemistry data from CAM.

A       cime_config/testdefs/testmods_dirs/cam/outfrq_trcdata_bam_derecho/shell_commands
A       cime_config/testdefs/testmods_dirs/cam/outfrq_trcdata_bam_derecho/user_nl_cam
  - new tracer_data chemistry and aerosols input data regression test.

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

M       cime_config/testdefs/testlist_cam.xml
  - new tracer_data chemistry and aerosols input data regression test.

M       src/data/registry.xml
  - add prescribed ozone for radiation.
  - add prescribed BAM aerosol fields for radiation (used for input check).

M       src/physics/utils/physics_data.F90
  - check for NaNs in physics data check.

M       src/physics/utils/physics_grid.F90
  - make public the `dycore_unstructured` logical for tracer_data.

M       src/utils/string_utils.F90
  - new string utilities `increment_string`, `last_non_digit`, and `get_last_significant_char` in support of tracer_data.

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

  SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam_multitape (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam_multitape NLCOMP
  - known NLFAIL due to cime validation error on multiple history tapes (#430)

derecho/gnu/aux_sima:

  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_trcdata_bam_derecho (Overall: DIFF) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_trcdata_bam_derecho NLCOMP
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_trcdata_bam_derecho BASELINE /glade/campaign/cesm/community/amwg/sima_baselines/latest_gnu: ERROR BFAIL baseline directory '/glade/campaign/cesm/community/amwg/sima_baselines/latest_gnu/SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_trcdata_bam_derecho' does not exist
  - new baseline for new trcdata_bam test.

  SMS_Ln9.ne3pg3_ne3pg3_mg37.FADIAB.derecho_gnu.cam-outfrq_se_cslam (Overall: FAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FADIAB.derecho_gnu.cam-outfrq_se_cslam RUN time=9
  - known FAIL on shr_reprosum_calc call from src/dynamics/se/dycore/global_norms_mod.F90

derecho/nvhpc/aux_sima: All PASS

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

… from CAM

… BAM prescribed aero

nusbaume

Thanks for bringing in this major new capability @jimmielin! I have some change requests, some of which would require non-trivial amounts of effort.

However, given that it is unclear what the future of all of this code will be, I am happy if you want to push back on some of my requests, or make them issues that we can tackle in the future. Just let me know!

cime_config/testdefs/testlist_cam.xml

src/physics/utils/physics_data.F90

src/physics/utils/physics_grid.F90

src/utils/string_utils.F90

src/utils/horizontal_interpolate.F90

src/utils/tracer_data.F90

nusbaume · 2026-01-02T21:43:14Z

src/utils/tracer_data.F90

+
+  ! rebin src (source) to trg (target).
+  ! originally from mo_util
+  pure subroutine rebin(nsrc, ntrg, src_x, trg_x, src, trg)


Can we get rid of this subroutine entirely, and instead just use the one from ccpp_tuvx_utils?

It's the same subroutine, but ccpp_tuvx_utils is in physics/ncar_ccpp/to_be_ccppized/. Could I confirm if we are OK with CAM-SIMA code depending on atmos_phys code? Thanks!

Yep, using atmospheric_physics code in CAM-SIMA is totally allowed. In fact we are doing that already, for example here:

https://github.com/ESCOMP/CAM-SIMA/blob/development/src/dynamics/mpas/dyn_coupling_impl.F90#L509

Thanks, updated to use rebin from atmos_phys!

Co-authored-by: Jesse Nusbaumer <nusbaume@ucar.edu>

…o hplin/tracer_data

…ation

…unstructured throughout

nusbaume

Everything looks good to me now, thanks @jimmielin! I think the only outstanding request is to remove the copy of rebin, but I don't think that requires an extra re-review from me. Thanks again!

fvitt

I do have 1 question

fvitt · 2026-03-06T20:10:14Z

src/core_utils/string_core_utils.F90

        end if
    end subroutine tokenize_into_tokens_separator

+    ! Increment a string whose ending characters are digits.


I don't think we actually use this feature in cam. This was left over from the old MOZART chemical transport model. Not sure that increment_string needs to migrated to CAM-SIMA.

It seems it's mentioned in this "ccm type filename" block in tracer_data.

CAM-SIMA/src/utils/tracer_data.F90

Lines 990 to 1005 in 8dcd0e8

if ((.not. present(filenames_list)) .or. (len_trim(filenames_list) == 0)) then

!-----------------------------------------------------------------------

! ... ccm type filename

!-----------------------------------------------------------------------

pos = len_trim(filename)

fn_new = filename(:pos)

if (masterproc) write (iulog, *) sub//': old filename = ', trim(fn_new)

if (fn_new(pos - 2:) == '.nc') then

pos = pos - 3

end if

istat = increment_string(fn_new(:pos), 1)

if (istat /= 0) then

write (iulog, *) sub//': increment_string returned ', istat

write (iulog, *) ' while trying to decrement ', trim(fn_new)

call endrun(sub//': increment_string failure')

end if

I agree it is likely unused. I am happy to make an issue to remove this in a future cleanup. Thanks!

It would be nice to clean this up. If you want to do it an a future PR, that's fine with me.

Thanks @fvitt - I added #474 as a future issue so I can go back and clean this up.

fvitt

Thanks for working on this.

jimmielin · 2026-03-07T04:46:31Z

need to make development -> main PR after merging in ESCOMP/atmospheric_physics#331

then update atmos_phys tag with latest then run regression tests.

may need an additional atmos_phys PR to make sure things work - not able to easily test right now, because some PRs into main are required for this to work...

@jimmielin

…x for bulk aerosols (BAM) (#331) Originator(s): @jimmielin Most of the code based off from @fvitt; changes had to be made to remove use of pbuf and use the constituents infrastructure in CAM-SIMA/CCPP framework to register store prescribed aerosols data (code borrowed from @peverwhee and @nusbaume) Companion SIMA PR ESCOMP/CAM-SIMA#441 There will not be a CAM PR to bring these changes back because `tracer_data` has breaking interface changes (and no longer using pbuf) that make it infeasible to bring back these schemes into CAM. Description (include issue title and the keyword ['closes', 'fixes', 'resolves'] and issue number): - CCPPize CAM `prescribed_ozone` -- closes #323 - CCPPize CAM `prescribed_aero` into `prescribed_aerosols` scheme -- closes #328 - Partially CCPPize CAM `aerodep_flx` into `prescribed_aerosol_deposition_flux` scheme: only BAM functionality is fully implemented; MAM functionality depends on conversion of the aerosol model -- closes #328; future work documented in #329 List all namelist files that were added or changed: ``` A schemes/chemistry/prescribed_ozone_namelist.xml A schemes/chemistry/prescribed_aerosols_namelist.xml A schemes/chemistry/prescribed_aerosol_deposition_flux_namelist.xml ``` List all files eliminated and why: N/A List all files added and what they do: ``` A schemes/chemistry/prescribed_ozone.F90 A schemes/chemistry/prescribed_ozone.meta - complete CCPPization of prescribed_ozone (#323) A schemes/chemistry/prescribed_aerosols.F90 A schemes/chemistry/prescribed_aerosols.meta - complete CCPPization of prescribed_aero (#328) A schemes/chemistry/prescribed_aerosol_deposition_flux.F90 A schemes/chemistry/prescribed_aerosol_deposition_flux.meta - CCPPize aerodep_flx for bulk aerosols (BAM) only. (#328) future work #329 A test/test_suites/suite_tracer_data_test.xml - suite for testing tracer_data chemistry and aerosols input test via prescribed ozone, aerosols, and aerosol deposition flux -- BAM configuration ``` List all existing files that have been modified, and describe the changes: (Helpful git command: `git diff --name-status development...<your_branch_name>`) N/A List all automated tests that failed, as well as an explanation for why they weren't fixed: N/A Is this an answer-changing PR? If so, is it a new physics package, algorithm change, tuning change, etc? new chemistry and aerosol schemes If yes to the above question, describe how this code was validated with the new/modified features: new regression test verified b4b with cam using same namelist options and input files as described in testmod - see CAM-SIMA PR --------- Co-authored-by: Jesse Nusbaumer <nusbaume@ucar.edu>

jimmielin added 13 commits October 9, 2025 14:42

Initial (lightly modified) version of tracer_data from CAM; utilities…

66bbc52

… from CAM

First try at dechunkization

92877f7

First try at removing pbuf support

d523030

Remove (for now) horizontal weighting for aircraft emiss.

977910f

to_lower units; use state directly in advance_trcdata

88649c6

Merge branch 'development' into hplin/tracer_data

322fd79

Fixes to build tracer_data in SIMA.

c4b2f17

Merge branch 'development' into hplin/tracer_data

37064db

Cleanup

35c9bbc

Merge branch 'development' into hplin/tracer_data

4a6bdef

Merge branch 'development' into hplin/tracer_data

d9802ea

Add tracer_data regression test; regridding fix; registry entries for…

a8c6b01

… BAM prescribed aero

Add NaN check in check data

1ee957f

jimmielin self-assigned this Oct 31, 2025

jimmielin added the enhancement New feature or request label Oct 31, 2025

jimmielin mentioned this pull request Nov 12, 2025

Complete CCPPization of prescribed_ozone; prescribed_aero; aerodep_flx for bulk aerosols (BAM) ESCOMP/atmospheric_physics#331

Merged

jimmielin marked this pull request as ready for review November 17, 2025 16:07

jimmielin requested a review from nusbaume December 3, 2025 19:48

jimmielin added 4 commits December 29, 2025 12:08

Merge branch 'development' into hplin/tracer_data

0d683ec

Update to sync with review comments atmos_phys (2)

3a50f89

Fix regression tests.

f8381c3

Fix regression tests, take two

9c1ece0

nusbaume requested changes Jan 2, 2026

View reviewed changes

jimmielin and others added 7 commits January 5, 2026 17:05

Update cime_config/testdefs/testlist_cam.xml

2b9cd4e

Co-authored-by: Jesse Nusbaumer <nusbaume@ucar.edu>

Merge branch 'development' into hplin/tracer_data

55f06db

Merge branch 'hplin/tracer_data' of github.com:jimmielin/CAM-SIMA int…

a49d68e

…o hplin/tracer_data

Address review comments: reduce extraneous computations in nan calcul…

27f8b9e

…ation

Address review comments: remove unstructured and replace with dycore_…

ffbc6a8

…unstructured throughout

Address various review comments in tracer_data

1c45bfb

Merge branch 'development' into hplin/tracer_data

81f93ba

jimmielin added 5 commits January 5, 2026 17:42

Remove duplicate O3

c06fe83

Move string functions to string_core_utils; add pfunit tests

d88de0e

Address review comments further: errmsg, check_allocate; cleanup

e4d2de0

Cleanup for review comments; use statements

9bbf406

Change to allocatables; clean up use statements

7c69f00

jimmielin requested a review from nusbaume January 12, 2026 17:37

Cleanup use statements

0f0dbcf

nusbaume approved these changes Jan 14, 2026

View reviewed changes

nusbaume requested a review from fvitt January 14, 2026 12:37

jimmielin added 6 commits January 15, 2026 11:28

Use rebin from atmos_phys.

c4edc59

Merge branch 'development' into hplin/tracer_data

f5e85f6

Merge branch 'development' into hplin/tracer_data

6500043

Merge branch 'development' into hplin/tracer_data

0fbbbda

Merge branch 'development' into hplin/tracer_data

033e390

Fix check_field_4d with nan_count feature

8dcd0e8

fvitt reviewed Mar 6, 2026

View reviewed changes

fvitt approved these changes Mar 6, 2026

View reviewed changes

jimmielin mentioned this pull request Mar 6, 2026

Clean-up tracer_data: increment_string use in incr_filename #474

Open

Merge branch 'development' into hplin/tracer_data

285fa65

Small cleanups to tracer_data

319e95f

jimmielin changed the title ~~Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check~~ sima0_12_000?: Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check Mar 9, 2026

jimmielin added 2 commits March 9, 2026 11:49

Update to latest atmos_phys hash (in main)

775ce6f

Fix plat, plon

d201a47

jimmielin changed the title ~~sima0_12_000?: Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check~~ sima0_12_000: Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check Mar 9, 2026

jimmielin merged commit d31e130 into ESCOMP:development Mar 9, 2026
16 checks passed

	if ((.not. present(filenames_list)) .or. (len_trim(filenames_list) == 0)) then
	!-----------------------------------------------------------------------
	! ... ccm type filename
	!-----------------------------------------------------------------------
	pos = len_trim(filename)
	fn_new = filename(:pos)
	if (masterproc) write (iulog, *) sub//': old filename = ', trim(fn_new)
	if (fn_new(pos - 2:) == '.nc') then
	pos = pos - 3
	end if
	istat = increment_string(fn_new(:pos), 1)
	if (istat /= 0) then
	write (iulog, *) sub//': increment_string returned ', istat
	write (iulog, *) ' while trying to decrement ', trim(fn_new)
	call endrun(sub//': increment_string failure')
	end if

Conversation

jimmielin commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nusbaume left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nusbaume left a comment

Choose a reason for hiding this comment

Uh oh!

fvitt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fvitt left a comment

Choose a reason for hiding this comment

Uh oh!

jimmielin commented Mar 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jimmielin commented Oct 31, 2025 •

edited

Loading