test_forward_Downwelling_Radiance* fails with Intel ifx when OMP_NUM_THREADS > 1 (internal formatted write overflow)

[BenjaminTJohnson]

Description

All test_forward_Downwelling_Radiance* tests fail when OpenMP is enabled with more than one thread. The failure does not occur with OMP_NUM_THREADS=1.

The error appears to be triggered by OpenMP-specific output (likely a WRITE to an internal formatted string) that exceeds the record limit when multiple threads are active.

Observed behavior

With OMP_NUM_THREADS=4, the following failure occurs consistently:

Start 50: test_forward_Downwelling_Radiance_atms_n21

Test command: /data/users/bjohnson/CRTM/CRTMv3-develop/build/bin/test_forward_test_Downwelling_Radiance "atms_n21"
Working Directory: /data/users/bjohnson/CRTM/CRTMv3-develop/build/test
Environment variables:
  OMP_NUM_THREADS=4

**********************************************************
                 test_Downwelling_Radiance

 Test program for the aircraft instrument option under
 clear sky conditions.

 CRTM Version: v3.1.2
**********************************************************

Running CRTM for atms_n21 sensor...

Initializing the CRTM...
SpcCoeff_ReadFile(Binary)(INFORMATION) : FILE: ./testinput/atms_n21.SpcCoeff.bin;
SpcCoeff RELEASE.VERSION:  8.04
  N_CHANNELS=22
CloudCoeff_ReadFile(Binary)(INFORMATION) : FILE: ./testinput/CloudCoeff.bin;
CloudCoeff RELEASE.VERSION:   3.004  N_FREQUENCIES(MW)=31  N_FREQUENCIES(IR)=61  N_RADII(MW)=10  N_RADII(IR)=10  N_TEMPERATURES=5  N_MW_DENSITIES=3  N_IR_DENSITIES=4  N_LEGENDRE_TERMS=38  N_PHASE_ELEMENTS=1
AerosolCoeff_ReadFile(Binary)(INFORMATION) : FILE: ./testinput/AerosolCoeff.bin;
AerosolCoeff RELEASE.VERSION:   4.005  N_WAVELENGTHS=61  N_RADII=36  N_SIGMA=1  N_TYPES=8  N_RH=1  N_RH_Radii=36  N_LEGENDRE_TERMS=37  N_PHASE_ELEMENTS=1

forrtl: severe (27): too many records in I/O statement, unit -5, file Internal Formatted Write
forrtl: severe (27): too many records in I/O statement, unit -5, file Internal Formatted Write
forrtl: severe (27): too many records in I/O statement, unit -5, file Internal Formatted Write

test_forward_Downwelling_Radiance_atms_n21 ... Failed

Expected behavior

• test_forward_Downwelling_Radiance* should pass identically for OMP_NUM_THREADS=1 and OMP_NUM_THREADS > 1
• No thread-dependent formatted I/O failures

Key observations

• Failure occurs only when increasing OpenMP threads (e.g., OMP_NUM_THREADS=4)
• Single-threaded OpenMP (OMP_NUM_THREADS=1) passes
• All other ctests pass.
• Error is an Intel runtime error:

forrtl: severe (27): too many records in I/O statement

• Error references internal formatted write, not file I/O
• Strongly suggests:
  • A shared internal character buffer
  • A non-thread-safe formatted WRITE inside an OpenMP region
  • Multiple threads appending to the same internal string or unit

Suspected cause

One or more diagnostic or informational WRITE statements (likely to internal character variables) are executed concurrently under OpenMP, causing record overflows or race conditions in the Intel runtime.

Likely candidates:

• Logging / message construction utilities
• Error or status reporting code
• Any internal formatted WRITE(string, fmt) inside OpenMP regions without thread-local storage or synchronization

Environment

• CRTMv3 develop branch
• CRTM Version reported by test: v3.1.2
• Compiler: Intel ifx (oneAPI)
• OpenMP enabled
• Failure only when OMP_NUM_THREADS > 1

Suggested next steps

• Audit internal formatted WRITE statements executed inside OpenMP regions
• Ensure internal character buffers are:
  • Thread-local, or
  • Guarded by OMP CRITICAL, or
  • Suppressed for multi-threaded execution
• Consider disabling nonessential diagnostic output during OpenMP execution in tests

[BenjaminTJohnson]

this should be resolved by PR #277