Graphics Maintenance - compatible with latest npl#2
Closed
ahijevyc wants to merge 13 commits intoJCSDA:developfrom
Closed
Graphics Maintenance - compatible with latest npl#2ahijevyc wants to merge 13 commits intoJCSDA:developfrom
ahijevyc wants to merge 13 commits intoJCSDA:developfrom
Conversation
is always a group in the h5 file. Sometimes it is DerivedObsError. Provide a warning to look in the file if a group does not exist.
predictors are plotted. Also make plot_predictors_distri a command line option. False by default, as usual.
to warn user instead of print statements. May want to use specific logger as done in scripts in parent graphics dir, but this is a start.
In older versions of NumPy, np.float was just an alias for the Python built-in float.
to parse first and last cycle times. Allows '20250610' and '20250610T00'.
instead of hard-coding /derecho/scratch/user Add checks: 1. if firstCycle time is defined, lastCycle must be too 2. firstCycle < lastCycle
Overhauled DataFrame Slicing (loc, levels): Replaced the fragile locTuple and locdf helper functions with native Pandas boolean masking inside DFWrapper.loc(). Replaced self.df.append with pd.concat() Stripped out custom Python loops in uniquevals, min, max, va, loc1 and replaced with native Pandas
Fixed insert method of MultipleBinnedStatistics Don't isinstance(val, Iterable) to verify val is array. It could be a str and slice it up. Fix Assertion message (comma inside parentheses). Unneeded copy (val[:]).
and oneHundredCenteredLimiter
Contributor
Author
|
Sorry. I meant to push to JCSDA-internal not JCSDA |
ahijevyc
deleted the
maintenance/graphics/use_pandas_not_custom_functions
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TYPE: maintenance
DESCRIPTION OF CHANGES:
This PR provides a comprehensive update to the MPAS-JEDI graphics suite. Key improvements focus on code simplicity, robustness, and compatibility with lastest Python conda environment (
npl-2025b).1. Modernization
datetimeparsing inAnalyzeStats.pytopd.to_datetimefor more flexible input handling.DFWrapperand associated helper functions (dfIndexLevels,uniquevals) to use native Pandas.unique(),.isin(), and boolean masking. This eliminates slow Python loops and drastically reduces memory overhead for large datasets.np.intandnp.floatwith nativeintandfloattypes to support NumPy 1.24+ and prevent future deprecation errors.2. Warning Elimination (Limits & Normalization)
Zero-Centered Limiter Logic: Reordered the logic in
oneHundredCenteredLimiterandzeroCenteredLimiterto apply safety clamps before performing division. This silencesRuntimeWarning: divide by zero encountered in scalar dividewhich occurred whendminwas initially 0.Safe Normalization: Fixed
BinValAxes2D.pyto explicitly check ifnormalizingStatis non-zero and finite before calculating relative differences. Gaps in control data now correctly result inNaNrather thaninfor runtime warnings.3. Dependency and Code Cleanup
plot_utilsDecoupling: Severed dependencies onplot_utilsin several core modules. Replacedpu.prependsandpu.postpendswith native Python.startswith()and.endswith().TDelta_dirinto a cleaner dictionary-based replacement loop.analyze_config.pyto useos.path.joinand generic environment variables ($SCRATCH) instead of hardcoded user paths.Note: This PR removes the requirement for the
npl-2023aconda environment; the suite now runs onnpl-2025b. No more divide by zero or invalid value warnings. Code is shorter but preserves legacy functionality.ISSUE:
N/A
LIST OF MODIFIED FILES:
graphics/AnalyzeStats.pygraphics/analysis/AnalysisBase.pygraphics/analysis/BinValAxisStatsComposite.pygraphics/analysis/StatisticsDatabase.pygraphics/analysis/category/CategoryBinMethodBase.pygraphics/analysis/category/FCScoreCard.pygraphics/analysis/multidim/BinValAxes2D.pygraphics/analysis/multidim/BinValAxisProfile.pygraphics/analyze_config.pygraphics/basic_plot_functions.pygraphics/standalone/plot_diag.pyTESTS CONDUCTED: