Important

Currently (March 2026), this repository contains an early version of ICONEval, which is targeted to the evaluation of ICON output on DKRZ's Levante.

---

Coverage Report

File	Stmts	Miss	Cover	Missing
init.py	16	11	31%	13–14, 18–25, 27
_config.py	11	11	0%	3, 5, 14, 17–21, 23–24, 26
_dependencies.py	20	20	0%	3, 5, 7, 9, 12, 14–15, 21, 24, 33, 36, 42–43, 48, 51, 60, 64, 70–71, 75
_io_handler.py	184	184	0%	3, 5–7, 9–11, 18, 20, 23, 26, 28–29, 31–36, 38, 45, 48–50, 52–59, 62–69, 71, 73, 78–79, 81, 83–84, 86, 88–89, 91–93, 95–96, 98–100, 102–103, 105–107, 109–110, 112–114, 116–117, 119–121, 123–124, 126, 128–129, 131, 133, 153, 156, 162, 165, 168–170, 175, 182–185, 191, 203, 205, 207, 217, 220–221, 224–226, 231–234, 237–240, 249, 252–253, 255, 266, 268, 270–272, 275–281, 284–286, 289, 291, 293, 295–301, 303, 311–315, 319–320, 324, 329–332, 337–338, 341–346, 348–351, 353, 355–356, 358–360, 362–363, 365, 367, 369–370, 372, 374–375, 379–381, 383, 385
_job.py	98	98	0%	3, 5–7, 16, 18, 21, 24, 39–42, 44–53, 55, 57, 63, 65, 67–68, 70, 72–73, 75, 77–78, 80, 82–83, 85–89, 91–92, 94–98, 100–101, 103, 105–106, 108, 110–111, 113, 115–116, 118, 120–121, 123, 133–134, 144–146, 148–150, 152, 154, 156, 158, 160, 162, 164, 166, 168, 170, 172–176, 178, 180–184, 186, 195, 204, 209, 211
_logging.py	13	13	0%	3, 5–6, 8, 10, 13, 19, 22, 26, 29–32
_recipe.py	10	10	0%	3, 5, 15, 18–21, 23–24, 26
_simulation_info.py	42	0	100%
_templates.py	220	220	0%	3, 5–8, 10, 12–13, 20, 23–24, 26, 28–29, 31, 33, 35–36, 38, 40–41, 43, 45–46, 48, 50, 52–54, 58, 60, 70–78, 81–83, 85, 89, 91, 93, 99–100, 104–121, 123–124, 126, 128–131, 134, 137–138, 140–143, 145, 147–151, 153–154, 156–157, 161, 163–164, 166–167, 170, 172–173, 175–176, 179, 181–182, 184–186, 188, 195, 205, 210–211, 214–215, 221–223, 230–233, 237, 241–247, 249, 251, 265–269, 271–273, 275–276, 279–282, 286–289, 294–297, 299–300, 302, 304–307, 316, 318, 320, 327, 330–335, 338–340, 345–349, 353–359, 362, 364, 366, 368, 370–375, 378, 381–382, 384, 392, 395–397, 399, 403, 411, 417–418, 421–423, 428–429, 435, 438–440, 467, 469–470
_typing.py	6	6	0%	3, 5–6, 8, 11, 13
main.py	171	171	0%	3, 5–8, 10–11, 13–14, 19–23, 32, 34, 45, 48, 190, 193–197, 200–207, 211–220, 224, 227, 230–237, 240–244, 247–249, 251, 254–256, 259, 275–279, 282–286, 289, 292–299, 302, 305–306, 309–311, 316, 318–325, 327, 333–335, 338, 341, 343–344, 347, 352–355, 359, 367, 369–371, 374–381, 384–387, 391–392, 395–416, 419–421, 425–429, 431–432, 435, 438, 440, 443, 445–446, 449–450
output_handling
init.py	0	0	100%
_summarize.py	193	193	0%	10, 12–16, 18, 20–21, 29, 31, 34, 37–40, 44, 65–66, 75, 101, 106–107, 114–115, 120, 126, 128, 131, 133–134, 136–144, 147–148, 150, 153–156, 159–160, 162–165, 193, 195, 198, 200, 202, 210, 212–216, 218–222, 224–228, 230–231, 233, 236, 238–243, 246, 248–250, 253, 255–257, 260, 262–267, 270, 274–279, 282, 284, 286–289, 291–294, 296, 299, 301–303, 305–307, 309–316, 318–320, 322–324, 327, 329–330, 332–333, 335, 338, 340, 343, 345–346, 348–349, 351–353, 355–357, 359–363, 365, 368, 370, 373, 375, 378, 380, 383, 385, 420, 427–428, 430–432, 435, 441–442, 444, 446–447, 462, 498, 517–519, 521–522, 524–526
plots2pdf.py	185	185	0%	3, 5–7, 9–13, 23, 25, 30, 32–34, 42, 105–106, 109–110, 114–124, 128–129, 132–134, 138–139, 144–149, 153–154, 157–158, 161, 164–169, 171, 174, 177–178, 181, 206, 209–210, 212, 215, 248–249, 252–254, 259, 261–264, 266–268, 271, 278–279, 283, 286–289, 296–300, 302–304, 306, 309–310, 314, 318–319, 321, 325, 328, 330, 334–337, 343–344, 347–348, 351–352, 356, 359, 379, 382–384, 386–388, 391–394, 396, 399, 401–403, 406, 408–410, 413, 427–430, 434–439, 444, 447–448, 452–458, 461, 464, 470–480, 484–487, 491–494, 498–500, 503, 505, 508–509
publish_html.py	116	116	0%	3–7, 9–13, 15–16, 18, 20–22, 25, 93–94, 96–99, 102–104, 107–109, 111, 116, 123, 130–133, 135–138, 140–141, 149, 151–155, 157–158, 166, 168–172, 174, 179, 181–184, 187, 193–196, 198–199, 203–204, 207, 211–215, 217–218, 227–231, 234, 236, 241–242, 244, 247, 249–250, 252–253, 255–256, 258, 261, 263–265, 267, 270–273, 276–280, 282, 285, 287, 290–291
TOTAL	1285	1238	3%

---

ICONEval

ICON model output evaluation with ESMValTool.

Table of Contents

1. Quick Start
2. Prerequisites
3. Installation
4. Customization
5. Common ICON Output Format
6. FAQs

Quick Start

ICONEval facilitates the evaluation of ICON model output with ESMValTool by automatically running a set of predefined ESMValTool recipes. Its only necessary input is a valid path to ICON model output:

iconeval path/to/ICON_output

This path should point to the directory whose name is identical to the experiment name of the ICON simulation you want to evaluate, e.g., /root/to/my_amip_run for the experiment my_amip_run. In this case, for example, the simulation output files should be named my_amip_run_atm_2d_ml_<date>.nc or my_amip_run_lnd_mon_<date>.nc. Multiple simulations can be evaluated simultaneously by specifying multiple directories:

iconeval path/to/ICON_output path/to/other/ICON_output

ICONEval reads a set of file templates (ESMValTool recipes and ESMValTool configuration) and fills these with the information from the ICON simulation that will be evaluated. If no further options are specified, a set of default recipes with default settings are run. If ICONEval is run as a standalone script, one Slurm job per recipe is launched. If ICONEval is run within an sbatch script or salloc session, one job step per recipe is created. The following sbatch script can be used to submit a job on a compute node of DKRZ's Levante in which 8 recipes are run in parallel (see here for details on this):

#!/bin/bash -e
#SBATCH --mem=0
#SBATCH --nodes=1
#SBATCH --partition=compute
#SBATCH --time=03:00:00

iconeval path/to/ICON_output --srun_options='{"--cpus-per-task": 16, "--mem-per-cpu": "1940M"}'

ICONEval is highly customizable. For example, the desired time range and frequency of the input data, as well as a flag to publish a summary HTML on a public website can be passed with

iconeval path/to/ICON_output --timerange='20070101/20080101' --frequency=mon --publish_html=True

Publishing results via the command line option --publish_html=True uses the Swift object storage of DKRZ and requires a DKRZ account. User authentication works via a Swift token that needs to be renewed monthly. If the token needs to be renewed, the user is prompted for their DKRZ account and password information when running ICONEval. The token can also be regenerated manually, see FAQs for details. The raw files (figures and data) from published results can be accessed via DKRZ's Swiftbrowser.

To only run a subset of available recipes, you can specify --tags when running ICONEval:

iconeval path/to/ICON_output --tags=timeseries,maps

An overview of all tags available in the default recipe templates is given here.

For more information on this and a list of all options, run

iconeval -- --help

or have a look at the section on Customization.

Installing ICONEval also provides the command line tools plots2pdf (create summary PDF for arbitrary ESMValTool output) and publish_html (publish summary HTML on public website for arbitrary ESMValTool output). For more information on them, run

plots2pdf -- --help
publish_html -- --help

Example Results

• Fully coupled historical ICON-XPP simulation

Prerequisites

ICONEval needs to run on a machine where the Slurm Workload Manager is available for the submission of jobs.

Installation

On DKRZ's Levante, a pre-installed version of ICONEval is available via a module. Thus, there is no need to install it yourself as a user on this machine.

However, on other machines, or if you would like to develop new features for ICONEval on Levante, an installation from source (development installation) is necessary.

Levante

To load the ICONEval module on Levante, use

module use -a /work/bd1179/modulefiles
module load iconeval

Add these lines to your shell configuration file if you use ICONEval regularly. This file describes how the ICONEval module is set up.

Installation from Source (Development Installation)

Installation from source is described here.

Customization

ICONEval is highly customizable. Detailed information on this can be found here.

Common ICON Output Format

To ensure that ICONEval works smoothly, the ICON simulation output should follow these criteria as closely as possible.

FAQs

A set of frequently asked questions can be found here.