Polytope Examples for DT Data Access

• Polytope Examples for DT Data Access
  • Access using your Destination Earth Service Platform credentials
  • Installation
    • Option 1: Conda Instructions
    • Option 2: Python Virtual Environment (venv)
  • Data Locations
  • Climate-DT Examples
    • Full Field
    • Full Field Post-Processing
    • Feature Extraction
    • Climate DT Explorer
  • Extremes-DT Examples
  • On-Demand Extremes-DT Examples
  • NextGEMS Examples
  • Polytope Quota Limits for DestinE

Access using your Destination Earth Service Platform credentials

This repository describes the process for accessing Destination Earth DT data via the Polytope web service hosted on the LUMI Databridge.

1. Clone the repo locally or if using polytope via Insula cd into the polytope directory.

git clone git@github.com:destination-earth-digital-twins/polytope-examples.git

2. Install polytope-client from PyPI:

pip install --upgrade polytope-client

3. Retrieve a token from the Destination Earth Service Platform (DESP) by running the script included in this repository:

python desp-authentication.py -u <username> -p <password>
# see --help for more options

Or you can run the script without arguments

python desp-authentication.py 
# see --help for more options

You will then be prompted to enter your username and password if no credentials are found in a config or through environment variables.

You will need some dependencies to run the script, which can be installed using pip:

pip install --upgrade lxml conflator

The script automatically places your token in ~/.polytopeapirc where the client will pick it up. The token is a long-lived ("offline_access") token.

4. Run the example scripts in this repository to download data, and customise them as you wish.

Installation

You can run the notebooks by setting up an appropriate environment using one of the following options:

• Option 1: Use the environment.yml file to create a Conda environment, or

• Option 2: Use the requirements.txt file to set up a Python virtual environment.

After creating the environment, the provided commands will also register an IPython kernel named earthkit, which you can select when working with the notebooks.

Option 1: Conda Instructions

envname=earthkit
conda create -n $envname -c conda-forge -y python=3.10
conda env update -n $envname -f environment.yml
conda activate $envname

# set earthkit environment to the default used by ipykernels
python3 -m ipykernel install --user --name=$envname

Option 2: Python Virtual Environment (venv)

Requires Python 3.10 or higher. You can check your version with python3 --version.

envname=earthkit

# Create a virtual environment (Python 3.10+ required)
python3 -m venv $envname

# Activate it
source $envname/bin/activate      # macOS/Linux

# Install dependencies
pip install -r requirements.txt

# If using Jupyter notebooks, register your environment as a kernel for ipykernel
python3 -m ipykernel install --user --name=$envname

Data Locations

Data can be found on both LUMI and MN5, to access data on either change the address argument in earthkit.data.from_source() to either polytope.mn5.apps.dte.destination-earth.eu for MN5 or polytope.lumi.apps.dte.destination-earth.eu for LUMI.

Climate-DT Examples

• Climate DT Example Directory — see also the Climate DT README for full details on available data and variables.

climate-dt/
├── explorer/                    # Browse the Climate DT data portfolio lazily without downloading upfront
├── feature-extraction/          # Extract specific spatial or temporal features from the datacube
├── full-field/                  # Retrieve and visualise complete global or regional fields
└── full-field-post-processing/  # Apply server-side or client-side post-processing to retrieved fields

Full Field

Polytope allows users to download complete global or regional fields from the Climate DT. Full field requests return GRIB data using standard MARS keys. Server-side options such as grid (interpolation to a target grid) and area (geographic subselection) can be applied at request time to reduce data volume before download. See the Polytope full fields documentation for more details.

• Climate DT python Script
• Climate DT notebook example
• Climate DT notebook domain example
• Climate DT notebook area example
• Climate DT notebook area of interest example
• Climate DT notebook grid example
• Climate DT notebook healpix data
• Climate DT notebook healpix ocean
• Climate DT notebook HighResMIP example
• Climate DT monthly means notebook example

Full Field Post-Processing

These notebooks demonstrate server-side and client-side post-processing of retrieved fields, including grid interpolation, and format conversion.

• Climate DT notebook healpix interpolation
• Climate DT notebook serverside interpolation
• Climate DT notebook GeoTIFF export

Feature Extraction

Polytope feature extraction reads only the user-requested data rather than whole fields, which significantly reduces I/O. Features are n-dimensional shapes (polytopes) cut from the datacube — such as time series at a point, vertical profiles, trajectories, bounding boxes, and polygons. See the Polytope feature extraction documentation for more details.

• Climate DT notebook feature extraction timeseries
• Climate DT notebook feature extraction trajectory
• Climate DT notebook feature extraction vertical profile
• Climate DT notebook feature extraction bounding box
• Climate DT notebook feature extraction country
• Climate DT notebook feature extraction polygon
• Climate DT notebook feature extraction pcolormesh
• Climate DT notebook feature extraction nudging story
• Climate DT notebook feature extraction monthly mean timeseries (baseline)
• Climate DT notebook feature extraction monthly mean timeseries (projections)
• Climate DT notebook feature extraction monthly mean polygon (baseline)
• Climate DT notebook feature extraction monthly mean polygon (projections)

Climate DT Explorer

The Climate DT Explorer notebooks allow you to lazily browse the full Climate DT data portfolio without downloading any data upfront. They expose the datacube as a virtual zarr store backed by Polytope, so you can inspect available variables, perform climate change analysis over multi-decadal periods, and generate ready-to-use request snippets — only fetching the data chunks you actually need.

• Climate DT lazy portfolio browser (monthly)
• Climate DT lazy portfolio browser (hourly)
• Climate DT climate change analysis
• Climate DT variable lookup

Extremes-DT Examples

• Extremes DT Example Directory
  • Full Field
    • Extremes DT python Script
    • Extremes DT notebook example
    • Extremes DT notebook domain example
    • Extremes DT notebook H3 example
  • Full Field Post-Processing
    • Extremes DT notebook regrid example
    • Extremes DT notebook serverside interpolation
  • Feature Extraction
    • Extremes DT notebook feature extraction timeseries
    • Extremes DT notebook feature extraction trajectory
    • Extremes DT notebook feature extraction 4D trajectory
    • Extremes DT notebook feature extraction vertical profile
    • Extremes DT notebook feature extraction bounding box
    • Extremes DT notebook feature extraction country
    • Extremes DT notebook feature extraction polygon
    • Extremes DT notebook feature extraction pcolormesh
    • Extremes DT notebook feature extraction polygon using H3
    • Extremes DT notebook feature extraction wave

On-Demand Extremes-DT Examples

• On-Demand Extremes DT Example Directory
  • Full Field
    • On-Demand Extremes DT python Script
    • On-Demand Extremes DT notebook example
  • Feature Extraction
    • On-Demand Extremes DT feature extraction country example
    • On-Demand Extremes DT feature extraction trajectory example
    • On-Demand Extremes DT feature extraction timeseries example
    • On-Demand Extremes DT feature extraction vertical profile example

NextGEMS Examples

• NextGEMS Example Directory
  • Full Field
    • NextGEMS historical 1990-2020 notebook
    • NextGEMS scenario 2020-2050 notebook
  • Feature Extraction
    • NextGEMS historical feature extraction timeseries
    • NextGEMS historical feature extraction trajectory
    • NextGEMS historical feature extraction vertical profile
    • NextGEMS historical feature extraction bounding box
    • NextGEMS historical feature extraction country

Polytope Documentation

General information about the Polytope Web service and how to use it can be found here: https://polytope.readthedocs.io/en/latest/Service/Full_fields/

Polytope Quota Limits for DestinE

To ensure system stability and fair usage, the following operational limits are enforced:

• API Rate Limit: Up to 50 requests per second. This limit may be adjusted based on system usage.
• Concurrent Operations Limit: A maximum of 5 download requests can be active at the same time.

Please plan your usage accordingly to avoid interruptions.