Skip to content

Data type and compression in the output of cutmaps #42

Description

@casadoj

The output NetCDF files from cutmaps do not inherit the data type and compression of the original NetCDF files. The results is that the output file size might be excessively large compared with the original file.

For instance, I've used the cutmaps on the GloFAS4 reservoir fill maps (rsfil.nc):

cutmaps -f ./path_in/ -c "-114.125 -103.475 41.925 50.875" -o ./path_out/

The original files are the daily global maps for every year in the GloFAS long run; their size is approximately 300 Mb. The resulting maps represent a small portion of the Mississippi river, however their size is over 100 Mb. I've done a dummy test with Xarray and the resulting maps are 330 kb, instead.

# load dataset
da = xr.open_dataset(path_in / f'rsfil_2019.nc')
# cut the data
da = da.sel(lon=slice(-114.125, -103.475), lat=slice(50.875, 41.925))
# export NetCDF
encoding = {'rsfil': {'dtype': 'float64', 'zlib': True, 'complevel': 4}}
da.to_netcdf(path / f'giuseppe/test/test_{year}.nc', encoding=encoding)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions