How to efficiently sort zarr archive by time axis

[meteoDaniel]

I am having the following problem:

I am using xarray.open_mfdataset(['archive1', 'archive2'], engine='zarr') to open multiple zarr archives at once. During creation of the archive2, data was append to the archive in a non chronological way. This means my time axis is neither monotonically increasing nor decreasing. This yields to an error when using open_mfdataset.

Actually it was possible to fix this for small to medium size archives by opening the archive with xarray.open_dataset, apply sort() and store the archive as a copy. Afterwards I removed the old one and renamed the new fixed version.

But now I am having this issue with a huge zarr archive and the machine runs out of memory (and it has 128 GB RAM ) .
So is there another way I can re-order a time axis of an zarr archive?

[rabernat]

Noting that this discussion continued on the Pangeo forum: https://discourse.pangeo.io/t/how-to-efficiently-overwrite-existing-zarr-archive-with-reordered-time-axis-updated-question/2714