Cropping inference rollout output

[stefanwayon]

I have recently implemented cropping of rollout files, since global rollouts take up a lot of space and I am interested in only smaller regions downstream. Do you think this would be a useful feature to include in the main repo?

If so, what else do you think is needed for a PR? For example:

• should the cropping extend to bias?
  easy to implement but not very useful since it’s O(1) not O(T) space as far as I can tell
• should the cropping extend to metrics?
  that would actually be very nice but I think pretty hard to do relative to the current changes
• should the in-memory buffer be already cropped?
  I tried to touch as little of the existing logic as possible with these changes, hence only cropping just before saving to disk
• how does this all interact with input cropping I’ve seen mentioned in metrics?
  no idea, but likely in complicated ways

[azrael417]

Thanks for reaching out. The input cropping basically crops input and output and thus would also crop metrics and outputs. It should (in principle) also crop the quadrature weights and related quantities, so it is basically end-to-end. However, that being said, while I tried to keep it working through many changes in makani, I am not using it and haven't tried it recently. But feel free to do so. You should be able to enable it by specifying img_crop_offset_x and img_crop_offset_y as well as img_crop_shape_x and img_crop_shape_y in the parameters file. Feel free to try that out and let me know if that works or what problems you run into.

[stefanwayon]

Does input cropping somehow crop the model as well? I can’t imagine it being possible to run a global model on a patch?

What I am trying to do is to use the inference.py script to output rollouts from the FCN3 global model, but only keep a crop of the output. I still need the global output for rolling out, but I don’t have enough disk space to store a few years of global rollouts.