Consider distributing ESMPy via PyPI

[billsacks]

We have had a number of requests to distribute ESMPy via PyPI so that it can be available via pip. e.g., see discussion in conda-forge/esmpy-feedstock#72

Ideally this will be set up using an automated workflow so that it will automatically be updated on PyPI as new ESMF versions come out.

[ciaransweet]

This would be really useful! I don't use Conda and it would be a big move to swap to using it, just for the sake of being able to install ESMF without having to do it from source

[semvijverberg]

Yes, that would be great! Now it we can't use hatch to install our environment because it has to rely on Conda. Maybe there is a workaround though?

[Zeitsperre]

Would also love to see this. It would simplify the maintenance for more than a few of the projects I work on!

[anntsay]

follow up with Ryan to see if he has experience on this and maybe adding to his scope. there is a lot of thumbsup from user about enabling this.

[nannau]

I too would love to see PyPI support for ESMpy! It would greatly simplify maintenance and internal tooling at my workplace, especially for platforms that don't support Conda. This would also enable us to use tools like xESMF more broadly -- especially given recent directions. I'd like to help keep this issue active and contribute if I can!

As mentioned, some of this is partially covered in this issue, but here seems like a more logical place to continue. Please let me know if not!

---

To explore the feasibility of adding PyPI support, I tried a preliminary exercise using the following setup:

System: Ubuntu 22.04.5
Python: 3.12.6

1. Built ESMF locally from this repository.
2. Symlinked the esmf/lib directory into esmf/src/addon/esmpy/src/esmpy (admittedly, this might be an odd approach—there's likely a cleaner method).
3. Updated the pyproject.toml to include the following under [tool.setuptools.package-data]:

[tool.setuptools.package-data]
esmpy = [ "lib/libO/Linux.gfortran.64.mpiuni.default/*" ]

4. Installed esmpy from the pyproject.toml configuration into a fresh Python environment. The binaries were correctly included under venv/lib/python3.12/site-packages/esmpy/lib/libO/Linux.gfortran.64.mpiuni.default/.
5. Manually set $ESMFMKFILE, which also seemed to work as intended.
6. I built a wheel and source distribution following these steps and it too seems to have worked well.

I stopped here because I'm unclear about the following:

• How does the Conda installation determine where to locate the ESMF binaries?
• Is the above approach similar to how the project is packaged for Conda?
• Did I miss any critical steps? I did not run comprehensive testing here.

Assuming I'm on the right track and the above steps are generalizable, I understand the tricky part is trying to automate this. There was some discussion about potentially using docker images from here, but I don't quite have the experience with the esmf CI/CD pipeline to continue without a few hints. Cheers!

[anntsay]

Thanks for the comments. @nannau thanks for letting us know what you have tried so far. we will take a look at this and add this to the backlog.