Add option to use system-provided HDF5 version

[Bisaloo]

Follow up from the discussion in Huber-group-EMBL/rhdf5#147.

For the Debian case specifically, a couple of points should be clarified before deciding the best way forward IMO:

What is Debian policy regarding Bioconductor versioning & R version compatibility? Are you always using the R version from the official repos for a given Debian version to determine the corresponding Bioconductor release?

The corollary question is whether packages depending on Rhdf5lib (such as, but not limited to, rhdf5) need to be compatible with multiple versions of HDF5. This seems like a big ask, especially because it doesn't just impact packages I'm maintaining, but also other downstream packages that may be working with this assumption that Rhdf5lib provides a pinned version of HDF5.

Do you have a link with the background to this project of building Bioconductor packages for Debian? Which packages are included?

For completeness, there is also interest for this beyond Debian use case: Huber-group-EMBL/rhdf5#104 (comment).

cc @mr-c

[jeroen]

@Bisaloo I was about to start the same discussion.

For Windows and MacOS, a recent version of libhdf5 has been included with the standard toolchain bundles for many years, and these are used by CRAN packages. I think bioconductor packages should do the same.

To get the proper flags, packages simply need to call pkg-config --cflags hdf5 or pkg-config --libs hdf5.

If you want I can help you migrate those packages.

[eddelbuettel]

(Got here via a link posted in the debian-r list)

These are all hard questions, and I am not aware of good 'general answers'.

Personally speaking, I generally prefer shared libraries (and that is influenced quite a bit by the Debian experience), but one does not always have good control over 'the system' one builds on. Other OSs without encompassing package managers have a harder task. So static libraries seem like shortcut. That said, I was always aware that BioConductor does (did ?) ship static libraries, but CRAN generally did not. Now we have https://cran.r-project.org/package=hdf5lib -- which got tossed off CRAN a bit when it started but now seems to be there for good. Are you folks aware of it? Would that be a viable base?

The other way about it, and increasingly more common, appears to be full vendoring. I don' think it is a great solution either, but I acknowledge that it makes some problems a developer may face go away.

As for the questions above, what @jeroen hints at with a minimal version somehow provided everywhere is a way out. As @Bisaloo's question, I would not say (with my #r2u bias) that the Debian packaging within the distro has cracked this nut so I am not sure I would recommend that approach as a model to follow in general (while it may of course work within Debian as it is easier for Debian to depend on its own libraries -- as I do say with RcppGSL). As I said, it's a loaded question. For nloptr we ended up with a hybrid: use the system library if you find it, else use a vendored copy. Again, no silver bullets.

[charles-plessy]

Hi @Bisaloo in Debian we always aim to build Bioconductor on the R version it was developed for. Debian's r-base package follows R's release very closely thanks to Dirk's work. We never build Bioconductor on an older R version but we may sometimes have to build it on a newer one when the release calendars of R, Debian and Bioconductor clash unfortunately. The list of Bioconductor packages that we package is quite ad-hoc at the moment but I aim to broaden it with time (the main bottleneck being that we have to copy copyright information in a debian-specific file and it is difficult to automate). We will never request that the Bioconductor and CRAN packages that we distribute support two HDF5 compatibility levels at the same time.

[eddelbuettel]

And in r2u I have of course builds Rhdflib for all variants so easy to compare. Using the local stash here

edd@paul:/var/local/r2u$ ls -l ubuntu/pool/dists/*/main/*rhdf5lib* 
-rw-rw-r-- 1 edd staff 2641556 May  1  2025 ubuntu/pool/dists/focal/main/r-bioc-rhdf5lib_1.30.0-1.ca2004.1_amd64.deb
-rw-rw-r-- 1 edd staff 2916698 Oct 31 07:13 ubuntu/pool/dists/jammy/main/r-bioc-rhdf5lib_1.32.0-1.ca2204.1_amd64.deb
-rw-rw-r-- 1 edd staff 2836712 Oct 30 14:46 ubuntu/pool/dists/noble/main/r-bioc-rhdf5lib_1.32.0-1.ca2404.1_amd64.deb
-rw-rw-r-- 1 edd staff 3224578 Nov  5 17:00 ubuntu/pool/dists/noble/main/r-bioc-rhdf5lib_1.32.0-1.ca2404.1_arm64.deb
edd@paul:/var/local/r2u$ 

A quick dpkg -f on these reveals no dependency on hdf5. That appears to be common as only these few do:

edd@paul:/var/local/r2u$ grep hdf ~/git/r2u-config/depends.dcf 
hdf5r: libhdf5-dev
vol2birdR: libhdf5-dev libproj-dev libgsl-dev
rhdf5filters: libzstd-dev
rlibkriging: libopenblas-dev cmake libhdf5-dev r-bioc-rhdf5lib
RcppPlanc: cmake libhdf5-dev
edd@paul:/var/local/r2u$ 

As mentioned you could look at what nloptr does to use a system library if found. I just realized that for r2u I do not give libnlopt-dev (but only cmake) so those builds are all statically linked across these three LTS releases.