Unconstrained forwarding of backend keyword arguments

[kmuehlbauer]

Is your feature request related to a problem?

Currently the backend keyword arguments have to be added to the signatures of open_dataset function of the respective BackendEntrypoint and to the open function of the respective Store. Every once in a while when a backend invents a new keyword argument xarray isn't capable to handle this out of the box and errors out.

There are requests/needs to add keyword arguments every once in a while (here only for netcdf4/h5netcdf backends):

• XArray does not support Latin characters in netCDF file names #9282, Support an encoding parameter when reading netCDF files with xarray #10357 and add **encoding** argument in netCDF4 backend #10372
• Support remote string paths for h5netcdf engine #8423, Added driver parameter for h5netcdf #8360, Add utility for opening remote files with fsspec #9797, phony_dims must be specified when opening HDF5 file without dimension scales #10049
• Allow track_order to be passed to h5netcdf #7680
• Add new h5netcdf backend phony_dims kwarg #3753, New library for handling complex numbers in netCDF #8288 and towards new h5netcdf/netcdf4 features #9509
• fix compatibility with h5py version 3 and unpin tests #4570 and FIX: h5py>=3 string decoding #4893

There are other attempts to simplify the calling signature for coders:

• Group together decoding options into a single argument #4490
• Opening dataset without loading any indexes? #6633/Allow setting (or skipping) new indexes in open_dataset #8051

Describe the solution you'd like

Consume all backend related keyword arguments in backend_kwargs. This is already part of the general api open_-functions. Instead of merging backend_kwargs with kwargs there, forward backend_kwargs to the backend and remove the backend related kwargs from the signature of those backend functions.

Instead check backend_kwargs for keywords known to the backend using the open_dataset_parameters class variable of the backend or similar. In case of yet unknown keywords issue a warning that this backend related keyword is currently not defined in the backend and the use is at own risk. Forward backend_kwargs to the respective backend open function.

Describe alternatives you've considered

• Keep everything as is and deal with it when issues arise.

  Works, but has it's downsides. Users are restricted to keyword arguments already available in the calling signature. No straightforward testing, knowledge of backend code needed.

Additional context

Until now there wasn't a problem with having backend keyword arguments mixed with the other keyword arguments. But in #9282/#10357 we get a naming clash which might not easily be resolved. Although this might be a one time solitary case, it introduces trouble.

[dcherian]

I support this. It's a constant maintenance burden to keep up with these new kwargs.

@shoyer what do you think?

[shoyer]

Works for me!

…

On Wed, Jun 4, 2025 at 5:55 PM Deepak Cherian ***@***.***> wrote: *dcherian* left a comment (pydata/xarray#10377) <#10377 (comment)> I support this. It's a constant maintenance burden to keep up with these new kwargs. @shoyer <https://github.com/shoyer> what do you think? — Reply to this email directly, view it on GitHub <#10377 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJJFVWOGOP6MHKRNSO7UOD3B4JH5AVCNFSM6AAAAAB6HLMOB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNBQGU3DINBWHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[keewis]

we had some more discussion about this in #8051, although these are probably more about arguments explicitly passed on by xarray instead of new kwargs for the backend.

[kmuehlbauer]

Thanks @keewis!

Good to see this from other perspectives, too. I'll have a closer look the next days. From a first glance it looks like adding **kwargs to the backends is undisputed.