Honour `_FillValue` attribute when loading ancillary files

[andrewcoughtrie]

Are there any linked Issues or Pull Requests?

• ticket:4168 - Original LFRic Trac ticket.
• ancil:2130 - ANCIL workaround Trac ticket.

Brief description

Presently, the _FillValue attribute is ignored when LFRic reads in ancillary files (and presumably other input files). This attribute (attribute conventions) is used to identify missing data] (e.g. to mask out land points for sea-only fields).

Further details of the issue.

Because the _FillValue attribute is ignored, this imposes a constraint on any tools creating files to be read into LFRic that they need to use the model RMDI value as a _FillValue.

Instead, the LFRic load should recognise _FillValue as representing missing data and translate this to RMDI.

There is an additional complexity in that the netCDF specification (linked above) includes "It is not necessary to define your own _FillValue attribute for a variable if the default fill value for the type of the variable is adequate". So the absence of a _FillValue attribute does not imply there is no missing data.

The default fill values can be found most easily with python:

In [1]: import netCDF4

In [2]: netCDF4.default_fillvals
Out[2]: 
{'S1': '\x00',
 'i1': -127,
 'u1': 255,
 'i2': -32767,
 'u2': 65535,
 'i4': -2147483647,
 'u4': 4294967295,
 'i8': -9223372036854775806,
 'u8': 18446744073709551614,
 'f4': 9.969209968386869e+36,
 'f8': 9.969209968386869e+36}

Note that none of these defaults match the LFRic RMDI (the largest finite negative value).