Skip to content

BUG: Fix display with nested NumPy arrays #10222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

basnijholt
Copy link

@basnijholt basnijholt commented Apr 11, 2025

Currently, the following will fail to display:

import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros(2)  # Set to 1D array of size 2
ds = xr.DataArray(x)
ds
Click to see the error message
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/IPython/core/formatters.py:347, in BaseFormatter.__call__(self, obj)
    345     method = get_real_method(obj, self.print_method)
    346     if method is not None:
--> 347         return method()
    348     return None
    349 else:

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/common.py:188, in AbstractArray._repr_html_(self)
    186 if OPTIONS["display_style"] == "text":
    187     return f"<pre>{escape(repr(self))}</pre>"
--> 188 return formatting_html.array_repr(self)

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting_html.py:323, in array_repr(arr)
    315 arr_name = f"'{arr.name}'" if getattr(arr, "name", None) else ""
    317 header_components = [
    318     f"<div class='xr-obj-type'>{obj_type}</div>",
    319     f"<div class='xr-array-name'>{arr_name}</div>",
    320     format_dims(dims, indexed_dims),
    321 ]
--> 323 sections = [array_section(arr)]
    325 if hasattr(arr, "coords"):
    326     sections.append(coord_section(arr.coords))

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting_html.py:232, in array_section(obj)
    226 collapsed = (
    227     "checked"
    228     if _get_boolean_with_default("display_expand_data", default=True)
    229     else ""
    230 )
    231 variable = getattr(obj, "variable", obj)
--> 232 preview = escape(inline_variable_array_repr(variable, max_width=70))
    233 data_repr = short_data_repr_html(obj)
    234 data_icon = _icon("icon-database")

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:304, in inline_variable_array_repr(var, max_width)
    302     return var._data._repr_inline_(max_width)
    303 if getattr(var, "_in_memory", False):
--> 304     return format_array_flat(var, max_width)
    305 dask_array_type = array_type("dask")
    306 if isinstance(var._data, dask_array_type):

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:223, in format_array_flat(array, max_width)
    220 # every item will take up at least two characters, but we always want to
    221 # print at least first and last items
    222 max_possibly_relevant = min(max(array.size, 1), max(math.ceil(max_width / 2.0), 2))
--> 223 relevant_front_items = format_items(
    224     first_n_items(array, (max_possibly_relevant + 1) // 2)
    225 )
    226 relevant_back_items = format_items(last_n_items(array, max_possibly_relevant // 2))
    227 # interleave relevant front and back items:
    228 #     [a, b, c] and [y, z] -> [a, z, b, y, c]

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:212, in format_items(x)
    209     elif np.logical_not(time_needed).all():
    210         timedelta_format = "date"
--> 212 formatted = [format_item(xi, timedelta_format) for xi in x]
    213 return formatted

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:193, in format_item(x, timedelta_format, quote_strings)
    191     return repr(x) if quote_strings else x
    192 elif hasattr(x, "dtype") and np.issubdtype(x.dtype, np.floating):
--> 193     return f"{x.item():.4}"
    194 else:
    195     return str(x)

ValueError: can only convert an array of size 1 to a Python scalar

Whenever there are size==1 arrays, it currently does work:

import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros((1, 1, 1))  # Set to 3D array of size 1
ds = xr.DataArray(x)
ds

Should I add these examples as regression tests anywhere, if so, where?

For context, I am running into this issue in https://github.com/pipefunc/pipefunc where one can put the resulting objects into xarray.Datasets.
For example, see the docs here https://pipefunc.readthedocs.io/en/latest/examples/physics-simulation/ (search for xarray.Dataset).

image
  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

Copy link

welcome bot commented Apr 11, 2025

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

Currently, the following will fail to display:
```python
import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros(2)  # Set to 1D array of size 2
ds = xr.DataArray(x)
ds
```

Whenever there are `size==1` arrays, it currently does work:
```python
import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros((1, 1, 1))  # Set to 3D array of size 1
ds = xr.DataArray(x)
ds

For context, I am running into this issue in https://github.com/pipefunc/pipefunc where one can put the resulting objects into `xarray.Dataset`s.
For example, see the docs here https://pipefunc.readthedocs.io/en/latest/examples/physics-simulation/ (search for `xarray.Dataset`).
@@ -101,6 +101,8 @@ def test_format_item(self) -> None:
(np.float16(1.1234), "1.123"),
(np.float32(1.0111111), "1.011"),
(np.float64(22.222222), "22.22"),
(np.zeros((1, 1)), "0.0"),
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the original behavior.

I would argue that this is technically incorrect.

As an alternative we could check if x.shape == ().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant