Skip to content

BUG: Fix display with nested NumPy arrays #10222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 20, 2025

Conversation

basnijholt
Copy link
Contributor

@basnijholt basnijholt commented Apr 11, 2025

Currently, the following will fail to display:

import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros(2)  # Set to 1D array of size 2
ds = xr.DataArray(x)
ds
Click to see the error message
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/IPython/core/formatters.py:347, in BaseFormatter.__call__(self, obj)
    345     method = get_real_method(obj, self.print_method)
    346     if method is not None:
--> 347         return method()
    348     return None
    349 else:

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/common.py:188, in AbstractArray._repr_html_(self)
    186 if OPTIONS["display_style"] == "text":
    187     return f"<pre>{escape(repr(self))}</pre>"
--> 188 return formatting_html.array_repr(self)

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting_html.py:323, in array_repr(arr)
    315 arr_name = f"'{arr.name}'" if getattr(arr, "name", None) else ""
    317 header_components = [
    318     f"<div class='xr-obj-type'>{obj_type}</div>",
    319     f"<div class='xr-array-name'>{arr_name}</div>",
    320     format_dims(dims, indexed_dims),
    321 ]
--> 323 sections = [array_section(arr)]
    325 if hasattr(arr, "coords"):
    326     sections.append(coord_section(arr.coords))

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting_html.py:232, in array_section(obj)
    226 collapsed = (
    227     "checked"
    228     if _get_boolean_with_default("display_expand_data", default=True)
    229     else ""
    230 )
    231 variable = getattr(obj, "variable", obj)
--> 232 preview = escape(inline_variable_array_repr(variable, max_width=70))
    233 data_repr = short_data_repr_html(obj)
    234 data_icon = _icon("icon-database")

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:304, in inline_variable_array_repr(var, max_width)
    302     return var._data._repr_inline_(max_width)
    303 if getattr(var, "_in_memory", False):
--> 304     return format_array_flat(var, max_width)
    305 dask_array_type = array_type("dask")
    306 if isinstance(var._data, dask_array_type):

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:223, in format_array_flat(array, max_width)
    220 # every item will take up at least two characters, but we always want to
    221 # print at least first and last items
    222 max_possibly_relevant = min(max(array.size, 1), max(math.ceil(max_width / 2.0), 2))
--> 223 relevant_front_items = format_items(
    224     first_n_items(array, (max_possibly_relevant + 1) // 2)
    225 )
    226 relevant_back_items = format_items(last_n_items(array, max_possibly_relevant // 2))
    227 # interleave relevant front and back items:
    228 #     [a, b, c] and [y, z] -> [a, z, b, y, c]

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:212, in format_items(x)
    209     elif np.logical_not(time_needed).all():
    210         timedelta_format = "date"
--> 212 formatted = [format_item(xi, timedelta_format) for xi in x]
    213 return formatted

File ~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:193, in format_item(x, timedelta_format, quote_strings)
    191     return repr(x) if quote_strings else x
    192 elif hasattr(x, "dtype") and np.issubdtype(x.dtype, np.floating):
--> 193     return f"{x.item():.4}"
    194 else:
    195     return str(x)

ValueError: can only convert an array of size 1 to a Python scalar

Whenever there are size==1 arrays, it currently does work:

import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros((1, 1, 1))  # Set to 3D array of size 1
ds = xr.DataArray(x)
ds

Should I add these examples as regression tests anywhere, if so, where?

For context, I am running into this issue in https://github.com/pipefunc/pipefunc where one can put the resulting objects into xarray.Datasets.
For example, see the docs here https://pipefunc.readthedocs.io/en/latest/examples/physics-simulation/ (search for xarray.Dataset).

image
  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

Copy link

welcome bot commented Apr 11, 2025

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

Currently, the following will fail to display:
```python
import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros(2)  # Set to 1D array of size 2
ds = xr.DataArray(x)
ds
```

Whenever there are `size==1` arrays, it currently does work:
```python
import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
    for j in range(2):
        x[i, j] = np.zeros((1, 1, 1))  # Set to 3D array of size 1
ds = xr.DataArray(x)
ds

For context, I am running into this issue in https://github.com/pipefunc/pipefunc where one can put the resulting objects into `xarray.Dataset`s.
For example, see the docs here https://pipefunc.readthedocs.io/en/latest/examples/physics-simulation/ (search for `xarray.Dataset`).
@basnijholt
Copy link
Contributor Author

Now test_slice_in_title_single_item_array is failing with:

  - d = [10.009]
  ?     -    ^^^
  + d = 10.01
  ?         ^

This test was introduced in #5948 and touches the exact same code that I changed. I think the test that is failing now is wrong and should be updated to d = [10.009] instead.

@basnijholt basnijholt requested a review from dcherian June 19, 2025 00:45
@dcherian
Copy link
Contributor

OK please update the test in that case. Merging in main should fix the other "array api" failures

@basnijholt
Copy link
Contributor Author

@dcherian, I updated the tests, all is passing except some (I believe) unrelated failure.

@dcherian dcherian merged commit 99a1ad2 into pydata:main Jun 20, 2025
31 of 33 checks passed
Copy link

welcome bot commented Jun 20, 2025

Congratulations on completing your first pull request! Welcome to Xarray! We are proud of you, and hope to see you again! celebration gif

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants