`Dataset.select` raises a raw OverflowError on in-range negative indices that its own validation allows

[codeAnqiang-ma]

Describe the bug

Dataset.select validates indices with _check_valid_indices_value, which implements Python-style negative indexing: it only raises for a negative index when index + size < 0, so in-range negative indices like -1 pass validation. But _select_with_indices_mapping then converts them with pa.array(indices, type=pa.uint64()), which crashes with a raw pyarrow OverflowError.

So the two layers disagree on whether negative indices are supported:

• out-of-range negative indices get a clean IndexError, while in-range ones crash with OverflowError: can't convert negative value to unsigned int;
• ds[[0, -1]] (__getitem__ with a list) has supported negative indices since OverflowError when slicing with an array containing negative ids #668/Fix negative ids when slicing with an array #679, and Fix IndexError on negative list indices for datasets with an indices mapping #8412 is currently fixing the same family in the formatting path — select remains the entry point that crashes.

Negative indices can't reach the contiguous fast path either (select requires indices.start >= 0 / start >= 0 there), so every negative index ends up in _select_with_indices_mapping and crashes.

Steps to reproduce the bug

from datasets import Dataset

ds = Dataset.from_dict({"x": list(range(10))})
ds[[0, -1]]              # {'x': [0, 9]} — getitem supports negative indices
ds.select([-11])         # IndexError: Index -11 out of range for dataset of size 10. (clean)
ds.select([-1])          # OverflowError: can't convert negative value to unsigned int
ds.select(range(-3, 0))  # same OverflowError
ds.select([9, 8, 7]).select([-1])  # same OverflowError with an indices mapping

Expected behavior

ds.select([-1]) returns the last row, consistent with ds[[-1]] and with the validation layer's own semantics (its error message "Index -11 out of range for dataset of size 10" implies -10..-1 are in range). At minimum, a clean IndexError instead of a raw pyarrow OverflowError. I'll open a PR that normalizes in-range negative indices after validation.

Environment info

• datasets 5.0.2.dev0 (main @ 836b82e), pyarrow 25.0.1, Python 3.13.3, macOS

---

Disclosure: this report was prepared with AI assistance; I reproduced the behavior locally and reviewed every claim.