Skip to content

cast_column to ClassLabel silently accepts out-of-range label indices #8494

Description

@shoutoutuoadi325

Describe the bug

Dataset.cast_column / Dataset.cast to a ClassLabel column silently keeps label indices that are out of range for the target number of classes. The corruption only shows up later — e.g. on the first int2str call — or silently trains on wrong labels:

from datasets import Dataset, ClassLabel

d = Dataset.from_dict({"label": [5, 1]})
casted = d.cast_column("label", ClassLabel(names=["neg", "pos", "oth"]))  # no error, num_classes=3
print(casted["label"])                    # [5, 1]
casted.features["label"].int2str(5)       # ValueError: Invalid integer class label 5

The write path does validate the same data — Dataset.from_dict(..., features=Features({"label": ClassLabel(names=[...])})) raises ValueError: Class label 5 greater than configured num_classes 3 — so the cast path bypasses a check the writer performs.

Cause: table_cast in src/datasets/table.py only runs the feature-level casts when the arrow schema changes, and pa.Schema.__eq__ ignores metadata. Casting int64 storage to ClassLabel leaves the arrow schema unchanged and only alters the metadata, so it takes the replace_schema_metadata branch and ClassLabel.cast_storage, which does the range check, never runs.

Expected behavior

cast_column/cast should raise the same ValueError as the write path when a label index is >= num_classes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions