Skip to content

interleave_columns wraps its output size for wide tables past INT32_MAX interleaved rows #23832

Description

@VaggelisGian

cudf::interleave_columns computes its output size as num_columns * num_rows in 32-bit signed arithmetic at every type path, with no product guard:

  • cpp/src/reshape/interleave_columns.cu:83 (structs path)
  • cpp/src/reshape/interleave_columns.cu:170 (strings path)
  • cpp/src/reshape/interleave_columns.cu:191 (fixed-width path)
  • cpp/src/lists/interleave_columns.cu:47 and :348 (lists sibling, independently callable)

There is no column-count limit upstream of these sites, so a table like 3000 columns x 800k rows (well within memory) asks for an output that cannot be represented as a libcudf column and triggers signed overflow instead of a clean error. Depending on the wrap residue:

  • positive wrap (e.g. 70000 x 70000 -> ~605M): fixed-width path silently returns a wrong-size column filled from wrapped indices; structs path passes all existing child-size EXPECTS self-consistently with the wrong row count; strings/lists produce silently truncated results.
  • negative wrap: surfaces later as misleading errors ("Column size cannot be negative", or an rmm OOM on a ~1.8e19-byte allocation).

Same class as the tile product overflow (#23830). The nested concatenate path is already protected by cudf::detail::concatenate's own size EXPECTS, so it needs no change.

Suggested fix: one repeat-style guard in detail::interleave_columns before the type dispatcher (static_cast<int64_t>(num_columns()) * num_rows() <= size_type_max, std::overflow_error), plus the same guard at the top of lists::detail::interleave_columns.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinglibcudfAffects libcudf (C++/CUDA) code.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions