Pandas version checks
Reproducible Example
import numpy as np
import pandas as pd
columns = pd.MultiIndex.from_product(
[["A", "B"], ["c", "d"], ["e", "f"]], names=["l0", "l1", "l2"]
)
df = pd.DataFrame(np.arange(8).reshape(1, 8), columns=columns)
df.stack([0, 0]) # AssertionError, with no message
df.stack(["l0", "l0"]) # AssertionError, with no message
df.stack([1, -2]) # AssertionError, with no message
df.stack([2, -1]) # IndexError: pop index out of range
Issue Description
Passing a level list that contains duplicate entries to DataFrame.stack fails with an
internal error instead of a user-facing one:
File "pandas/core/reshape/reshape.py", line 1023, in stack_v3
assert isinstance(stack_cols, MultiIndex)
AssertionError
stack_v3 computes the columns to stack from the deduplicated set(level):
set_levels = set(level)
stack_cols = frame.columns._drop_level_numbers(
[k for k in range(frame.columns.nlevels - 1, -1, -1) if k not in set_levels]
)
but later branches on the length of the original, non-deduplicated list:
if len(level) > 1:
sorter = np.argsort(level)
assert isinstance(stack_cols, MultiIndex)
With level=[0, 0], set_levels has one entry, so stack_cols is a plain Index, while
len(level) is still 2 and the assertion fires.
Note that the duplicates need not be written literally. DataFrame.stack resolves level
names and negative level numbers with _get_level_number before reaching stack_v3, so on
3-level columns ["l0", "l0"], [1, -2] and [2, -1] all normalize to duplicated level
numbers and hit the same code path.
This is also a change in behavior from the previous implementation, which accepted duplicate
levels and stacked sequentially:
>>> df.stack([0, 0], future_stack=False) # legacy implementation
l2 e f
l0 l1
0 A c 1 2
d 3 4
B c 5 6
d 7 8
>>> df.stack(0).stack(0) # identical result
Expected Behavior
An informative exception rather than a bare AssertionError or an IndexError from an
internal list.pop.
There are two defensible behaviors, so I wanted to raise it rather than pick one silently:
- Raise an informative
ValueError. The current implementation stacks the specified
original column levels simultaneously, and "the same level twice" has no meaning in that
model. stack_v3 already raises ValueError("Columns with duplicate values are not supported in stack") for the analogous duplicate-column case. This also avoids silently
choosing semantics that differ from the legacy implementation, which stacked two distinct
levels for [0, 0].
- Deduplicate, so that
stack([0, 0]) behaves as stack([0]). This would match the
sibling APIs, which all accept duplicate levels today: unstack([0, 0]),
groupby(level=[0, 0]), droplevel([0, 0]) and reorder_levels all succeed. The
downside is that it silently changes the legacy result rather than flagging it.
I have a PR ready implementing option 1, following the precedent of #25413, where concat
with keys was changed to raise an informative ValueError instead of an AssertionError.
Happy to switch it to option 2 if maintainers prefer consistency with unstack.
Installed Versions
Details
Built from source at current main (1b1e271), Python 3.13.1,
NumPy 2.5.1, Windows 11 (x86_64, MSVC build).
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
Passing a
levellist that contains duplicate entries toDataFrame.stackfails with aninternal error instead of a user-facing one:
stack_v3computes the columns to stack from the deduplicatedset(level):but later branches on the length of the original, non-deduplicated list:
With
level=[0, 0],set_levelshas one entry, sostack_colsis a plainIndex, whilelen(level)is still 2 and the assertion fires.Note that the duplicates need not be written literally.
DataFrame.stackresolves levelnames and negative level numbers with
_get_level_numberbefore reachingstack_v3, so on3-level columns
["l0", "l0"],[1, -2]and[2, -1]all normalize to duplicated levelnumbers and hit the same code path.
This is also a change in behavior from the previous implementation, which accepted duplicate
levels and stacked sequentially:
Expected Behavior
An informative exception rather than a bare
AssertionErroror anIndexErrorfrom aninternal
list.pop.There are two defensible behaviors, so I wanted to raise it rather than pick one silently:
ValueError. The current implementation stacks the specifiedoriginal column levels simultaneously, and "the same level twice" has no meaning in that
model.
stack_v3already raisesValueError("Columns with duplicate values are not supported in stack")for the analogous duplicate-column case. This also avoids silentlychoosing semantics that differ from the legacy implementation, which stacked two distinct
levels for
[0, 0].stack([0, 0])behaves asstack([0]). This would match thesibling APIs, which all accept duplicate levels today:
unstack([0, 0]),groupby(level=[0, 0]),droplevel([0, 0])andreorder_levelsall succeed. Thedownside is that it silently changes the legacy result rather than flagging it.
I have a PR ready implementing option 1, following the precedent of #25413, where
concatwith
keyswas changed to raise an informativeValueErrorinstead of anAssertionError.Happy to switch it to option 2 if maintainers prefer consistency with
unstack.Installed Versions
Details
Built from source at current
main(1b1e271), Python 3.13.1,NumPy 2.5.1, Windows 11 (x86_64, MSVC build).