Skip to content

BUG: DataFrame.stack raises a bare AssertionError when level contains duplicate entries #66588

Description

@Aniket-a14

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd

columns = pd.MultiIndex.from_product(
    [["A", "B"], ["c", "d"], ["e", "f"]], names=["l0", "l1", "l2"]
)
df = pd.DataFrame(np.arange(8).reshape(1, 8), columns=columns)

df.stack([0, 0])        # AssertionError, with no message
df.stack(["l0", "l0"])  # AssertionError, with no message
df.stack([1, -2])       # AssertionError, with no message
df.stack([2, -1])       # IndexError: pop index out of range

Issue Description

Passing a level list that contains duplicate entries to DataFrame.stack fails with an
internal error instead of a user-facing one:

  File "pandas/core/reshape/reshape.py", line 1023, in stack_v3
    assert isinstance(stack_cols, MultiIndex)
AssertionError

stack_v3 computes the columns to stack from the deduplicated set(level):

set_levels = set(level)
stack_cols = frame.columns._drop_level_numbers(
    [k for k in range(frame.columns.nlevels - 1, -1, -1) if k not in set_levels]
)

but later branches on the length of the original, non-deduplicated list:

if len(level) > 1:
    sorter = np.argsort(level)
    assert isinstance(stack_cols, MultiIndex)

With level=[0, 0], set_levels has one entry, so stack_cols is a plain Index, while
len(level) is still 2 and the assertion fires.

Note that the duplicates need not be written literally. DataFrame.stack resolves level
names and negative level numbers with _get_level_number before reaching stack_v3, so on
3-level columns ["l0", "l0"], [1, -2] and [2, -1] all normalize to duplicated level
numbers and hit the same code path.

This is also a change in behavior from the previous implementation, which accepted duplicate
levels and stacked sequentially:

>>> df.stack([0, 0], future_stack=False)  # legacy implementation
l2       e  f
  l0 l1
0 A  c   1  2
     d   3  4
  B  c   5  6
     d   7  8

>>> df.stack(0).stack(0)  # identical result

Expected Behavior

An informative exception rather than a bare AssertionError or an IndexError from an
internal list.pop.

There are two defensible behaviors, so I wanted to raise it rather than pick one silently:

  1. Raise an informative ValueError. The current implementation stacks the specified
    original column levels simultaneously, and "the same level twice" has no meaning in that
    model. stack_v3 already raises ValueError("Columns with duplicate values are not supported in stack") for the analogous duplicate-column case. This also avoids silently
    choosing semantics that differ from the legacy implementation, which stacked two distinct
    levels for [0, 0].
  2. Deduplicate, so that stack([0, 0]) behaves as stack([0]). This would match the
    sibling APIs, which all accept duplicate levels today: unstack([0, 0]),
    groupby(level=[0, 0]), droplevel([0, 0]) and reorder_levels all succeed. The
    downside is that it silently changes the legacy result rather than flagging it.

I have a PR ready implementing option 1, following the precedent of #25413, where concat
with keys was changed to raise an informative ValueError instead of an AssertionError.
Happy to switch it to option 2 if maintainers prefer consistency with unstack.

Installed Versions

Details

Built from source at current main (1b1e271), Python 3.13.1,
NumPy 2.5.1, Windows 11 (x86_64, MSVC build).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions