Skip to content

Implement binary version of make_index_sequence #5751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

rmjarvis
Copy link

@rmjarvis rmjarvis commented Jul 12, 2025

Description

I started getting problems with template-recursion-depth starting with pybind version 3.0. It seems that something caused the N values in pybind11::detail::descr<N> to get significantly larger. doubles went from 7 to 30. And pybind11:array_t went from 30 to 81. For one of my initializers with lots of parameters (including a bunch of numpy arrays), this led to a call of pybind11::detail::make_index_sequence_impl<2251>, which required a very high -ftemplate-recursion-depth value when compiling. (At least 2260, due to a few extra levels beyond the 2251 for this class.)

Even once I made that high enough, I still got warnings of stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely, which is presumably why one of my CI runners crashed trying to compile my code, when it used to work just fine with pybind11 version 2.13.

The underlying problem is that the implementation of make_index_sequence is linear in N, so it requires making N template from 0 to N. With N=2251, this exhausted the default allowed template depth and apparently used up most of the stack.

This PR switches the implementation to a binary version whose complexity is logarithmic, so it only requires a depth of log(N). For my use case mentioned above, it worked fine with a template depth of only 60, which is much lower than the default.

Suggested changelog entry:

  • Implement binary version of make_index_sequence to reduce template depth requirements for functions with many parameters.

📚 Documentation preview 📚: https://pybind11--5751.org.readthedocs.build/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant