Implement binary version of make_index_sequence #5751
+40
−5
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I started getting problems with template-recursion-depth starting with pybind version 3.0. It seems that something caused the N values in
pybind11::detail::descr<N>
to get significantly larger. doubles went from 7 to 30. And pybind11:array_t went from 30 to 81. For one of my initializers with lots of parameters (including a bunch of numpy arrays), this led to a call ofpybind11::detail::make_index_sequence_impl<2251>
, which required a very high-ftemplate-recursion-depth
value when compiling. (At least 2260, due to a few extra levels beyond the 2251 for this class.)Even once I made that high enough, I still got warnings of
stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
, which is presumably why one of my CI runners crashed trying to compile my code, when it used to work just fine with pybind11 version 2.13.The underlying problem is that the implementation of
make_index_sequence
is linear in N, so it requires making N template from 0 to N. With N=2251, this exhausted the default allowed template depth and apparently used up most of the stack.This PR switches the implementation to a binary version whose complexity is logarithmic, so it only requires a depth of log(N). For my use case mentioned above, it worked fine with a template depth of only 60, which is much lower than the default.
Suggested changelog entry:
📚 Documentation preview 📚: https://pybind11--5751.org.readthedocs.build/