Bind a custom exception class, with attributes/methods

[Sarcasm]

Hello,
I asked quickly on gitter (https://gitter.im/pybind/Lobby?at=5a7b7f1c6117191e610a8304),
but I feel like this may take a bit longer to resolve.

I'd like to bind a custom exception class to Python.
This custom exception class has attributes,
which I want to expose to Python.

This is very similar to the std::system_error,
which has a code() member function:

• http://en.cppreference.com/w/cpp/error/system_error/code

If this issue is sorted out,
maybe as a result a std::system_error binding could be provided?
There was interests shown here:

• Support for custom exception types #140

My issue right now, is how to inherit a Python built-in type, more precisely, PyExc_RuntimeError.
I'm not yet trying to register a custom translator.

If I attempt to use the code proposed on Gitter:

py::class_<CppException>(m, "PyException",
    py::reinterpret_borrow<py::object>(PyExc_RuntimeError))
        .def(...)

I get an assertion error when importing the module:

python: pybind11/detail/class.h:591:
PyObject* pybind11::detail::make_new_python_type(const pybind11::detail::type_record&):
Assertion `rec.dynamic_attr ?  (((type)->tp_flags & ((1L<<14))) != 0) 
                            : !(((type)->tp_flags & ((1L<<14))) != 0)' failed.

I have a little piece of code that I use to understand how things work,
when inheriting a PyObject *.

#include <pybind11/pybind11.h>

namespace py = pybind11;

struct FooBase { int base_value = 444; };
struct Foo // : FooBase // 1. commented on purpose
{
    int value = 42;
};

PYBIND11_MODULE(p11, m)
{
    auto pyfoobase = py::class_<FooBase>(m, "PyFooBase");
    pyfoobase.def(py::init<>());
    pyfoobase.def_readonly("base_value", &FooBase::base_value);

    PyObject* obj = pyfoobase.ptr();
    // obj = PyExc_RuntimeError; // 2. uncomment to get the assertion error

    py::class_<Foo>(m, "PyFoo", py::reinterpret_borrow<py::object>(obj))
        .def(py::init<>())
        .def_readonly("value", &Foo::value);
}

If I run this code, everything compiles
even if FooBase is not a base class of Foo.
However, if I run the code, base_value returns 42, not 444:

$ python -c 'import p11; print(p11.PyFoo().base_value)'
42

If I uncomment struct Foo // : FooBase => struct Foo : FooBase,
I get the expected behavior:

$ python -c 'import p11; print(p11.PyFoo().base_value)'
444

It kind of make sense to me,
but I'm wondering what to do if I wanted to use a Python builtin type as a base,
instead of my FooBase class:

- PyObject* obj = pyfoobase.ptr();
+ PyObject* obj = PyExc_RuntimeError;

How does my PyFoo type store the base class data?
In case of FooBase, the data is stored when Foo inherits FooBase.
If I inherit PyExc_RuntimeError, what would be my base class?
I don't really want add anything Python to my base class,
so I assume the right thing to do would be to give this base class information
only to the PyFoo wrapper.

How can I inherit a builtin Python type such as PyExc_RuntimeError?

Related issue:

• Subclass python class in C++ #1193

[wojdyr]

Regarding std::system_error that you mentioned – I'm converting it to IOError which also stores errno:

  py::register_exception_translator([](std::exception_ptr p) {
    try {
      if (p) std::rethrow_exception(p);
    } catch (const std::system_error &e) {
      const int errornum = e.code().value();
      PyErr_SetObject(PyExc_IOError, py::make_tuple(errornum, e.what()).ptr());
    }
  });

In Python 3 IOError is an alias for OSError, and depending on the error code the actual exception raised can be FileNotFoundError, PermissionError or other exceptions derived from OSError.
What is not ideal here is that IOError often has the filename property set, but std::system_error has no such property.

[earonesty]

ran into this as well. maybe can formalize this hack, where user specifys that a class is an exception, and we register a pure python class derived from both it and Exception that preserves the initializer (super().__init passthrough). then it will pass isinstance() checks but should have all the same functionality as the exposed class. and there's no change the the rest of pybind11

having exceptions that expose some important fields (like an error code or conflicting name or whatever) isn't all that rare

my current solution is to:

1. py::exec(...create a custom exception class in python..., py::globals())

2.    // register the translator to use it (no captures allowed here, so we have to find the class we made above by name)
      py::register_exception_translator([](std::exception_ptr p) {
          try {
              if (p) std::rethrow_exception(p);
          } catch (const MyCppEx &e) {
              py::object obj = py::module_::import("thismodulename").attr("MyPythonEx")(...params from e...);
              PyErr_SetObject(PyExc_RuntimeError, obj.ptr());
          }
      });
   

[feltech]

@earonesty - just wanted to say thanks for sketching this workaround. Took us a while to work out the kinks, but we've implemented it successfully and it seems to be working out so far.

[battleguard]

@feltech @earonesty thanks for the great examples. I had some issues following it so I created a minset example here for people to leverage.

#include <pybind11/pybind11.h>
#include <pybind11/eval.h>

namespace py = pybind11;

PYBIND11_MODULE(pybind11_example, m)
{
   struct BatchElementException : std::runtime_error
   {
      BatchElementException(std::size_t idx, const std::string& aMessage)
         : std::runtime_error(aMessage), mIndex{ idx } {}

      std::size_t mIndex;
   };

   struct DerivedBatchException : BatchElementException
   {
      using BatchElementException::BatchElementException;
   };


   py::exec(R"pybind(
class BatchElementException(RuntimeError):
    def __init__(self, index: int, aMessage: str):
        self.index = index
        super().__init__(aMessage)
    def get_index(self) -> int:
        return self.index      
)pybind",
m.attr("__dict__"), m.attr("__dict__"));

   // Retrieve a handle the the exception type just created by executing the string literal above.
   const py::object exceptionBase = m.attr("BatchElementException");

   // Register Derived Variants. Note: any derived classes with more methods will need a new py::exec variant
   // note: no need to register BatchElementException since this was done in py::exec
   py::register_exception<DerivedBatchException>(m, "DerivedBatchException", exceptionBase);

   // Register a function that will translate our C++ exceptions to the
   // appropriate Python exception type.
   //
   // Note that capturing lambdas are not allowed here, so we must
   // `import` the exception type in the body of the function.
   py::register_exception_translator([](std::exception_ptr p) {

      const auto setPyException = [](const char* pyTypeName, const auto& exc) {
         const py::object pyClass = py::module_::import("pybind11_example").attr(pyTypeName);
         const py::object pyInstance = pyClass(exc.mIndex, exc.what());
         PyErr_SetObject(pyClass.ptr(), pyInstance.ptr());
      };

      // Handle the different possible C++ exceptions, creating the
      // corresponding Python exception and setting it as the active
      // exception in this thread.
      try {
         if (p) std::rethrow_exception(p);
      }
      catch (const DerivedBatchException& exc) {
         setPyException("DerivedBatchException", exc);
      }
      catch (const BatchElementException& exc) {
         setPyException("BatchElementException", exc);
      }
      });
   m.def("raise_batch_element_exception", []()
      {
         throw BatchElementException(10, "HelloWorld");
      });
   m.def("raise_derived_exception", []()
      {
         throw DerivedBatchException(20, "GoodbyeWorld");
      });
}

// pybind11_example.pyi from pybind11_stubgen pybind11_example -o .
from __future__ import annotations
__all__ = ['BatchElementException', 'DerivedBatchException', 'raise_batch_element_exception', 'raise_derived_exception']
class BatchElementException(RuntimeError):
    def __init__(self, index: int, aMessage: str):
        ...
    def get_index(self) -> int:
        ...
class DerivedBatchException(BatchElementException):
    pass
def raise_batch_element_exception() -> None:
    ...
def raise_derived_exception() -> None:
    ...

// example.py
import pybind11_example as py

try:
    py.raise_batch_element_exception()
except py.BatchElementException as e:
    print(f'Exception {e.__class__.__name__} caught! index={e.get_index()} msg={e}')

try:
    py.raise_derived_exception()
except py.DerivedBatchException as e:
    print(f'Exception {e.__class__.__name__} caught! index={e.get_index()} msg={e}')

# output.log
Exception BatchElementException caught! index=10 msg=HelloWorld
Exception DerivedBatchException caught! index=20 msg=GoodbyeWorld