feat(pandas): support memtable in pandas backend

[anjakefala]

Closes #5467

(Relatively new contributor, and welcome feedback on anything nuanced that I am missing here!)

[anjakefala]

I see that I have some tests to remove notimpl from; I had only checked in test_generic.py.

It seems I should also remove them for the dask backend.

[cpcloud]

Thanks for the PR!

Except for the XPASSing tests, which are hopefully pretty easy to fix (just remove the corresponding backend from the list!) this looks good to me!

[anjakefala]

Locally, I still have this failure: FAILED ibis/backends/tests/test_client.py::test_interactive_repr_max_columns[pandas-False] - AssertionError: assert ' c_19 ' not in '┏━━━━━━━┳━━...─────┴───┘\n'

I want to see if it also fails in the CI, and I will take a look!

[cpcloud]

I tried out

pytest 'ibis/backends/tests/test_client.py::test_interactive_repr_max_columns[pandas-False]'

and that passes for me.

[cpcloud]

commitlint is failing because

tests(pandas): memtable for pandas backend is now supported

should be

test(pandas): ...

[cpcloud]

Couple of micronits, not blocking.

Merging on green!

[anjakefala]

Also, feel free to squash this PR!

[cpcloud]

I reported the pytest failure upstream: pytest-dev/pytest#10874.

[cpcloud]

No need to squash!

[anjakefala]

As a general FYI, I was also running into a DeprecationWarning with one of our dependencies and pytest locally, that I worked around with pytest -sv -m pandas ibis/backends/tests/ -W ignore::DeprecationWarning

Example:

ibis/backends/tests/test_temporal.py:34: in <module>
    from google.api_core.exceptions import BadRequest as GoogleBadRequest
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/google/api_core/exceptions.py:29: in <module>
    from google.rpc import error_details_pb2
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/google/rpc/__init__.py:18: in <module>
    import pkg_resources
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/pkg_resources/__init__.py:3324: in <module>
    def _initialize_master_working_set():
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/pkg_resources/__init__.py:3298: in _call_aside
    f(*args, **kwargs)
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/pkg_resources/__init__.py:3349: in _initialize_master_working_set
    tuple(dist.activate(replace=False) for dist in working_set)
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/pkg_resources/__init__.py:3349: in <genexpr>
    tuple(dist.activate(replace=False) for dist in working_set)
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/pkg_resources/__init__.py:2870: in activate
    declare_namespace(pkg)
../../miniconda3/envs/ibis-dev/lib/python3.10/site-packages/pkg_resources/__init__.py:2338: in declare_namespace
    warnings.warn(msg, DeprecationWarning, stacklevel=2)
E   DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`

[cpcloud]

That's something we can address in a separate PR, we usually do that by including a specific pattern after the "ignore" string in pyproject.toml so that we aren't propagating a bunch of useless warning spam to users from other libraries.

[cpcloud]

It doesn't seem to be triggering the failure in CI or locally for me, so it could be an environment issue.

[cpcloud]

🚢-ing it!