fix: copy client options instead of deepcopy #1130

rild-software · 2025-05-30T05:42:03Z

Deepcopy fails to copy thread objects, like I can't implement a config class that has a client instance of Supabase as a attribute

What kind of change does this PR introduce?

Bug fix for this issue: #1129

What is the current behavior?

SyncClient.init receives a option argument that could be a ClientOptions, this ClientOptions instance can also have specific implementations for storage.

Then, init tries to copy the options attribute to it's instance, since options.headers is a dictionary, a deepcopy garantees recursively that headers is copied too. But this drives to another kind's of unexpected erros since we don't have control of the implementation of any ClientOptions derived class.

The bug is:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/project/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 291, in app
    solved_result = await solve_dependencies(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/fastapi/dependencies/utils.py", line 640, in solve_dependencies
    solved = await run_in_threadpool(call, **solved_result.values)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 37, in run_in_threadpool
    return await anyio.to_thread.run_sync(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/app/core/database.py", line 54, in get_supabase
    return create_client(
           ^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/supabase/_sync/client.py", line 341, in create_client
    return SyncClient.create(
           ^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/supabase/_sync/client.py", line 103, in create
    client = cls(supabase_url, supabase_key, options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/.venv/lib/python3.12/site-packages/supabase/_sync/client.py", line 71, in __init__
    self.options = copy.deepcopy(options)
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 151, in deepcopy
    rv = reductor(4)
         ^^^^^^^^^^^
TypeError: cannot pickle '_thread.RLock' object

You can reproduce my scenario by checking this gist:

https://gist.github.com/rild-software/ad23c35c49f64508cf2cc530e72b703f

What is the new behavior?

init method only copy the contents (not recurively) and then explicitly copy the headers.

…py thread objects, like I can't implement a config class that has a client instance of Supabase as a attribute

rild-software · 2025-05-30T05:43:31Z

@AsahiSoftWareEngineer hope you find time to take a look here, it has total relation with your last week contribution #1095

grdsdev · 2025-06-02T10:49:20Z

Hi @rild-software, thanks for taking the time to fix this.

Asking @silentworks for some guidance on how to better solve this issue, since the last fix didn't produce the desired effect. So let’s think again about how we should handle it this way, or is there a better way?

silentworks · 2025-06-03T22:39:34Z

I will look into this a bit more in the morning. Your example is a bit strange though as you have supabase inside of supabase. Normally if you are creating a storage driver to a database you would use direct database connection string with something like sqlalchemy or psycopg, you wouldn't be doing so with a Restful client library.

ricardodani · 2025-06-05T20:01:59Z

I will look into this a bit more in the morning. Your example is a bit strange though as you have supabase inside of supabase. Normally if you are creating a storage driver to a database you would use direct database connection string with something like sqlalchemy or psycopg, you wouldn't be doing so with a Restful client library.

Sure it's a odd architecture, but I don't think the "weirdness" of this architecture is relevant to this bug (it's a special situation when I wan't to minimize the libraries used and external resources, like relying only in supabase and supabase-py, a serverless minimal code/effort aproach). If the class accepts any StorageClass implementation it should also be able to construct it, no matter how it's implemented. Don't you think?
(Ps: It`s me @rild-software )

ricardodani · 2025-06-05T20:06:10Z

any other specific implementation of a storage class should also face this bug if used a not deepcopyable object inside, like a supabase client instance (my example).
Following "Zen of Python" philosophy: "Explicit is better than implicit". Saying that, it's better to explicitly copy the objects without any underlying recursive deepcopy that we don't have control of what's trying to copy

coveralls · 2025-06-09T00:54:40Z

Pull Request Test Coverage Report for Build 15524373424

Details

6 of 6 (100.0%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 86.216%

Totals
Change from base Build 15236200929:	-0.1%
Covered Lines:	344
Relevant Lines:	399

💛 - Coveralls

silentworks · 2025-06-09T00:56:39Z

This is all looking good to me and I've tested it in my repo which uses Redis for storage and it is indeed working as expected. When I tested my repo with the currently released version (2.15.2) I do get the pickle error for my Redis storage class.

Fixed supabase#1129 by coping only what matters, deepcopy fails to co…

22254d4

…py thread objects, like I can't implement a config class that has a client instance of Supabase as a attribute

grdsdev changed the title ~~Fixed #1129 by coping only what matters~~ fix: by copy client options instead of deepcopy Jun 2, 2025

grdsdev changed the title ~~fix: by copy client options instead of deepcopy~~ fix: copy client options instead of deepcopy Jun 2, 2025

chore: update async version of library with deepcopy to copy change

c3b2d48

silentworks approved these changes Jun 9, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: copy client options instead of deepcopy #1130

fix: copy client options instead of deepcopy #1130

Uh oh!

rild-software commented May 30, 2025 •

edited

Loading

Uh oh!

rild-software commented May 30, 2025

Uh oh!

grdsdev commented Jun 2, 2025

Uh oh!

silentworks commented Jun 3, 2025

Uh oh!

ricardodani commented Jun 5, 2025 •

edited

Loading

Uh oh!

ricardodani commented Jun 5, 2025 •

edited

Loading

Uh oh!

coveralls commented Jun 9, 2025

Uh oh!

silentworks commented Jun 9, 2025

Uh oh!

Uh oh!

Uh oh!

fix: copy client options instead of deepcopy #1130

Are you sure you want to change the base?

fix: copy client options instead of deepcopy #1130

Uh oh!

Conversation

rild-software commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What kind of change does this PR introduce?

What is the current behavior?

What is the new behavior?

Uh oh!

rild-software commented May 30, 2025

Uh oh!

grdsdev commented Jun 2, 2025

Uh oh!

silentworks commented Jun 3, 2025

Uh oh!

ricardodani commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ricardodani commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Jun 9, 2025

Pull Request Test Coverage Report for Build 15524373424

Details

💛 - Coveralls

Uh oh!

silentworks commented Jun 9, 2025

Uh oh!

Uh oh!

rild-software commented May 30, 2025 •

edited

Loading

ricardodani commented Jun 5, 2025 •

edited

Loading

ricardodani commented Jun 5, 2025 •

edited

Loading