resource delay in db : Async wait for [Cancelled]

[nishika26]

Summary

Target issue is #217
Explain the motivation for making this change. What existing problem does the pull request solve?
we'll be implementing a temporary solution using asyncio.wait_for(). This will allow us to:

Start the background task (like creating the collection or updating the vector store).

Wait for the task to complete within a specified timeout. If the task finishes within the timeout, we send a success response. This ensures that the response, which includes the collection ID, will only be sent after the task is completed, so the collection is already present in the database and we avoid 'no row found' errors.

After using Python's time.time() to measure the time taken by the background task, it took around 7.67 seconds to complete when I uploaded an example file used in Dalgo's data analysis feature. Given this, we can estimate a timeout of approximately 15-20 seconds. Any task that finishes within that window will trigger a response before the timeout.

Notes

A few additional changes I’ve made:

• Updated def create_collection to async def create_collection since await can only be used within async def functions. Reference

• Switched to using an async test client to properly test asynchronous route handlers.

• Installed the pytest-asyncio library, which is required for running async tests with pytest.

• Logging few important parameters such as - INFO - Collection created: 425111c7-25a3-42d3-b373-c97202d9aad5 | Time: 7.514757871627808s | Files: 1 |Sizes:[137.57] KB |Types: ['txt']

• To log file size(s) of files being uploaded, this logic was used to first calculate the file size and then the logic was added to core/cloud/storage.py

[codecov]

Codecov Report

Attention: Patch coverage is 88.77551% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
backend/app/core/cloud/storage.py	16.66%	5 Missing ⚠️
backend/app/api/routes/collections.py	88.00%	3 Missing ⚠️
.../api/routes/collections/test_create_collections.py	95.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[avirajsingh7]

Any reason it is here?

[nishika26]

Leaving it for any future reference, although it is self explanatory but still just in case