Postgres locking after DELETE /api/v1/datasets

[LStromann]

=I am wondering if anyone has had a similar issue with the Database locking I am having atm with trying to issue the delete dataset api. I am trying to call the http://localhost:9081/api/v1/datasets/{dataset_id} on a locally running docker image of image: cognee/cognee:latest. But every time I call this the image: pgvector/pgvector:pg17 database locks and I have to restart the cognee container and the postgres container in order for cognee to start working again but it means I can never actually delete any data and instead I have to completely delete all the data from the volume and re upload all my main dataset again.

This discussion was automatically pulled from Discord.

[LStromann]

<@338099417620152320> please continue the discussion here.

[LStromann]

thanks for sending the full lock output from the logs -

the delete isn't crashing. It's waiting. postgres is refusing to drop the table because another connection is still open and hasn't finished its work. That open connection is blocking the delete. that's why your request hangs.

can you try this on a completely fresh setup brand new containers, no other requests, no UI calls and then immediately run the DELETE? or could you check anything else running against the same postgres instance? any other services or scripts?

[LStromann]

The only thing accessing cognee is the cognee container itself as I have downed my UI and API. The delete script is running in a shell script and all it does is login to get a token, check the dataset name to find the id and then tries to delete it.

[LStromann]

okay that helps. since it's just the cognee container running can you try one quick thing restart everything and then call the DELETE endpoint directly with a known dataset ID skipping the login + lookup step entirely. Just a single DELETE call. that'll tell us whether it's the delete itself or one of the earlier requests leaving a transaction open.

[LStromann]

No same issue, nothing running at all and the delete still hangs when only trying to delete via the api the only dataset i have in the db e154b314-db06-5018-9515-bfd90f230597

[LStromann]

same kind of lock logs

[LStromann]

and same line in the cognee logs:

cognee_1 | 2026-03-03T12:53:04.315422 [info ] Deleted Kuzu database files at /app/cognee/.cognee_system/databases/51560c0e-8bf0-4c14-b3c0-cacf079aa1b1/e154b314-db06-5018-9515-bfd90f230597.pkl [cognee.shared.logging_utils]

[LStromann]

After restarting both containers and making only a single DELETE /api/v1/datasets/{id} call (no other traffic) the request still hangs. postgres shows cognee sessions sitting idle in transaction after SELECT queries which blocks the DROP TABLE

<@&1459132796818821192> could you take a look? full lock trace above.

[LStromann]

hey <@338099417620152320> , I'll try and take a look at this today, thanks for flagging the issue!

[LStromann]

Thanks <@506030217710665738>

[LStromann]

is it possible to use my postgres db for the graph database provider instead of kuzu and maybe that will fix it?

[LStromann]

I am using postgres for the storage atm but wondering if using it for the graphs (which i thought it was doing tbh until i saw the Deleted Kuzu databse files line in the logs)

[LStromann]

No, we tried looking into the Apache AGE extension for postgres to enable graphs there but the performance for graph operations was not good for our needs

[LStromann]

It is possible to use QDrant for the vector storage instead of PGVector while we look into and resolve the issue with delete you are having

[LStromann]

So you suggest for now just leaving it as Kuzu or look into neptune or something if the software does everything we need. Im testing it atm you see so if neptune only increases performance we can leave that til last?

[LStromann]

In that case I'd stick with kuzu for the graph DB as there is no need for multiple users to read write and access data at the same time, other options would require hosting which would not be cheap. Kuzu is a simple and cheap file based graph DB that performs well in a single user per DB scenario

[LStromann]

So Neo4j or something better for multi tenant scenarios and kuzu for single user access

[LStromann]

Kuzu works fine as well in a multi-tenant / multi-user scenario as long as users are not working on the same dataset (Kuzu graph DB)

As when there is a write operation on the Kuzu DB it creates a lock and other users can't access the data until the write operation is finished (so they have to wait until the lock is released)

[LStromann]

I have switched to using a local neo4j instance for dev and it seems to be working quite well atm. And it allows for the deleting of datasets now since switching.

[LStromann]

We don't have support yet for local neo4j for the multi-tenant/multi-user scenario in Cognee though

only with neo4j aura hosting

As each dataset has to be a new neo4j instance

neo4j doesn't allow creation of new databases in a neo4j instance for their local community edition version.

When you set ENABLE_BACKEND_ACCESS_CONTROL to False in Cognee it also disables the functionality of datasets not just the permission system

[LStromann]

<@338099417620152320> we found and resolved the issue with the dataset deletion for PGVector, it will be part of our next release