neo4j and multiple datasets

[LStromann]

=Hey team,

In a self-hosted installation of Cognee with Neo4j, is it possible to have multiple datasets?

In my tests, I get the impression that when I query a particular dataset, data from all datasets is used to generate the response... Is this intentional ?

For example, if I create a dataset with information about PHP (D1) and another dataset with information about Python (D2), if I ask a question about Python specifying dataset D1, I get an answer, whereas normally it should respond that there is no information about Python in D1.

This discussion was automatically pulled from Discord.

[LStromann]

Neo4j local community edition does not allow creation of a separate database in its docker instance (a dataset is a separate database)

Currently we have Neo4j Aura hosting support for a multi dataset scenario

But it is possible to expand the support to work with Azure/AWS and other platforms to deploy a new Neo4j instance per dataset

For more info on how this could be done you can check our docs on dataset database handlers:

https://docs.cognee.ai/core-concepts/multi-user-mode/dataset-database-handlers/dataset-database-handlers-what-are-they

[LStromann]

If by any chance you write your own dataset database handler to add support for a new hosted scenario a PR on our community repo with the new handler would be greatly appreciated

[LStromann]

Thank you for your reply, it confirms what I've observed ;).

In my project, I need all the data to remain within my infrastructure, so I can't use Neo4j Aura in the cloud. I tested Falkordb with Cognee, but the adapter seems to have several bugs when using graphs. Is there currently a graph database that works with Cognee in a self-hosted production environment?

I did all my testing with the version that uses Kuzu, SQLite, etc., and when I go into production with real data and servers, I encounter either incompatibilities or functionality issues.

[LStromann]

Can you write the bugs you have noticed with the FalkorDB adapter?

If you can deploy a Neo4j instance per dataset in your infra your best bet is to write your own dataset database handler, it's just an interface we have that needs to defines which dataset maps to which database / db instance and can also be written to handle new database creation/deletion

[LStromann]

<@1206965934762496081> what are the issues with Falkor? As for neo4j, I will add next week the community db as an option, was on my list for a while