Researching cognee - how do doc IDs work?

[LStromann]

=Not sure if already there but I can't see on the docs that I can use my own unique ID when indexing documents. That would allow me to maintain a file indexing system based on say filepathname+date on a hash, so when I delete a document I can instruct cognee to remove it. I am not even certain now that when I add a document via API I receive a unique ID to save, if that is so deleting/updating documents seems problematic.

The purpose is to maintain a process that detects new documents... generates my ID for it so I can check if it has been indexed in which case I would need to delete pre insert. Ideally I would like to handle that myself but alterantively at least I need to have a uniqueID return on add so I can store it and use it when I need to delete. Can't see that on the API

This discussion was automatically pulled from Discord.

[LStromann]

hey <@576433490518736974> IDs are automatically generated from the content hash, making them deterministic.
The add API response includes data_ingestion_info with the data_id for each document
GET /v1/datasets/{dataset_id}/data returns all data items with their IDs. Since IDs are content-based, re-adding the same content updates the existing record rather than creating a duplicate.
You can build your tracking system by storing the data_id returned from each add operation.
Would that help?
Otherwise your filepath+date hash would require adding an optional data_id parameter to the add API