Skip to content

Check write-ahead-logging setup of sqlite #71

@texhnolyze

Description

@texhnolyze

Summary

Currently, we are using WAL for our sqlite data access as it is "is significantly faster in most scenarios" and allows for concurrent non-blocking reads.

However, on generation of DINO embeddings during dataset generation it seems that very big files are generated, and we assume this is due to the unordered data access for the dataset generation leading to entries being in the WAL multiple times.
It should be investigated if WAL is still the best option in this scenario.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    Status

    Ready

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions