feat: new higher-performance rnote file format - #1578
Conversation
…ation, lots left to do
…mplement cli and legacy loading, as well as clean things up
…ole (not finished though)
…nor changes to `set_compression`, added documentation for v1
…ements to save (less unwraps)
|
@flxzt, @Doublonmousse, I'd say this PR is ready for review. Take all the time you need of course, and feel free to ask me for information, changes, etc. 🫡 |
|
Another thought on the atomic saving denomination. That would be the case for a file format where any change we do inside the file is mapped back to the file through an efficient diff (and not a full write of the new engine state). For sqlite (#1173 (comment)), that would be the case because modifications to the file can be mapped back to the file through a push/commit |
I have seen "atomic saving" be used to mean somewhat different things, but for the most part it seems to mean what we are doing here (Geany wiki, 15 year old stackoverflow thread, etc...). A more efficient saving mechanism using diffs might be interesting, but would probably be very complex to implement and then have to deal with forward- and backward-compatibility... That being said, we should maybe tinker a bit more with the implementation as for instance metadata, permissions, and symlinks are not handled... |
|
@Doublonmousse Apologies for the delay, let me know what you think |
|
Atomic FS was split-off into it's own branch (+PR), see #1678 |
The overall goal of this PR is to improve the handling of
.rnotesave files.tl;dr
* This improvement factor only concerns going back and forth between the already loaded bytes of the file and their corresponding
EngineSnapshot, on my system of course, your results may vary.Main changes:
EngineSnapshotinstead of always passing through a compatibility layer.gzip) compression algorithm for Zstandard (zstd).Here's a diagram of the simplified process to get from
EngineSnapshotto a.rnotesave file and vice-versa.Note that this PR is a re-write of #1177, some of the important changes being:
(+)parallelism for serialization and compression(+)the introduction of a file version separate from the version of Rnote(-)scrapped save preferences (not worth complexity cost)(-)scrapped alternative serialization methods (for compatibility reasons)