Document how to access resource versions in _history

[AliakseiT]

Existing readme details that old version of any resource is preserved in _history, but there is no (documented?) interface to access those copies for read or, possibly, purge as outdated.

[AliakseiT]

@FireJuun, if you hint me how it is expected to work, I'll try and then open the PR to document it.

[Dokotela]

So I've actually been working on updating the fhir_db package (I assume that's the one that you're referring to). It's not done yet, but it involves some significant changes (including using Hive instead of Sembast_Sqlflite because it's encrypted by default, and it's much, much faster, and it doesn't require flutter, so you can use it in a pure dart application if you want). It will have historical versions, but I don't remember if I've implemented them yet, and I certainly haven't documented it. I can work on that in the next couple of weeks though. Feel free to take a look at so far here: https://github.com/fhir-fli/fhir_db. And if you have suggestions on how history should be stored/accessed please let me know, I'm open to recommendations (I think currently I have a box for each type of resource, and then just one for all historical versions. The only drawback of Hive is that I can only have a total of 224 box types - although it's incredibly fast so I don't think it will make much of a difference).

[Dokotela]

Wow, yeah. I just looked at the documentation for that package and for the actually how to use it is way out of date.

[AliakseiT]

Thanks, @Dokotela, I'll be happy to jump to the new variant in the next couple of month as such a breaking change fits the roadmap. Do you plan to have a stable release by the end of the summer?

I don't have specific wishes for the history interface. In the medical context it will largely be used to produced audit log once in a while, thats it.

I do, however, have a wish to separate the current box and history box in terms of access. I believe it will greatly speed up search throw entities like observations.

Alternatively, it maybe achieved by allowing custom configuration for the same FHIR entity being saved as a separate box. Example:

• Weight is saved as Observation in Observation box 1
• Workout is saved as Observation in Observation box 2

Considering 224 box types, I would limit overall permitted same type boxes.

[AliakseiT]

Just to be clear: you don't plan a non-breaking variant, do you? One that supports on the fly migration from Sembast to Hive?

[Dokotela]

It should certainly be stable by the end of the summer. Hopefully by the end of July. I think the way I was planning to save things would be all observations saved in box1/observation_box, and then all historical versions (of any resource, so observations, conditions, etc), would be saved in box2/history_box (because usually you're only going to need the newest version, and historical versions, as you say, are mostly for auditing).

How much data are you planning to store locally? Hive handles at least 1GB pretty well (which is about 1 million FHIR resources). It stores it all in memory and backs it up on disk, so it's extremely fast to query.

I tried to keep the function calls very similar to the last iteration, but I'm pretty sure there will still be some breaking changes. I'll go back and look and see if I can fix some of them though (I rarely get any responses from people using the packages, so I never know if it's going to matter to anyone when I change things, lol). It wouldn't be a whole lot of work to also include the old version as well if that would work better for you.

[AliakseiT]

It should certainly be stable by the end of the summer. Hopefully by the end of July. I think the way I was planning to save things would be all observations saved in box1/observation_box, and then all historical versions (of any resource, so observations, conditions, etc), would be saved in box2/history_box (because usually you're only going to need the newest version, and historical versions, as you say, are mostly for auditing).

Perfect solution. If custom boxes for Observation would be supported, it is all I can think off for the future.

How much data are you planning to store locally? Hive handles at least 1GB pretty well (which is about 1 million FHIR resources). It stores it all in memory and backs it up on disk, so it's extremely fast to query.

We aim to stay <1GB / per year.

I tried to keep the function calls very similar to the last iteration, but I'm pretty sure there will still be some breaking changes. I'll go back and look and see if I can fix some of them though (I rarely get any responses from people using the packages, so I never know if it's going to matter to anyone when I change things, lol). It wouldn't be a whole lot of work to also include the old version as well if that would work better for you.

After giving it another thought, I think the migration can be implemented as a client application function (both versions can be used with different namespace). So lets not bother about it for now.

As long as the data fetched and saved is the same, function call interface can be different. We can adjust.

[Dokotela]

@AliakseiT I had a question I was hoping I could pose to you. I don't plan on changing db dramatically from where it is. But I was working on kind of a lighter-weight repository that had fewer features in a different project. I was curious, how many of the methods that exist in FhirDb do you actually use? I'm trying to think of how little I can get away with.

[AliakseiT]

@AliakseiT I had a question I was hoping I could pose to you. I don't plan on changing db dramatically from where it is. But I was working on kind of a lighter-weight repository that had fewer features in a different project. I was curious, how many of the methods that exist in FhirDb do you actually use? I'm trying to think of how little I can get away with.

• We use: init, find, save, delete, deleteSingleType, deleteAllResources, closeAllBoxes. All of those with a cypher obtained with cipherFromKey.
• We limit entities to r4b.
• We also "extended" to allow getting the full history with getHistory as follows

extension on FhirDb {
  Future<Iterable<Map<dynamic, dynamic>>> getHistory([String? password]) async {
    Box<Map<dynamic, dynamic>> box;
    if (!Hive.isBoxOpen('history')) {
      box = await Hive.openBox('history', encryptionCipher: cipherFromKey(key: password));
    } else {
      box = Hive.box('history');
    }
    return box.values;
  }
}