Allow iterating and altering storage keys without the need to seek()
all the time #10295
Description
Just wanna propose an idea I had while I was digging into the migration issues we had. Also I am not sure if this is sanely doable because of the FFI requirements or other stuff that is going on when altering the state from the runtime.
The goal of the proposal here would be that translate
or translate_values
on StorageMaps
do not implicitly use the sp_io::storage::next_key
runtime interface anymore. The reason for this is that every call to this interface uses a seek
call on the underlying trie. Which when the trie has a lot of branches, is expensive.
Furthermore, I think from a runtime developer perspective, the current translate
and translate_values
, give you the feel of security to work with some kind of iterator that would cache its location in the trie.
Idea 1: Expose a trait object of Iterator
to the runtime
I played around with this and this seems to be doable but a bad idea as
- FFI with trait objects is complex
- Needs externalities to be passed with
static
lifetime as 'TrieDB' always works with aHashDbRef
. Not even sure, if the reference will be valid for the wasm instance.
Idea 2: Pass a FnMut(&[u8], &[u8])
from the runtime to the state_machine::TrieBackend::for_key_values_with_prefix
call
for_key_values_with_prefix
will execute the closure provided by thetranslate
ortranslate_values
on all keys with the prefix- Needs to extend trait
Externalities
with some method that provides access to this backend call.
In any case, both ideas would need to extend the storage
runtime interface.