Vlasiator-style sparsity handling and sparse blockwise output from re…

[alhom]

…construct_vdf

This also means that the sparsify function is now a no-op - we can handle that block-wise now. Maybe that should enforce non-negativity instead?

Modularized several functions for easier reuse.

Also has a dangling new DWT in compression methods - no wonder the old one crashes. Still need to wrap that up.

Here's a snippet to test the blockwise ouput.

import pytools as pt, vdf_replace as v, numpy as np
import tools

f = pt.vlsvfile.VlsvReader('/home/mjalho/Documents/ASTERIX/assets/restart.0000400.2024-08-08_15-03-48.vlsv')
sparsity = 1e-16
_, original_vdf,_ = v.vdf_extract.extract(f, 300,sparsity,False)

blocks, block_data, cm_ratio = v.reconstruct_vdf(f, 300, sparsity, v.cm.reconstruct_cid_zfp, sparsify=False)


vids,vvalues = v.blockdata_to_vdfmap(f, blocks, block_data)

reconstructed_vdf = v.vdf_extract.inflate(f, vids, vvalues)

print(reconstructed_vdf.shape)
print(original_vdf.shape)

tools.plot_vdfs(original_vdf, reconstructed_vdf, sparsity,True,"nosparsify")


blocks, block_data, cm_ratio = v.reconstruct_vdf(f, 300, sparsity, v.cm.reconstruct_cid_zfp, sparsify=True)


vids,vvalues = v.blockdata_to_vdfmap(f, blocks, block_data)

reconstructed_vdf = v.vdf_extract.inflate(f, vids, vvalues)

print(reconstructed_vdf.shape)
print(original_vdf.shape)

tools.plot_vdfs(original_vdf, reconstructed_vdf, sparsity,True,"sparsify")

[alhom]

Requires this
fmihpc/analysator#258

[kstppd]

Thanks a bunch Markku. Will test this shortly.