Skip to content
This repository was archived by the owner on Sep 29, 2025. It is now read-only.

Vlasiator-style sparsity handling and sparse blockwise output from re…#16

Open
alhom wants to merge 5 commits intokstppd:vdf_replacefrom
alhom:vdf_replace
Open

Vlasiator-style sparsity handling and sparse blockwise output from re…#16
alhom wants to merge 5 commits intokstppd:vdf_replacefrom
alhom:vdf_replace

Conversation

@alhom
Copy link
Contributor

@alhom alhom commented Aug 16, 2024

…construct_vdf

This also means that the sparsify function is now a no-op - we can handle that block-wise now. Maybe that should enforce non-negativity instead?

Modularized several functions for easier reuse.

Also has a dangling new DWT in compression methods - no wonder the old one crashes. Still need to wrap that up.

Here's a snippet to test the blockwise ouput.

import pytools as pt, vdf_replace as v, numpy as np
import tools

f = pt.vlsvfile.VlsvReader('/home/mjalho/Documents/ASTERIX/assets/restart.0000400.2024-08-08_15-03-48.vlsv')
sparsity = 1e-16
_, original_vdf,_ = v.vdf_extract.extract(f, 300,sparsity,False)

blocks, block_data, cm_ratio = v.reconstruct_vdf(f, 300, sparsity, v.cm.reconstruct_cid_zfp, sparsify=False)


vids,vvalues = v.blockdata_to_vdfmap(f, blocks, block_data)

reconstructed_vdf = v.vdf_extract.inflate(f, vids, vvalues)

print(reconstructed_vdf.shape)
print(original_vdf.shape)

tools.plot_vdfs(original_vdf, reconstructed_vdf, sparsity,True,"nosparsify")


blocks, block_data, cm_ratio = v.reconstruct_vdf(f, 300, sparsity, v.cm.reconstruct_cid_zfp, sparsify=True)


vids,vvalues = v.blockdata_to_vdfmap(f, blocks, block_data)

reconstructed_vdf = v.vdf_extract.inflate(f, vids, vvalues)

print(reconstructed_vdf.shape)
print(original_vdf.shape)

tools.plot_vdfs(original_vdf, reconstructed_vdf, sparsity,True,"sparsify")

@alhom
Copy link
Contributor Author

alhom commented Aug 16, 2024

Requires this
fmihpc/analysator#258

@kstppd
Copy link
Owner

kstppd commented Aug 16, 2024

Thanks a bunch Markku. Will test this shortly.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet