Skip to content

Commit 9fe6e66

Browse files
authored
Revert info_per_bit tolerance to 1.5
The xbitinfo implementation uses a tolerance factor of 1.5. I lowered the tolerance to 1.1, because I was getting poor results with my test data, but it was pointed out that the problem was that my test data were quantized. Quantization is an open issue with xbitinfo too, and we should address it first there, then patch the codec.
1 parent e9efcc4 commit 9fe6e66

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

numcodecs/bitinfo.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -274,7 +274,7 @@ def get_keepbits(info_per_bit, inflevel=0.99):
274274

275275
def _cdf_from_info_per_bit(info_per_bit):
276276
"""Convert info_per_bit to cumulative distribution function"""
277-
tol = info_per_bit[-4:].max() * 1.1 # reduced from 1.5
277+
tol = info_per_bit[-4:].max() * 1.5
278278
info_per_bit[info_per_bit < tol] = 0
279279
cdf = info_per_bit.cumsum()
280280
return cdf / cdf[-1]

0 commit comments

Comments
 (0)