You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
My understanding is that Zarr is capable of storing ML model weights in distributed storage. Reading the documentation on filters, it seems like numcodecs already has all the requisite interfaces to support reading and writing bfloat16 datatypes (or really, any other crazy data type researchers find useful for modeling). I'm curious if anyone reading this knows the proper recipe of [codec/filter/etc]s to represent bfloat16 dtypes.
My understanding is that Zarr is capable of storing ML model weights in distributed storage. Reading the documentation on filters, it seems like numcodecs already has all the requisite interfaces to support reading and writing bfloat16 datatypes (or really, any other crazy data type researchers find useful for modeling). I'm curious if anyone reading this knows the proper recipe of [codec/filter/etc]s to represent
bfloat16
dtypes.In think in practice, folks might use this library along with
AsType()
during encoding/decoding: https://github.com/jax-ml/ml_dtypesThe text was updated successfully, but these errors were encountered: