XXX version-specific blurb XXX
-
Multithreading checks only apply to Python defined codecs and filters. Now it is possible to use multithreading with C codecs and filters plugins. See PR #127.
-
New support for dynamic filters registry for Python.
-
Now params for codec and filter plugins are correctly initialized when using
register_codec
andregister_filter
functions. -
Some fixes for Cython 3.0.0. However,compatibility with Cython 3.0.0 is not here yet, so build and install scripts are still requiring Cython<3.
-
Updated to latest C-Blosc2 2.10.1.
-
Updated to latest C-Blosc2 2.10.0.
-
Use the new, fixed bytedelta filter introduced in C-Blosc2 2.10.0.
-
Some small fixes in tutorials.
-
Added a new section of tutorials for a quick get start.
-
Added a new section on how to cite Blosc.
-
New method
interchunks_info
forSChunk
andNDArray
classes. This iterates through chunks for getting meta info, like decompression ratio, whether the chunk is special or not, among others. For more information on how this works see this example. -
Now it is possible to register a dynamic plugin by passing None as the
encoder
anddecoder
arguments in the register_codec function. -
Make shape of scalar slices NDArray objects to follow NumPy conventions. See #117.
-
Updated to latest C-Blosc2 2.9.3.
-
Updated to latest C-Blosc2 2.9.2.
-
Better GIL handling. Thanks to @martaiborra.
- Wheels are not including blosc2.pc (pkgconfig) anymore. For details see: Blosc#111 Thanks to @bnavigator for the PR.
- Updated to latest C-Blosc2 2.9.1.
-
New bytedelta filter. We have blogged about this: https://www.blosc.org/posts/bytedelta-enhance-compression-toolset/. See the examples/ndarray/bytedelta_filter.py for a sample script. We also have a short video on how bytedelta works: https://www.youtube.com/watch?v=5OXs7w2x6nw
-
The compression defaults are changed to get a better balance between compression ratio, compression speed and decompression speed. The new defaults are:
cparams.typesize = 8
cparams.clevel = 1
cparams.compcode = Codec.ZSTD
filters = [Filter.SHUFFLE]
splitmode = SplitMode.ALWAYS_SPLIT
These changes have been based on the mentioned blog post above.
-
dtype.itemsize
will have preference over typesize in cparams (as it was documented). -
blosc2.compressor_list(plugins=False)
do not list codec plugins by default now. If you want to list plugins too, you need to passplugins=True
. -
Internal C-Blosc2 updated to latest version (2.8.0).
-
New
NDArray
class for handling multidimensional arrays using compression. It includes:- Data type handling (fully compatible with NumPy)
- Double partitioning
See examples at: https://github.com/Blosc/python-blosc2/tree/main/examples/ndarray NDarray docs at: https://www.blosc.org/python-blosc2/reference/ndarray_api.html Explanatory video on why double partitioning: https://youtu.be/LvP9zxMGBng Also, see our blog on C-Blosc2 NDim counterpart: https://www.blosc.org/posts/blosc2-ndim-intro/
-
Internal C-Blosc2 bumped to latest 2.7.1 version.
- Add support for user-defined filters and codecs.
- Add arm64 wheels for macosx (this time for real).
- Add arm64 wheels for macosx.
- Add arm64 wheels and remove musl builds (NumPy not having them makes the build process too long).
- Use oldest-supported-numpy for maximum compatibility.
- Updated C-Blosc2 to 2.6.0.
-
Support for Python prefilters and postfilters. With this, you can pre-process or post-process data in super-chunks automatically. This machinery is handled internally by C-Blosc2, so it is very efficient (although it cannot work in multi-thread mode due to the GIL). See the examples/ directory for different ways of using this.
-
Support for fillers. This is a specialization of a prefilter, and it allows to use Python functions to create new super-chunks from different kind of inputs (NumPy, SChunk instances, scalars), allowing computations among them and getting the result automatically compressed. See a sample script in the examples/ directory.
-
Lots of small improvements in the style, consistency and other glitches in the code. Thanks to Dimitri Papadopoulos for hist attention to detail.
-
No need to compile C-Blosc2 tests, benchmarks or fuzzers. Compilation time is much shorter now.
-
Added
cratio
,nbytes
andcbytes
properties toSChunk
instances. -
Added setters for
dparams
andcparams
attributes inSChunk
.
-
Honor nested cparams properties in kwargs.
-
C-Blosc2 upgraded to 2.4.3. It should improve cratio for BloscLZ in combination with bitshuffle.
-
Prefer pack_tensor/save_tensor in benchmarks and examples
- Remove the testing of packing PyTorch or TensorFlow objects during wheels build.
-
New
pack_tensor
,unpack_tensor
,save_tensor
andload_tensor
functions for serializing/deserializing PyTorch and TensorFlow tensor objects. They also understand NumPy arrays, so these are the new recommended ones for serialization. -
pack_array2
do not modify the value of a possiblecparams
parameter anymore. -
The
pack_array2
/save_array
have changed the serialization format to follow the new standard introduced inpack_tensor
. In the futurepack_array2
/save_array
will probably be deprecated, so please change topack_tensor
/save_tensor
as soon as you can. -
The new 'standard' for serialization relies on using the 'pack_tensor' attribute as a
vlmeta
(variable length) metalayer.
- Add
msgpack
as a runtime requirement
-
New
pack_array2()
andunpack_array2()
functions for packing NumPy arrays. Contrarily topack_array()
andunpack_array()
counterparts, the new ones allow for compressing arrays larger than 2 GB in size. -
New
SChunk.to_cframe()
andblosc2.from_cframe()
methods for serializing/deserializingSChunk
instances. -
New
SChunk.get_slice()
,SChunk.__getitem__()
andSChunk.__setitem__()
methods for getting/setting slices from/toSChunk
instances. -
The
compcode
parameter has been renamed tocodec
. ANameError
exception will be raised when using the old name. Please update your code when you see this exception. -
More doc restructurings. Hopefully, they are more pleasant to read now :-)
-
Several leaks fixed. Thanks to Christoph Gohlke.
-
Internal C-Blosc2 updated to 2.3.1
- Internal C-Blosc2 updated to 2.3.0
-
Added a new
blosc2.open(urlpath, mode)
function to be able to open persisted super-chunks. -
Added a new tutorial in notebook format (
examples/tutorial-basics.ipynb
) about the basics of python-blosc2. -
Internal C-Blosc2 updated to 2.2.0
- Internal C-Blosc updated to 2.0.4.
-
New
SChunk
class that allows to create super-chunks. This includes the capability of storing data in 4 different ways (sparse/contiguous and in memory/on-disk), as well as storing variable length metalayers. -
Also, during the construction of a
SChunk
instance, an arbitrarily large data buffer can be given so that it is automatically split in chunks and those are appended to theSChunk
. -
See
examples/schunk.py
andexamples/vlmeta.py
for some examples. -
Documentation of the new API is here: https://www.blosc.org/python-blosc2/python-blosc2.html
This release is the result of a grant offered by the Python Software Foundation to Marta Iborra. A blog entry was written describing the difficulties and relevant aspects learned during the work: https://www.blosc.org/posts/python-blosc2-initial-release/
- Release with C-Blosc 2.0.2 sources and binaries.
- Release with C-Blosc 2.0.1 sources and binaries.
- New versions of Blosc2 library added: plugins and lite.
-
Headers and binaries for the C-Blosc2 library are starting to being distributed inside wheels.
-
Internal C-Blosc2 submodule updated to 2.0.0-rc2.
-
Repeating measurements 4 times in benchmarks so as to get more consistent figures.
- Fix some issues with packaging. See: Blosc#9
-
The functions
compress_ptr
anddecompress_ptr
are replaced by pack and unpack since Pickle protocol 5 comes with out-of-band data. -
The function
pack_array
is equivalent topack
, which accepts any object with attributesitemsize
andsize
. -
On the other hand, the function
unpack
doesn't return a numpy array whereas theunpack_array
builds that array. -
The
compcode
parameter has been renamed tocodec
. ANameError
exception will be raised when using the old name. Please update your code when you see this exception. -
The different codecs are accessible via the
Codec
enumerated. E.g.Codec.LZ4
orCodec.Zlib
-
The different filters are accessible via the
Filter
enumerated. E.g.Filter.SHUFFLE
orFilter.BITSHUFFLE
-
The
blosc.NOSHUFFLE
is replaced by theblosc2.Filter.NOFILTER
. -
A bytearray or NumPy object can be passed to the
blosc2.decompress
function to store the decompressed data.