v1.1.4
This is a minor release with several significant improvements over v1.1.3 that do not break the API. Most of the changes revolve around improving the filesystem clients and their performance.
- Improve performance of all
Duct
instances by skipping connection checks for every operation, and instead only check if actions fail. This was achieved by replacingself.connect()
patterns with arequires_connection
decorator, which only lazily checks the connection. - Expose a wrapper around all Omniduct
FileSystemClient
implementations for use withpyarrow
as_.pyarrow_fs
. - Fix serialization of
pandas
dataframes when using filehandles. - Improve the robustness of cached methods when the serializer fails to serialize (or deserialize) objects.
FileSystemFile
fixes and improvements:- Fix a
seek
implementation bug wherebywhence
was not passed through to the internal buffer. - Fix the spelling of the
writable
attribute: "writeable" -> "writable" - Add a few extra methods to the API to match that offered by
io
file implementations.
- Fix a
S3Client
fixes and improvements:- Fix enumerating root "directory".
- Add support for passing in a pre-configured boto3 session.
- Refactor class to allow for subclasses to override boto3 session generation.