Skip to content

v1.1.4

Compare
Choose a tag to compare
@matthewwardrop matthewwardrop released this 10 Apr 03:55

This is a minor release with several significant improvements over v1.1.3 that do not break the API. Most of the changes revolve around improving the filesystem clients and their performance.

  • Improve performance of all Duct instances by skipping connection checks for every operation, and instead only check if actions fail. This was achieved by replacing self.connect() patterns with a requires_connection decorator, which only lazily checks the connection.
  • Expose a wrapper around all Omniduct FileSystemClient implementations for use with pyarrow as _.pyarrow_fs.
  • Fix serialization of pandas dataframes when using filehandles.
  • Improve the robustness of cached methods when the serializer fails to serialize (or deserialize) objects.
  • FileSystemFile fixes and improvements:
    • Fix a seek implementation bug whereby whence was not passed through to the internal buffer.
    • Fix the spelling of the writable attribute: "writeable" -> "writable"
    • Add a few extra methods to the API to match that offered by io file implementations.
  • S3Client fixes and improvements:
    • Fix enumerating root "directory".
    • Add support for passing in a pre-configured boto3 session.
    • Refactor class to allow for subclasses to override boto3 session generation.