blocksize vs block_size in HTTPFilesystem

[groutr]

Is there a convention between blocksize and block_size?
Trying to understand this baffling behavior...

import fsspec

fs = fsspec.filesystem('https', block_size=1024*1024)
fs.block_size    # this is 1048576
fs.blocksize      # this is 4194304??

fs.blocksize appears to be inherited from AbstractFileSystem and subsequently seems to be the actual blocksize used for operations. It is confusing to me why there are two different blocksizes. I cannot seem to influence the value of fs.blocksize unless I set the attribute directly.
Is this intended behavior?

EDIT: The behavior is even more baffling when caching is used.

fs = fsspec.filesystem('cached', target_protocol='https', blocksize=1024*1024, block_size=1024*1024)
fs.blocksize   # value of 4194304
fs.block_size  # value of 5242880

This doesn't seem like correct behavior. Can someone please explain what is happening here?

[martindurant]

You are right, there seems to be a bit of a mess!

The most common place where block sizes are relevant is in the File-like API.

• AbstractBufferedFile has a DEFAULT_BLOCK_SIZE class attribute and block_size kwarg, which are combined to a blocksize attribute. This is inherited by many implementations and probably the value you actually care about. The 5MB value was typical everywhere.
• most versions of _open() take block_size as an argument to override whatever the instance default value when passing it to the file's init.
• AbstractFIleSystem has a blocksize class attribute inherited by all implementations.
• Some implementations have another set of variables, e.g., default_block_size is the class attribute, init kwarg and instance attribute in s3fs
• for HTTP alone, the filesystem instance attribute is called block_size and .blocksize is ignored, and DEFAULT_BLOCK_SIZE is a module global.

Some implementations use a concept of "block size" for other operations such as get/put/cat/pipe_file, often as default kwargs on those methods or hard-coded.

Note: for cached, it more or less doesn't matter, because the whole point is to have a local copy of the file rather that an in-memory bytes cache. The only difference it would/should make is during the file copy.

I would be happy to see all of this consolidated to something reasonable!