fix: download directories with http fs

[frostming]

Signed-off-by: Frost Ming [email protected]

This PR includes two fixes:

• Skip directories when getting file from remote http to local
• Unquote file names when creating local files

[martindurant]

Thanks for the fix! I have a comment about where best to insert it.

On the issue of unquote, do you have an example of the kind of thing you were seeing? This is essentially an HTTP href-> path translation? If yes, maybe it should be done at listing time?

[martindurant]

This will result in essentially every file getting downloaded twice, once to check the directory. This information should already be available in _get(), so we can skip there rather than here. It doesn't really make sense to call _get_file() on a path that is not considered a file.

[frostming]

I tried to exclude dirs from the return result of self._expand_path but it will always add the root to the result, if we do not check is_dir in get_file, it will also fail there.

I think I might need to change more things at this point, which is a bit risky for a first contribution. What is your suggestion?

[frostming]

On the issue of unquote, do you have an example of the kind of thing you were seeing? This is essentially an HTTP href-> path translation? If yes, maybe it should be done at listing time?

I don't think so. ls()/find() might return absolute URLs, and URL encoding is reasonable here. It only needs to be unquoted when downloading to local paths(get)