`HfApi.dataset_info` is case insensitive

[polinaeterna]

Describe the bug

HfApi.dataset_info gives the same info for names cased differently which leads to returning info for datasets that do not exist.

For example, HfApi.dataset_info('MBZUAI/Bactrian-X') and HfApi.dataset_info('mbzuai/bactrian-x') give the same info while the second dataset mbzuai/bactrian-x doesn't even exist. This leads to weird behavior in the datasets library (datasets are trying to be loaded with different loaders while the second one shouldn't be loaded at all).

Reproduction

from huggingface_hub import HfApi

api = HfApi("https://huggingface.co")

info1 = api.dataset_info('MBZUAI/Bactrian-X')
info2 = api.dataset_info('mbzuai/bactrian-x')
info3 = api.dataset_info('MbZuAi/bactrian-X')  # any random casing

info1.id == info2.id == info3.id
>> True
info1.sha == info2.sha == info3.sha
>> True

Logs

No response

System info

- huggingface_hub version: 0.14.1
- Platform: Linux-5.14.0-1059-oem-x86_64-with-glibc2.31
- Python version: 3.9.16
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/polina/.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers: store
- FastAI: N/A
- Tensorflow: 2.11.0
- Torch: 1.13.1
- Jinja2: 3.1.2
- Graphviz: N/A
- Pydot: N/A
- Pillow: 9.4.0
- hf_transfer: N/A
- gradio: N/A
- ENDPOINT: https://huggingface.co
- HUGGINGFACE_HUB_CACHE: /home/polina/.cache/huggingface/hub
- HUGGINGFACE_ASSETS_CACHE: /home/polina/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/polina/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False

[Wauplin]

Arg yeah I see your problem. Actually the Hub API is case-insensitive on the server by design (cc @julien-c). I don't think we want to fix this in huggingface_hub as it seems a quite specific use case (checking info about a dataset and wanting to fail if it's lowercased).

What datasets could do is to check the id value returned by the server, that will always be the correct one:

>>> from huggingface_hub import dataset_info
>>> dataset_info('mbzuai/bactrian-x').id
'MBZUAI/Bactrian-X'

[julien-c]

i know we discussed this before, but i don't remember where/the outcome of the discussion 😅 cc @Pierrci @coyotte508 just in case

For me, i wouldn't be shocked if the API was case-sensitive, i.e. if dataset_info('mbzuai/bactrian-x') would fail with a 404

If we don't want to change the server API (leading to possible unintended breakage side effects), possibly erroring out in huggingface_hub OR doing what you suggest, are valid options IMO

[julien-c]

But yes in any case simplest / most natural fix is to add a check in datasets

[polinaeterna]

Actually the Hub API is case-insensitive on the server by design (cc @julien-c).

Ah okay. What is the reason for it?
To me it looks a bit confusing that I can access datasets that basically don't exist on the Hub with API.

it seems a quite specific use case (checking info about a dataset and wanting to fail if it's lowercased).

I assume it might be the case not only with .dataset_info() but I haven't checked.
From the datasets side, it also leads, for example, to that when I push to the repo named like some other repo but with another casing, I push to the same repo. Like if push a dataset to "username/DSNAME" and then another dataset to "username/DsNaMe", the last push will rewrite the first repository ("username/DSNAME")