Add document-retrieval to Hub as a task

[merveenoyan]

This PR adds document retrieval to Hub for following models that are used heavily and now there's a lot of them:

• MrLight/dse-qwen2-2b-mrl-v1
• vidore/colpali-v1.2
• llamaindex/vdr-2b-multi-v1

Icon looks like this:

Other names I thought of:

• multimodal-feature-extraction (ambiguous, also overlaps with zero shot image classification)
• document-feature-extraction (sounds like plain document embeddings for a document backbone, best if this doesn't happen and those models are covered under image-feature-extraction instead)
• zero-shot-document-classification (too long, sounds odd but would be accurate, but we use these models often for retrieval and not this purpose)
  this name is very to the point so best if it stays like it

[merveenoyan]

@NielsRogge

[merveenoyan]

@pcuenca can you leave a review? 👀💗

[julien-c]

is it not already very close to image-feature-extraction? How many models would be covered by this? (i'm wondering if it isn't too specific)

[julien-c]

(no strong opinion though)

[pcuenca]

Sounds a bit NLP-y to me. Could we use something like "visual-document-retrieval", for symmetry with "visual-question-answering"?

[pcuenca]

How many models would be covered by this? (i'm wondering if it isn't too specific)

I have the same question. Would, for example, all the ColPali models be included here?

[pcuenca]

We'd also have to tag multiple models before merging, as usual.

[merveenoyan]

@pcuenca I wanted to wait for the naming consensus before opening PRs to them. we can do visual-document-retrieval yes.

@julien-c it's actually not. those are singular image backbones used to train traditional vision models. these models on the contrast are zero shot models built on VLMs to do document retrieval on multimodal RAG pipelines. they're a bit like CLIP, but for documents, and they're not used for classification (they have long context length and have fine grained image understanding). the number of models keep increasing as number of VLMs increase and they're all wrongly tagged hence this PR. (there's ColPali, ColQwen, ColSmolVLM, DSE models and more now)

here's an tldr explainer on how they're used https://x.com/mervenoyann/status/1831409380040044762?s=46

[julien-c]

ok, sounds good

[Vaibhavs10]

maybe image-document-retrieval so that it's more in sync with the current task vision/ audio related tasks names?

[Vaibhavs10]

Suggested change

	color: "yellow",
	color: "yellow",
	hideInDatasets: true,

Don't think there are many datasets related to this task?

[merveenoyan]

there's a whole benchmark called ViDoRe and more datasets outside of this benchmark, and similar datasets exist in multiple languages so I would expect more datasets -- https://huggingface.co/spaces/vidore/vidore-leaderboard

for instance these are what I found for Turkish

• https://huggingface.co/datasets/selimc/tr-textbook-ColPali
• https://huggingface.co/datasets/muhammetfatihaktug/bilim_teknik_mini_colpali

[merveenoyan]

I'm not in favor of image-document-retrieval, it sounds a bit like images are involved separately and sounds off imo. we have a similar task actually, visual-question-answering which separates from textual QA