No chat template for processor

[dtdo90]

Dear author,

Thank you for the amazing work!!!

I am studying finetuning the Llama 3.2 11B Vision model, but I encounter the following error

ValueError: No chat template is set for this processor. Please either set the chat_template attribute, or provide a chat template as an argument. See https://huggingface.co/docs/transformers/main/en/chat_templating for more information.

Based on ocrvqa_dataset.py, apply_chat_template is a step in processing the data (as in the code snippet below). Very much look forward to your reply!

def tokenize_dialogs(dialogs, images, processor):
    text_prompt = processor.apply_chat_template(dialogs)
    text_prompt = [prompt.replace('<|begin_of_text|>','') for prompt in text_prompt]

[wukaixingxp]

@dtdo90 Can you show me your code that can reproduce this error? Thanks!

[dtdo90]

Thank you for responding!

I found a workaround with the formatted prompt in the form

formatted_prompt=("<|begin_of_text|>"
            f"<|start_header_id|>system<|end_header_id|>{system_message}<|eot_id|>"
            f"<|start_header_id|>user<|end_header_id|>{formatted_prompt}<|image|><|eot_id|>"
            f"<|start_header_id|>assistant<|end_header_id|>{description}<|eot_id|><|end_of_text|>"
)

I have checked again today and found .apply_chat_template work now. I'm not sure what caused the error the other day.

Regarding the formatted prompt by method .apply_chat_template, is there a documentation that I can look up to understand the types of role, or message, or image the processor of Llama supports?

Thank you again for providing us the amazing Llama model and making it open source!