About DWQ on VLM

[sudo-0x2a]

Hi! I'm wondering if it is technically possible to apply DWQ on a VLM?

[awni]

Yes it's very possible. For DWQ you just need a calibration dataset. So for a VLM it may be important to use some images as well as text in the calibration set.. though you could probably get decent results with just text.

In general I think it's less common to quantize the vision encoder of the model. So mostly it should work the same as an LLM with possible augmentation of the calibration set to include image+text data

[sudo-0x2a]

Thank you awni!
Does the mlx-lm still not support VLM conversion? Since the mlx-vlm does not support quantization techniques like AWQ and DWQ, do you have any ideas to bypass these limitations?