[ONNX] Weight compression for opset < 21

[andrey-churkin]

Changes

Replace the MatMul operation with the MatMulNBits operation from ONNX Runtime contrib operators.
See https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md

Reason for changes

The DequantizeLinear operation supports the block_size attribute only starting from opset 21.

Related tickets

Ref: 163946

Tests

[alexsu52]

Please add an unit test for MatMulNBits + pack_int4_to_uint8

[andrey-churkin]

@alexsu52 The req_qa_onnx.csv file was obtained using the following "gold" model:

model_gold = ORTModelForCausalLM.from_pretrained(
    self.fp32_model_dir,
    trust_remote_code=True,
    provider="OpenVINOExecutionProvider"
)

It is identical to ref_qa.csv, which was generated using the model below:

model_gold = OVModelForCausalLM.from_pretrained(
    self.fp32_model_dir,
    trust_remote_code=True,
    load_in_8bit=False,
    compile=False,
    stateful=is_stateful,
    ov_config={"KV_CACHE_PRECISION": "f16"},
)

So, if you don't mind, I suggest using only the ref_qa.csv file.

[alexsu52]

ok

[andrey-churkin]

We can't specify the opset version with ORTModelForCausalLM.from_pretrained(). To obtain a model with an opset version lower than 21, we should use the main_export() method (or a similar approach).

[andrey-churkin]

WC run: https://github.com/openvinotoolkit/nncf/actions/runs/15191579763

[MaximProshin]

To be merged after #3510

[andrey-churkin]

WC (Passed): https://github.com/openvinotoolkit/nncf/actions/runs/15251374658