fp8 weight compression into 4 bit types

[alexsu52]

Changes

• Added support for compression of f8e4m3 and f8e5m2 weights into 4-bit types.
• Added skipping compression of f8e4m3 and f8e5m2 weights into 8-bit types.
• Added support of f8e4m3 and f8e5m2 in optimized OpenVINO functions.
• Added weight_dtype in WeightCompressionParameters.
• Refactored WeightCompression algorithm to minimize number of collected statistics.
• Refactored AWQ algorithm by removing nodes_to_compress argument in the apply function.
• Removed the redundant Convert in the compression subgraph for f8e4m3 and f8e5m2 weights.

Reason for changes

Support for DeepSeek-R1

Related tickets

CVS-162315

Tests

tests/openvino/native/quantization/test_weights_compression.py
https://github.com/openvinotoolkit/nncf/actions/runs/15563432412

[ljaljushkin]

Thanks, good to see a lot of improvements in the weight compression algorithm!
Suggest adding the following tests for them:

Added support of f8e4m3 and f8e5m2 in optimized OpenVINO functions.

• Tests for optimized functions can be extended https://github.com/openvinotoolkit/nncf/tree/develop/tests/openvino/optimized_functions

Removed the redundant Convert in the compression subgraph for f8e4m3 and f8e5m2 weights.

• Quite tricky change, tracking number of convert in some test would be helpful.

Refactored WeightCompression algorithm to minimize number of collected statistics.

• It would be helpful to have a test checking number of collected statistics.

[alexsu52]

* Tests for optimized functions can be extended https://github.com/openvinotoolkit/nncf/tree/develop/tests/openvino/optimized_functions

Done: tests/openvino/optimized_functions/test_compression_functions.py

* Quite tricky change, tracking number of convert in some test would be helpful.

This was covered in tests/openvino/native/quantization/test_weights_compression.py

* It would be helpful to have a test checking number of collected statistics.

Done via checking that all statistics are used: tests/cross_fw/test_templates/template_test_weights_compression.py

[nikita-savelyevv]

Overall looks good, some minor comments below

[nikita-savelyevv]

I have a concern that previously this variable was named is_last_layer_shared and from that it was clear why the node is skipped. Now it is less obvious. Perhaps, some comment would help.

[alexsu52]

Added a comment to the code.

[nikita-savelyevv]

I have difficulties matching this logic with the previous one. Where does this come from / what's the logic behind it?

[alexsu52]

This is a fix to the previous logic. The previous logic was that if the last layer is skipped for some reason, the layer preceding the last one is also skipped. The current logic only skips the last layer.

[nikita-savelyevv]

👍

[nikita-savelyevv]

Suggested change

	def itemsize(self) -> int:
	"""
	Returns the size of a single item in bits for the tensor data type.
	:return: The item size in bits.
	"""
	if self in [TensorDataType.nf4, TensorDataType.uint4, TensorDataType.int4]:
	return 4
	if self in [TensorDataType.f8e4m3, TensorDataType.f8e5m2, TensorDataType.int8, TensorDataType.uint8]:
	return 8
	if self in [TensorDataType.float16, TensorDataType.bfloat16]:
	return 16
	if self in [TensorDataType.float32, TensorDataType.int32]:
	return 32
	return 64
	def itemsize(self) -> int:
	"""
	Returns the size of a single item in bits for the tensor data type.
	:return: The item size in bits.
	"""
	return ITEMSIZE_BITS[self]
	ITEMSIZE_BITS = {
	TensorDataType.nf4: 4,
	TensorDataType.uint4: 4,
	TensorDataType.int4: 4,
	TensorDataType.f8e4m3: 8,
	TensorDataType.f8e5m2: 8,
	TensorDataType.int8: 8,
	TensorDataType.uint8: 8,
	TensorDataType.float16: 16,
	TensorDataType.bfloat16: 16,
	TensorDataType.float32: 32,
	TensorDataType.int32: 32,
	TensorDataType.float64: 64,
	TensorDataType.int64: 64,
	}

What do you think about this?

[alexsu52]

Done