feat: support zero-point decompression for asymmetric quantization (packed) #463
Conversation
- Fix decompress_weight method in PackedQuantizationCompressor to support unpacking zero-points - Add comprehensive tests for zero-point packing/unpacking with GROUP and CHANNEL strategies - Add end-to-end integration tests for asymmetric quantization workflow - Ensure packed tensors are contiguous for safetensors compatibility Resolves issue referenced in vllm-project/llm-compressor#1704
…move manual creation
…v similarity; cleanup temp usage
Etelis
force-pushed
the
fix/zero-point-decompression
branch
from
2cf7124 to
126fc89
Compare
|
I've addressed all your concerns. |
tests/test_compressors/quantized_compressors/test_packed_asym_decompression.py
Outdated
Show resolved
Hide resolved
tests/test_compressors/quantized_compressors/test_packed_asym_decompression.py
Outdated
Show resolved
Hide resolved
tests/test_compressors/quantized_compressors/test_packed_asym_decompression.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
This looks good pending @dsikka 's comments, thanks for updating!
There was a problem hiding this comment.
LGTM pending @dsikka 's comments! Thank you for this contribution
|
Hi @Etelis - do you think you'll get chance to address the remaining comments to get this PR over the line? |
|
Yes yes sorry. |
|
Addressed review comments: Switched to in-memory methods: Replaced compressor.compress() / compressor.decompress() (disk-based) with ModelCompressor.compress_model() / decompress_model() (in-memory). Removed manual parameter registration: The manual register_parameter() logic is no longer needed since compress_model() / decompress_model() handle parameter management internally. all 4 test cases pass |
|
Please run |
Signed-off-by: Brian Dellabetta <[email protected]>
brian-dellabetta
dismissed stale reviews from dsikka and themself
via
3ffb213
Signed-off-by: Brian Dellabetta <[email protected]>
Signed-off-by: Brian Dellabetta <[email protected]>
Signed-off-by: Brian Dellabetta <[email protected]>
Signed-off-by: Brian Dellabetta <[email protected]>
4 participants
Follow-up to PR #459 (which was closed due to an accidental history rewrite). The branch has been restored to the pre-rewrite tip (2cf7124).
This PR:
Refs vllm-project/llm-compressor#1704.