crc32c-rs

High-performance CRC32C implementation compliant with RFC 3720 (iSCSI)

Features

• High-performance CRC32C implementation written in Rust

• Runtime SIMD dispatch with CPU-model-tuned implementations (see below)

• Direct access to specific implementations for advanced use cases

Installation

Using pip:

pip install crc32c-rs

Using uv:

uv pip install crc32c-rs

Using poetry:

poetry add crc32c-rs

Examples

Basic usage

from crc32c_rs import crc32c, Hasher

print(crc32c(b"Hello world!"))  # 2073618257

crc = crc32c(b"Hello")
print(crc32c(b" world!", crc))  # 2073618257

h = Hasher()
h.update(b"Hello")
h.update(b" ")
print(h.checksum)  # 4220938453
h.update(b"world")
h.update(b"!")
print(h.checksum)  # 2073618257
print(h.digest())  # b'{\x98\xe7Q'
print(h.hexdigest())  # 7b98e751

By default, crc32c_rs.crc32c selects an implementation at runtime. Dispatch is keyed by CPU model first: the vendor and CPUID model (family/model) are looked up, and if the CPU is a known one, its microarchitecture-tuned implementation is used whenever the required instruction sets are present. Only for unknown CPUs does dispatch fall back to a generic feature-based choice.

x86/x86_64

Model-tuned selection (checked in this order):

CPU model	With AVX-512VL + VPCLMULQDQ	With AVX-512VL + PCLMULQDQ	With SSE4.2 + PCLMULQDQ
Sapphire Rapids	v3s1_s3	×	v8s3x3
Genoa	v3s2x4	×	v1s3x2
Ice Lake	v4s5x3	×	v7s3x3
Cascade Lake	×	v9s3x4e	v8s3x3
Milan	×	×	v1s4x2
Rome	×	×	v1s3x3

Unknown model: AVX-512VL + VPCLMULQDQ -> v4s5x3, else AVX-512VL + PCLMULQDQ -> v9s3x4e, else SSE4.2 + PCLMULQDQ -> v8s3x3, else fallback.

AArch64/ARM64EC

1. CRC + AES + SHA3 -> v9s3x2e_s3
2. CRC + AES -> v12e_v1 on Apple, v3s4x2e_v2 elsewhere
3. fallback

Other platforms

Portable fallback.

Direct implementation selection

The default crc32c_rs.crc32c function selects an implementation at runtime, but you can also call a specific implementation directly:

Function	Required CPU features
crc32c	Runtime CPU dispatch
crc32c_fallback	No special CPU features
crc32c_avx512vl_vpclmulqdq_v3s1_s3	AVX-512F + AVX-512VL + VPCLMULQDQ
crc32c_avx512vl_vpclmulqdq_v3s2x4	AVX-512F + AVX-512VL + VPCLMULQDQ
crc32c_avx512vl_vpclmulqdq_v4s5x3	AVX-512F + AVX-512VL + VPCLMULQDQ
crc32c_avx512vl_pclmulqdq_v9s3x4e	AVX-512VL + PCLMULQDQ
crc32c_sse42_pclmulqdq_v1s3x2	SSE4.2 + PCLMULQDQ
crc32c_sse42_pclmulqdq_v1s3x3	SSE4.2 + PCLMULQDQ
crc32c_sse42_pclmulqdq_v1s4x2	SSE4.2 + PCLMULQDQ
crc32c_sse42_pclmulqdq_v7s3x3	SSE4.2 + PCLMULQDQ
crc32c_sse42_pclmulqdq_v8s3x3	SSE4.2 + PCLMULQDQ
crc32c_aes_crc_v12e_v1	CRC + AES
crc32c_aes_v3s4x2e_v2	CRC + AES
crc32c_aes_sha3_v9s3x2e_s3	CRC + AES + SHA3

Architecture-specific implementations are available only on compatible builds. If the current processor does not support the required features, calling one of these functions raises crc32c_rs.UnsupportedCPUFeatureError.

from crc32c_rs import crc32c_avx512vl_vpclmulqdq_v3s1_s3

checksum = crc32c_avx512vl_vpclmulqdq_v3s1_s3(b"Hello world!")
print(checksum)  # 2073618257

Contributors