model-xl8r

xl8r = "translator". A pure-Python, zero-binary-dependency converter hub for ML model weights.

Convert HuggingFace safetensors to and from GGUF, ONNX, OpenVINO IR, CoreML, MLX, AWQ, CompressedTensors, pytorch.bin — without ever installing the official llama.cpp, onnxruntime, openvino, or coremltools C++ binaries on your build host. Everything runs through PyPI / PyTorch wheels, so it works inside any conda env or container.

Status: alpha. The safetensor2gguf path (Llama-3 family, multimodal-VL split, Q4_0/Q5_0/Q8_0/F16) is production-tested across Chrome, Firefox, Safari Technology Preview WebGPU/WebAssembly. See ROADMAP.md for what's working vs planned.

Install

# Core (safetensors ↔ GGUF only)
pip install model-xl8r

# With format-specific extras
pip install 'model-xl8r[awq]'         # AWQ-quantized safetensors as input
pip install 'model-xl8r[mlx]'         # MLX-quantized safetensors as input (macOS)
pip install 'model-xl8r[ct]'          # CompressedTensors-quantized safetensors as input
pip install 'model-xl8r[onnx]'        # ONNX export/import
pip install 'model-xl8r[openvino]'    # OpenVINO IR export/import
pip install 'model-xl8r[coreml]'      # CoreML export (Apple ANE)
pip install 'model-xl8r[validate]'    # quality-vs-FP32 STS-B / perplexity

# Everything
pip install 'model-xl8r[all]'

Quick start

# HF safetensors → GGUF (Llama-family, multimodal-VL auto-split)
safetensor2gguf \
    --model-dir ~/Projects/models/google_gemma-4-E2B-it \
    --output-dir ~/Projects/models/gemma-4-e2b-gguf \
    --lang-quants q4_0,q8_0,f16

# Multimodal model — produces mmproj + lang GGUFs separately
safetensor2gguf \
    --model-dir ~/Projects/models/nvidia_llama-nemotron-embed-vl-1b-v2 \
    --output-dir ~/Projects/models/nemotron-gguf/embed \
    --mmproj-precisions f16 \
    --lang-quants q8_0,q5_0,q4_0,f16

# Top-level dispatcher (lists subcommands)
xl8r --list

What's implemented today

Direction	Status	Notes
HF safetensors → GGUF	✅ Working	Llama-3-family, VL auto-split (Nemotron-VL pattern), Q4_0/Q5_0/Q8_0/F16/BF16
GGUF → safetensors	🚧 Stub	Vendored from nemotron-encode/scripts/gguf_to_safetensors.py, needs migration
Quality validation	🚧 Stub	Vendored from validate_gguf.py, STS-B correlation vs FP32
Quant recipes (IQ-quants, TurboQuant)	🚧 Stub	Vendored from quantize_model.py

What's planned

See ROADMAP.md for the full plan covering:

• AWQ / MLX / CompressedTensors-quantized safetensors as input formats
• Auto interim-F16 → target quant pipeline
• IQ-quants (IQ2_XXS … IQ4_NL) + TurboQuant variants
• Diffusion (SD/Flux), ColBERT/ColPali, audio/video mmproj, factcheck (T5)
• ONNX / OpenVINO IR / CoreML / pytorch.bin round-trips
• safetensor↔safetensor cross-format (regular ↔ MLX ↔ AWQ ↔ CT)
• NPU/accelerator-specific export targets

Testing philosophy

Each conversion class is tested against a real HF model from ~/Projects/models/, loaded into the upstream, unmodified llama.cpp / ONNX Runtime / OpenVINO / CoreML / MLX runtime, and verified to produce numerically-correct output. Disk space is limited, so we test → prove → discard → next class, keeping a small set of canonical models for re-test.

Sister project: ane-inference

model-xl8r produces the artifacts; ane-inference consumes them on Apple Silicon's Neural Engine via CoreML, with Voyage.ai-compatible REST endpoints for embeddings and rerank.

License

Apache 2.0 — see LICENSE.