Dear experts, this issue is to initiate the feature addition that supports the sparse CNN models built from the sparsepixels library, which is the code implementation of the paper arXiv:2512.06208. Please see below the descriptions. Thanks.
Details
New behavior
Add end-to-end conversion of sparse CNN models, built with the sparsepixels Keras package (HGQ2-based), to HLS firmware via the Vivado and Vitis backends. Sparse CNNs propagate only active pixels (those above a threshold) plus their spatial hash coordinates through the network instead of operating on the full image grid.
Motivation
For inputs with naturally sparse activation patterns (e.g., particle detector data, neutrino events, sparse imaging), sparse convolution drastically reduces FPGA latency and resource usage compared to dense convolution.
Parts of hls4ml being affected
- Five new layer classes in
hls4ml/model/layers.py: SparseInputReduce, SparseConv2D, SparseActivation, SparsePooling2D, SparseFlatten
- Keras v3 handlers for the sparse-pixels layers (
InputReduce, QConv2DSparse, AveragePooling2DSparse)
- Vivado backend: graph optimizer, input-precision fix pass, and config/function templates for each sparse layer
- New HLS kernel header
nnet_sparsepixels.h
- Bit-exact precision dispatches for all five sparse layer types
Patch is fully self-contained. Every new pass and dispatch is gated on the new layer types via isinstance, so no existing model conversion path is touched.
Dear experts, this issue is to initiate the feature addition that supports the sparse CNN models built from the
sparsepixelslibrary, which is the code implementation of the paper arXiv:2512.06208. Please see below the descriptions. Thanks.Details
New behavior
Add end-to-end conversion of sparse CNN models, built with the
sparsepixelsKeras package (HGQ2-based), to HLS firmware via the Vivado and Vitis backends. Sparse CNNs propagate only active pixels (those above a threshold) plus their spatial hash coordinates through the network instead of operating on the full image grid.Motivation
For inputs with naturally sparse activation patterns (e.g., particle detector data, neutrino events, sparse imaging), sparse convolution drastically reduces FPGA latency and resource usage compared to dense convolution.
Parts of hls4ml being affected
hls4ml/model/layers.py:SparseInputReduce,SparseConv2D,SparseActivation,SparsePooling2D,SparseFlattenInputReduce,QConv2DSparse,AveragePooling2DSparse)nnet_sparsepixels.hPatch is fully self-contained. Every new pass and dispatch is gated on the new layer types via
isinstance, so no existing model conversion path is touched.