TY - JOUR
T1 - Dynamic Probabilistic Pruning
T2 - A General Framework for Hardware-Constrained Pruning at Different Granularities
AU - Gonzalez-Carabarin, Lizeth
AU - Huijben, Iris A.M.
AU - Veeling, Bastian
AU - Schmid, Alexandre
AU - van Sloun, Ruud J.G.
PY - 2024/1/1
Y1 - 2024/1/1
N2 - Unstructured neural network pruning algorithms have achieved impressive compression ratios. However, the resulting, typically irregular sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic that diminishes the benefits of unstructured pruning. This has spurred structured coarse-grained pruning solutions that prune entire feature maps or even layers, enabling efficient implementation at the expense of reduced flexibility. Here, we propose a flexible new pruning mechanism that facilitates pruning at different granularities (weights, kernels, and feature maps) while retaining efficient memory organization (e.g., pruning exactly k-out-of-n weights for every output neuron or pruning exactly k-out-of-n kernels for every feature map). We refer to this algorithm as dynamic probabilistic pruning (DPP). DPP leverages the Gumbel-softmax relaxation for differentiable k-out-of-n sampling, facilitating end-to-end optimization. We show that DPP achieves competitive compression ratios and classification accuracy when pruning common deep learning models trained on different benchmark datasets for image classification. Relevantly, the dynamic masking of DPP facilitates for joint optimization of pruning and weight quantization in order to even further compress the network, which we show as well. Finally, we propose novel information-theoretic metrics that show the confidence and pruning diversity of pruning masks within a layer.
AB - Unstructured neural network pruning algorithms have achieved impressive compression ratios. However, the resulting, typically irregular sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic that diminishes the benefits of unstructured pruning. This has spurred structured coarse-grained pruning solutions that prune entire feature maps or even layers, enabling efficient implementation at the expense of reduced flexibility. Here, we propose a flexible new pruning mechanism that facilitates pruning at different granularities (weights, kernels, and feature maps) while retaining efficient memory organization (e.g., pruning exactly k-out-of-n weights for every output neuron or pruning exactly k-out-of-n kernels for every feature map). We refer to this algorithm as dynamic probabilistic pruning (DPP). DPP leverages the Gumbel-softmax relaxation for differentiable k-out-of-n sampling, facilitating end-to-end optimization. We show that DPP achieves competitive compression ratios and classification accuracy when pruning common deep learning models trained on different benchmark datasets for image classification. Relevantly, the dynamic masking of DPP facilitates for joint optimization of pruning and weight quantization in order to even further compress the network, which we show as well. Finally, we propose novel information-theoretic metrics that show the confidence and pruning diversity of pruning masks within a layer.
KW - Deep learning (DL)
KW - Hardware
KW - hardware-oriented pruning
KW - Kernel
KW - model compression
KW - Optimization
KW - Periodic structures
KW - Probabilistic logic
KW - Quantization (signal)
KW - Training
UR - http://www.scopus.com/inward/record.url?scp=85131808874&partnerID=8YFLogxK
U2 - 10.1109/TNNLS.2022.3176809
DO - 10.1109/TNNLS.2022.3176809
M3 - Article
C2 - 35675247
AN - SCOPUS:85131808874
SN - 2162-237X
VL - 35
SP - 733
EP - 744
JO - IEEE Transactions on Neural Networks and Learning Systems
JF - IEEE Transactions on Neural Networks and Learning Systems
IS - 1
M1 - 9790881
ER -