Abstract
Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018 |
| Editors | Nikos Konofaos, Martin Novotny, Amund Skavhaug |
| Place of Publication | Piscataway |
| Publisher | Institute of Electrical and Electronics Engineers |
| Pages | 357-364 |
| Number of pages | 8 |
| ISBN (Electronic) | 9781538673768 |
| ISBN (Print) | 978-1-5386-7377-5 |
| DOIs | |
| Publication status | Published - 12 Oct 2018 |
| Event | 21st Euromicro Conference on Digital System Design, DSD 2018 - Prague, Czech Republic Duration: 29 Aug 2018 → 31 Aug 2018 Conference number: 21 http://dsd-seaa2018.fit.cvut.cz/dsd/ |
Conference
| Conference | 21st Euromicro Conference on Digital System Design, DSD 2018 |
|---|---|
| Abbreviated title | DSD 2018 |
| Country | Czech Republic |
| City | Prague |
| Period | 29/08/18 → 31/08/18 |
| Internet address |
Fingerprint
Keywords
- Convolutional neural networks
- Fixed-point efficient inference
- Narrow accumulators
- Quantization
Cite this
}
Quantization of constrained processor data paths applied to convolutional neural networks. / de Bruin, E.; Zivkovic, Zoran; Corporaal, H.
Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018. ed. / Nikos Konofaos; Martin Novotny; Amund Skavhaug. Piscataway : Institute of Electrical and Electronics Engineers, 2018. p. 357-364 8491840.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review
TY - GEN
T1 - Quantization of constrained processor data paths applied to convolutional neural networks
AU - de Bruin, E.
AU - Zivkovic, Zoran
AU - Corporaal, H.
PY - 2018/10/12
Y1 - 2018/10/12
N2 - Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.
AB - Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.
KW - Convolutional neural networks
KW - Fixed-point efficient inference
KW - Narrow accumulators
KW - Quantization
UR - http://www.scopus.com/inward/record.url?scp=85056446714&partnerID=8YFLogxK
U2 - 10.1109/DSD.2018.00069
DO - 10.1109/DSD.2018.00069
M3 - Conference contribution
AN - SCOPUS:85056446714
SN - 978-1-5386-7377-5
SP - 357
EP - 364
BT - Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018
A2 - Konofaos, Nikos
A2 - Novotny, Martin
A2 - Skavhaug, Amund
PB - Institute of Electrical and Electronics Engineers
CY - Piscataway
ER -