Quantization of constrained processor data paths applied to convolutional neural networks

E. de Bruin, Zoran Zivkovic, H. Corporaal

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)
167 Downloads (Pure)

Abstract

Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.

Original languageEnglish
Title of host publicationProceedings - 21st Euromicro Conference on Digital System Design, DSD 2018
EditorsNikos Konofaos, Martin Novotny, Amund Skavhaug
Place of PublicationPiscataway
PublisherInstitute of Electrical and Electronics Engineers
Pages357-364
Number of pages8
ISBN (Electronic)9781538673768
ISBN (Print)978-1-5386-7377-5
DOIs
Publication statusPublished - 12 Oct 2018
Event21st Euromicro Conference on Digital System Design, DSD 2018 - Prague, Czech Republic
Duration: 29 Aug 201831 Aug 2018
Conference number: 21
http://dsd-seaa2018.fit.cvut.cz/dsd/

Conference

Conference21st Euromicro Conference on Digital System Design, DSD 2018
Abbreviated titleDSD 2018
CountryCzech Republic
CityPrague
Period29/08/1831/08/18
Internet address

Fingerprint

Neural networks
Image classification
Computer vision
Particle accelerators
Processing

Keywords

  • Convolutional neural networks
  • Fixed-point efficient inference
  • Narrow accumulators
  • Quantization

Cite this

de Bruin, E., Zivkovic, Z., & Corporaal, H. (2018). Quantization of constrained processor data paths applied to convolutional neural networks. In N. Konofaos, M. Novotny, & A. Skavhaug (Eds.), Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018 (pp. 357-364). [8491840] Piscataway: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/DSD.2018.00069
de Bruin, E. ; Zivkovic, Zoran ; Corporaal, H. / Quantization of constrained processor data paths applied to convolutional neural networks. Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018. editor / Nikos Konofaos ; Martin Novotny ; Amund Skavhaug. Piscataway : Institute of Electrical and Electronics Engineers, 2018. pp. 357-364
@inproceedings{59298778fafa4cb1b72dcfd81d60ad68,
title = "Quantization of constrained processor data paths applied to convolutional neural networks",
abstract = "Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1{\%} of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.",
keywords = "Convolutional neural networks, Fixed-point efficient inference, Narrow accumulators, Quantization",
author = "{de Bruin}, E. and Zoran Zivkovic and H. Corporaal",
year = "2018",
month = "10",
day = "12",
doi = "10.1109/DSD.2018.00069",
language = "English",
isbn = "978-1-5386-7377-5",
pages = "357--364",
editor = "Nikos Konofaos and Martin Novotny and Amund Skavhaug",
booktitle = "Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018",
publisher = "Institute of Electrical and Electronics Engineers",
address = "United States",

}

de Bruin, E, Zivkovic, Z & Corporaal, H 2018, Quantization of constrained processor data paths applied to convolutional neural networks. in N Konofaos, M Novotny & A Skavhaug (eds), Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018., 8491840, Institute of Electrical and Electronics Engineers, Piscataway, pp. 357-364, 21st Euromicro Conference on Digital System Design, DSD 2018, Prague, Czech Republic, 29/08/18. https://doi.org/10.1109/DSD.2018.00069

Quantization of constrained processor data paths applied to convolutional neural networks. / de Bruin, E.; Zivkovic, Zoran; Corporaal, H.

Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018. ed. / Nikos Konofaos; Martin Novotny; Amund Skavhaug. Piscataway : Institute of Electrical and Electronics Engineers, 2018. p. 357-364 8491840.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Quantization of constrained processor data paths applied to convolutional neural networks

AU - de Bruin, E.

AU - Zivkovic, Zoran

AU - Corporaal, H.

PY - 2018/10/12

Y1 - 2018/10/12

N2 - Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.

AB - Artificial Neural Networks (NNs) can effectively be used to solve many classification and regression problems, and deliver state-of-the-art performance in the application domains of natural language processing (NLP) and computer vision (CV). However, the tremendous amount of data movement and excessive convolutional workload of these networks hampers large-scale mobile and embedded productization. Therefore these models are generally mapped to energy-efficient accelerators without floating-point support. Weight and data quantization is an effective way to deploy high-precision models to efficient integer-based platforms. In this paper a quantization method for platforms without wide accumulation registers is being proposed. Two constraints to maximize the bit width of weights and input data for a given accumulator size are introduced. These constraints exploit knowledge about the weight and data distribution of individual layers. Using these constraints, we propose a layer-wise quantization heuristic to find a good fixed-point network approximation. To reduce the number of configurations to consider, only solutions that fully utilize the available accumulator bits are being tested. We demonstrate that 16-bit accumulators are able to obtain a Top-1 classification accuracy within 1% of the floating-point baselines on the CIFAR-10 and ILSVRC2012 image classification benchmarks.

KW - Convolutional neural networks

KW - Fixed-point efficient inference

KW - Narrow accumulators

KW - Quantization

UR - http://www.scopus.com/inward/record.url?scp=85056446714&partnerID=8YFLogxK

U2 - 10.1109/DSD.2018.00069

DO - 10.1109/DSD.2018.00069

M3 - Conference contribution

AN - SCOPUS:85056446714

SN - 978-1-5386-7377-5

SP - 357

EP - 364

BT - Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018

A2 - Konofaos, Nikos

A2 - Novotny, Martin

A2 - Skavhaug, Amund

PB - Institute of Electrical and Electronics Engineers

CY - Piscataway

ER -

de Bruin E, Zivkovic Z, Corporaal H. Quantization of constrained processor data paths applied to convolutional neural networks. In Konofaos N, Novotny M, Skavhaug A, editors, Proceedings - 21st Euromicro Conference on Digital System Design, DSD 2018. Piscataway: Institute of Electrical and Electronics Engineers. 2018. p. 357-364. 8491840 https://doi.org/10.1109/DSD.2018.00069