Abstract
Deep learning models have replaced conventional methods for machine learning tasks. Efficient inference on edge devices with limited resources is key for broader deployment. In this work, we focus on the tool selection challenge for inference deployment. We present an extensive evaluation of the inference performance of deep learning software tools using state-of-the-art CNN architectures for multiple hardware platforms. We benchmark these hardware-software pairs for a broad range of network architectures, inference batch sizes, and floating-point precision, focusing on latency and throughput. Our results reveal interesting combinations for optimal tool selection, resulting in different optima when considering minimum latency and maximum throughput.
Original language | English |
---|---|
Title of host publication | SCOPES '20: Proceedings of the 23th International Workshop on Software and Compilers for Embedded Systems |
Editors | Sander Stuijk |
Publisher | Association for Computing Machinery, Inc |
Pages | 48-53 |
Number of pages | 6 |
ISBN (Electronic) | 9781450371315 |
DOIs | |
Publication status | Published - 25 May 2020 |
Event | 23rd International Workshop on Software and Compilers for Embedded Systems (SCOPES 2020) - St. Goar, Germany Duration: 25 May 2020 → 26 May 2020 |
Conference
Conference | 23rd International Workshop on Software and Compilers for Embedded Systems (SCOPES 2020) |
---|---|
Country/Territory | Germany |
City | St. Goar |
Period | 25/05/20 → 26/05/20 |