Efficient Embedded Machine Learning applications using Echo State Networks

Luca Cerina, Marco D. Santambrogio, Giuseppe Franco, Claudio Gallicchio, Alessio Micheli

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

The increasing role of Artificial Intelligence (AI) and Machine Learning (ML) in our lives brought a paradigm shift on how and where the computation is performed. Stringent latency requirements and congested bandwidth moved AI inference from Cloud space towards end-devices. This change required a major simplification of Deep Neural Networks (DNN), with memory-wise libraries or co-processors that perform fast inference with minimal power. Unfortunately, many applications such as natural language processing, time-series analysis and audio interpretation are built on a different type of Artifical Neural Networks (ANN), the so-called Recurrent Neural Networks (RNN), which, due to their intrinsic architecture, remains too complex and heavy to run efficiently on embedded devices. To solve this issue, the Reservoir Computing paradigm proposes sparse untrained non-linear networks, the Reservoir, that can embed temporal relations without some of the hindrances of Recurrent Neural Networks training, and with a lower memory usage. Echo State Networks (ESN) and Liquid State Machines are the most notable examples. In this scenario, we propose a performance comparison of a ESN, designed and trained using Bayesian Optimization techniques, against current RNN solutions. We aim to demonstrate that ESN have comparable performance in terms of accuracy, require minimal training time, and they are more optimized in terms of memory usage and computational efficiency. Preliminary results show that ESN are competitive with RNN on a simple benchmark, and both training and inference time are faster, with a maximum speed-up of 2.35x and 6.60x, respectively.
Original languageEnglish
Title of host publicationProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
EditorsGiorgio Di Natale, Cristiana Bolchini, Elena-Ioana Vatajelu
PublisherInstitute of Electrical and Electronics Engineers
Pages1299-1302
Number of pages4
ISBN (Electronic)978-3-9819263-4-7
DOIs
Publication statusPublished - 15 Jun 2020
Externally publishedYes
Event23rd Design, Automation and Test in Europe Conference and Exhibition (DATE 2020) - Grenoble, France
Duration: 9 Mar 202013 Mar 2020
Conference number: 23

Conference

Conference23rd Design, Automation and Test in Europe Conference and Exhibition (DATE 2020)
Abbreviated titleDATE 2020
Country/TerritoryFrance
CityGrenoble
Period9/03/2013/03/20

Fingerprint

Dive into the research topics of 'Efficient Embedded Machine Learning applications using Echo State Networks'. Together they form a unique fingerprint.

Cite this