Radar-based hand gesture recognition using spiking neural networks

Ing Jyh Tsang; Federico Corradi; Manolis Sifalakis; Werner Van Leekwijck; Steven Latré

doi:10.3390/electronics10121405

Radar-based hand gesture recognition using spiking neural networks

Ing Jyh Tsang (Corresponding author)
, Federico Corradi
, Manolis Sifalakis
, Werner Van Leekwijck
, Steven Latré

Research output: Contribution to journal › Article › Academic › peer-review

51 Citations (Scopus)

Abstract

We propose a spiking neural network (SNN) approach for radar-based hand gesture recognition (HGR), using frequency modulated continuous wave (FMCW) millimeter-wave radar. After pre-processing the range-Doppler or micro-Doppler radar signal, we use a signal-to-spike conversion scheme that encodes radar Doppler maps into spike trains. The spike trains are fed into a spiking recurrent neural network, a liquid state machine (LSM). The readout spike signal from the SNN is then used as input for different classifiers for comparison, including logistic regression, random forest, and support vector machine (SVM). Using liquid state machines of less than 1000 neurons, we achieve better than state-of-the-art results on two publicly available reference datasets, reaching over 98% accuracy on 10-fold cross-validation for both data sets.

Original language	English
Article number	1405
Number of pages	20
Journal	Electronics
Volume	10
Issue number	12
DOIs	https://doi.org/10.3390/electronics10121405
Publication status	Published - 11 Jun 2021
Externally published	Yes

Funding

Acknowledgments: The resources and services used in this work were provided by the VSC (Flemish Supercomputer Center), funded by the Research Foundation—Flanders (FWO) and the Flemish Government.

Funders
Fonds Wetenschappelijk Onderzoek

Keywords

Hand gesture recognition
Liquid state machine
Radar
Spiking neural network

Access to Document

10.3390/electronics10121405

Cite this

@article{3cb78d772e7e4dcdb4bd6b5c591cc9bd,

title = "Radar-based hand gesture recognition using spiking neural networks",

abstract = "We propose a spiking neural network (SNN) approach for radar-based hand gesture recognition (HGR), using frequency modulated continuous wave (FMCW) millimeter-wave radar. After pre-processing the range-Doppler or micro-Doppler radar signal, we use a signal-to-spike conversion scheme that encodes radar Doppler maps into spike trains. The spike trains are fed into a spiking recurrent neural network, a liquid state machine (LSM). The readout spike signal from the SNN is then used as input for different classifiers for comparison, including logistic regression, random forest, and support vector machine (SVM). Using liquid state machines of less than 1000 neurons, we achieve better than state-of-the-art results on two publicly available reference datasets, reaching over 98\% accuracy on 10-fold cross-validation for both data sets.",

keywords = "Hand gesture recognition, Liquid state machine, Radar, Spiking neural network",

author = "Tsang, \{Ing Jyh\} and Federico Corradi and Manolis Sifalakis and \{Van Leekwijck\}, Werner and Steven Latr{\'e}",

note = "Publisher Copyright: {\textcopyright} 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article.",

year = "2021",

month = jun,

day = "11",

doi = "10.3390/electronics10121405",

language = "English",

volume = "10",

journal = "Electronics",

issn = "2079-9292",

publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",

number = "12",

}

TY - JOUR

T1 - Radar-based hand gesture recognition using spiking neural networks

AU - Tsang, Ing Jyh

AU - Corradi, Federico

AU - Sifalakis, Manolis

AU - Van Leekwijck, Werner

AU - Latré, Steven

PY - 2021/6/11

Y1 - 2021/6/11

N2 - We propose a spiking neural network (SNN) approach for radar-based hand gesture recognition (HGR), using frequency modulated continuous wave (FMCW) millimeter-wave radar. After pre-processing the range-Doppler or micro-Doppler radar signal, we use a signal-to-spike conversion scheme that encodes radar Doppler maps into spike trains. The spike trains are fed into a spiking recurrent neural network, a liquid state machine (LSM). The readout spike signal from the SNN is then used as input for different classifiers for comparison, including logistic regression, random forest, and support vector machine (SVM). Using liquid state machines of less than 1000 neurons, we achieve better than state-of-the-art results on two publicly available reference datasets, reaching over 98% accuracy on 10-fold cross-validation for both data sets.

AB - We propose a spiking neural network (SNN) approach for radar-based hand gesture recognition (HGR), using frequency modulated continuous wave (FMCW) millimeter-wave radar. After pre-processing the range-Doppler or micro-Doppler radar signal, we use a signal-to-spike conversion scheme that encodes radar Doppler maps into spike trains. The spike trains are fed into a spiking recurrent neural network, a liquid state machine (LSM). The readout spike signal from the SNN is then used as input for different classifiers for comparison, including logistic regression, random forest, and support vector machine (SVM). Using liquid state machines of less than 1000 neurons, we achieve better than state-of-the-art results on two publicly available reference datasets, reaching over 98% accuracy on 10-fold cross-validation for both data sets.

KW - Hand gesture recognition

KW - Liquid state machine

KW - Radar

KW - Spiking neural network

UR - https://www.scopus.com/pages/publications/85107524014

U2 - 10.3390/electronics10121405

DO - 10.3390/electronics10121405

M3 - Article

AN - SCOPUS:85107524014

SN - 2079-9292

VL - 10

JO - Electronics

JF - Electronics

IS - 12

M1 - 1405

ER -

Radar-based hand gesture recognition using spiking neural networks

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this