SpikeVision: A Fully Spiking Neural Network Transformer-Inspired Model for Dynamic Vision Sensors

Research output: Contribution to conferencePaperAcademic

113 Downloads (Pure)

Abstract

Dynamic Vision Sensors (DVS) offer unique advantages, such as high temporal resolution and low power consumption, making them ideal for low-latency, energy-efficient applications. Current techniques frequently underutilize their capabilities because they depend on conventional frame-based deep neural networks, which sacrifice temporal detail and demand high computational resources. In this work, we propose SpikeVision, a Transformer-inspired Spiking Neural Network (SNN) model with a fully event-based encoding and processing strategy tailored for DVS input streams. SpikeVision integrates attention-inspired mechanisms adapted for spiking computations, enabling efficient spatial feature extraction without relying on matrix multiplications while leveraging stateful neurons for temporal event processing. We demonstrate that SpikeVision achieves state-of-the-art classification accuracy (99.3%) on the DVS128 Gesture benchmark while maintaining low energy consumption in Field-Programmable Gate Array (FPGA) implementations, highlighting its potential for real-time, edge-based vision tasks.
Original languageEnglish
Publication statusAccepted/In press - 2024
EventThe Asilomar Conference on Signals, Systems, and Computers - Asilomar, Pacific Grove, United States
Duration: 27 Oct 202430 Oct 2024
https://www.asilomarsscconf.org/

Conference

ConferenceThe Asilomar Conference on Signals, Systems, and Computers
Abbreviated titleACSSC 2024
Country/TerritoryUnited States
CityPacific Grove
Period27/10/2430/10/24
Internet address

Funding

This work was supported by the Dutch Research Council (NWO) IMAGINE project, Grant ID: 17911, KICH1.ST04.22.033.

FundersFunder number
Not addedKICH1.ST04.22.033

    Keywords

    • spiking neural network
    • Transformers
    • dynamic vision sensor
    • Edge AI
    • gesture classification
    • FPGA

    Fingerprint

    Dive into the research topics of 'SpikeVision: A Fully Spiking Neural Network Transformer-Inspired Model for Dynamic Vision Sensors'. Together they form a unique fingerprint.

    Cite this