CATS: Combined Activation and Temporal Suppression for Efficient Network Inference

Zeqi Zhu, Arash Pourtaherian, Luc J.W. Waeijen, Ibrahim Batuhan Akkaya, Egor Bondarau, Orlando Moreira

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Brain-inspired event-driven processors execute deep neural networks (DNNs) in a sparsity-aware manner, leading to superior performance compared to conventional platforms. In the pursuit of higher event sparsity, prior studies suppress non-zero events by either eliminating the intra-frame activations (spatially) or leveraging the redundancy in the inter-frame differences for a video (temporally). However, we have empirically observed that simultaneously enhancing activation and temporal sparsity can lead to a synergistic suppression outcome. To this end, we propose an end-to-end event suppression training approach CATS - Combined Activation and Temporal Suppression for efficient network inference. It utilizes a gradient-based method to search for the optimal temporal thresholds per layer while penalizing the presence of events in both spatial and temporal domains. Our experimental results show that CATS achieves 2 ∼ 6× higher event suppression compared to the inherent ReLU suppression across a wide range of vision applications, consistently outperforming the state-of-the-art (SOTA) methods by a significant margin at all accuracy levels. Furthermore, a case study on the commercial event-driven processor GrAI-VIP highlights that the induced event sparsity in SSD on the EgoHands dataset can be efficiently translated into a performance enhancement of 2.5× in FPS, 2.1× in latency, and 3.8× in energy consumption, while maintaining the model accuracy.

Original languageEnglish
Title of host publication2024 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2024
PublisherInstitute of Electrical and Electronics Engineers
Pages8151-8160
Number of pages10
ISBN (Electronic)9798350318920
DOIs
Publication statusPublished - 9 Apr 2024
Event2024 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2024 - Waikoloa, United States
Duration: 3 Jan 20248 Jan 2024

Conference

Conference2024 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2024
Abbreviated titleWACV 2024
Country/TerritoryUnited States
CityWaikoloa
Period3/01/248/01/24

Bibliographical note

Publisher Copyright:
© 2024 IEEE.

Keywords

  • Applications
  • Embedded sensing / real-time techniques
  • Smartphones / end user devices

Fingerprint

Dive into the research topics of 'CATS: Combined Activation and Temporal Suppression for Efficient Network Inference'. Together they form a unique fingerprint.

Cite this