Hardware- and Situation-Aware Sensing for Robust Closed-Loop Control Systems

Sayandip De, Yingkai Huang, Sajid Mohamed, Dip Goswami, Henk Corporaal

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

155 Downloads (Pure)


While vision is an attractive alternative to many sensors targeting closed-loop controllers, it comes with high time-varying workload and robustness issues when targeted to edge devices with limited energy, memory and computing resources. Replacing classical vision processing pipelines, e.g., lane detection using Sobel filter, with deep learning algorithms is a way to deal with the robustness issues while hardware-efficient implementation is crucial for their adaptation for safe closed-loop systems. However, while implemented on an embedded edge device, the performance of these algorithms highly depends on their mapping on the target hardware and situation encountered by the system. That is, first, the timing performance numbers (e.g., latency, throughput) depends on the algorithm schedule, i.e., what part of the AI workload runs where (e.g., GPU, CPU) and their invocation frequency (e.g., how frequently we run a classifier). Second, the perception performance (e.g., detection accuracy) is heavily influenced by the situation - e.g., snowy and sunny weather condition provides very different lane detection accuracy. These factors directly influence the closed-loop performance, for example, the lane-following accuracy in a lane-keep assist system (LKAS). We propose a hardware- and situation-aware design of AI perception where the idea is to define the situations by a set of relevant environmental factors (e.g., weather, road etc. in an LKAS). We design the learning algorithms and parameters, overall hardware mapping and its schedule taking the situation into account. We show the effectiveness of our approach considering a realistic LKAS case-study on heterogeneous NVIDIA AGX Xavier platform in a hardware-in-the-loop framework. Our approach provides robust LKAS designs with 32% better performance compared to traditional approaches.
Original languageEnglish
Title of host publicationProceedings of the 2021 Design, Automation and Test in Europe, DATE 2021
PublisherInstitute of Electrical and Electronics Engineers
Number of pages6
ISBN (Electronic)978-3-9819263-5-4
Publication statusPublished - 16 Jul 2021
EventDesign, Automation and Test in Europe Conference 2021 - Grenoble, France
Duration: 1 Feb 20215 Feb 2021


ConferenceDesign, Automation and Test in Europe Conference 2021


Dive into the research topics of 'Hardware- and Situation-Aware Sensing for Robust Closed-Loop Control Systems'. Together they form a unique fingerprint.

Cite this