A scenario- and platform-aware design flow for image-based control systems

Sajid Mohamed (Corresponding author), Dip Goswami, Vishak Nathan, Raghu Rajappa, Twan Basten

Research output: Contribution to journalArticleAcademicpeer-review

10 Citations (Scopus)


Image-based control (IBC) systems are increasingly being used in various domains including autonomous driving. The key challenge in IBC is to deal with high computation demand while guaranteeing performance and safety requirements such as stability. While modern industrial heterogeneous platforms, such as NVIDIA Drive, offer the necessary compute power, application development on these platforms with performance and safety guarantees is still challenging. Alternative time-predictable platforms are not yet in widespread use. A typical design flow for IBC systems consists of three distinct elements: (i) mapping tasks onto platform resources; (ii) timing analysis, consisting of task-level worst-case execution time (WCET) analysis and application-level analysis to obtain worst-case performance bounds on aspects such as latency and throughput; (iii) controller design using the obtained performance bounds, ensuring performance and safety. While such a three-step design process is modular in nature, it usually leads to over-dimensioned systems with sub-optimal performance, because task- and/or application-level timing bounds are pessimistic. We present a scenario- and platform-aware design flow for IBC systems that exploits frequently occurring workload scenarios to optimize performance. For industrial platforms, where tight task-level WCET bounds are difficult to obtain, we moreover propose to use frequently occurring task execution times instead of WCET estimates to obtain tight application-level temporal bounds. During controller design, we then optimize performance and guarantee stability by identifying appropriate system scenarios and by designing a switched controller that switches between those scenarios. We illustrate our method considering a predictable multiprocessor system-on-chip platform - CompSOC. We validate the proposed method using hardware-in-the-loop (HiL) experiments with an industrial heterogeneous multiprocessor platform - NVIDIA Drive PX2 - considering a lane keeping assist system (LKAS). We obtain an improved control performance compared to state-of-the-art IBC design.

Original languageEnglish
Article number103037
Number of pages16
JournalMicroprocessors and Microsystems
Publication statusPublished - Jun 2020


  • Hardware-in-the-loop validation
  • Image-based control
  • Multiprocessor implementation
  • Platform-aware design
  • Scenario-based design
  • Switched linear control


Dive into the research topics of 'A scenario- and platform-aware design flow for image-based control systems'. Together they form a unique fingerprint.

Cite this