Explaining complex systems: a tutorial on transparency and interpretability in machine learning models (part I)

This tutorial seeks to serve as a foundational entry point for eXplainable AI (XAI) as a tool to address the inherent black box challenge associated with many machine learning approaches. Also, it is designed to encourage researchers in systems theory to actively engage with the increasing integration of data-driven methods in control design. Indeed, with AI becoming more and more pervasive, understanding the decisions made by these sophisticated models is becoming paramount. The tutorial's motivation stems from the realization that traditional control design is evolving, incorporating data-driven techniques that demand a nuanced understanding to be safely and reliably deployed. Drawing on the connection between XAI and system identification, the session aims to introduce several methodologies that have been developed to shed light on the underlying decision process of machine learning models and make such models more intepretable. By exploring a relatively wide range of XAI methods, particular emphasis is given to more quantitative approaches, which, notably, also exhibit stronger direct connections with the area of system identification. Methods providing local explanations, namely explanations of the model output for a single specific instance the input features, are described first, delving into their application and their relation with system identification. As the tutorial progresses, it tackles more global perspectives, touching upon counterfactual explanations, physics-informed AI and equation discovery. The main goal is to provide insights into the intricacies of quantitative XAI, positioning the audience to better comprehend and leverage the fusion of AI and control design effectively.