Automated Calibration of an Automotive Thermal Control System using Reinforcement Learning

The complexity of thermal systems for future electric vehicles is increasing to maximize range, prolong battery life and maximize driver comfort. Traditional map-based control approach achieves sub-optimal performance with little robustness to real world operating conditions and requires significantly large calibration times and experiment costs. To address these challenges, new control approaches are needed. Self Learning control shows potential to maximize system performance in real-world operation while determining optimal control settings by interacting with the environment autonomously. Machine Learning methods offer viable solutions for realizing Self Learning control as it utilizes the increasing availability of data to solve complex modeling and decision-making problems. In this paper, we present a Reinforcement Learning (RL)-based automated calibration approach to maximize heatpump efficiency i.e., Coefficient of Performance (COP) in steady-state operation for a battery electric vehicle. A Deep Q-learning algorithm is applied in a simulation environment to learn the policy, which is the reference setpoint for the heat pump control system. This algorithm captures added controller complexity i.e., sensitivity of vehicle speed on heat pump efficiency with minimal calibration effort unlike the benchmark map-based control where the calibration effort increases exponentially for added complexity. This results in improved robustness to known disturbances compared to the benchmark controller. The expert effort required to manually tune the reference setpoint maps in the benchmark process is replaced by significantly less expert for tuning RL policy and less requirement of system knowledge. A significant reduction of 69% in calibration effort is achieved with RL-based calibration process compared to the benchmark process. The control system performance is validated by applying the trained RL policy in simulation over a Worldwide Harmonized Light Vehicles Test Procedure. The simulation results show that RL-based heat pump control system achieves on average an increase of 13.7% in COP and 10.6% lower compressor work compared to the benchmark controller over the test cycle.