Learning sequential composition control

Sequential composition is an effective supervisory control scheme for addressing control problems in complex dynamical systems. It executes a set of controllers sequentially to achieve a challenging task. These classes of controllers are typically designed offline. Although sequential composition is a well-defined control methodology, it is not designed to address unmodeled situations that might occur during runtime. This paper proposes a learning approach that augments the standard supervisory framework using online learning to handle unpredicted situations. New controllers based on the acquired knowledge via learning are added to the existing supervisory control structure. Since the learning experiments are restricted to explore just within the domain of attraction (DOA) of the existing controllers, it is guaranteed that the learning process is safe. In addition, the DOA of the new learned controller is approximated after each learning trial. Thus, the learning process can last a short amount of time and is terminated as soon as the DOA of the learned controller is sufficiently large. The proposed approach has been implemented on two different nonlinear systems. The results show that, in both cases, a new controller can be rapidly learned via safe reinforcement learning experiments and then added to the supervisory control structure.