New Probabilistic Guarantees on the Accuracy of Extreme Learning Machines: An Application to Decision-Making in a Reliability Context

This work investigates new generalization error bounds on the predictive accuracy of Extreme Learning Machines (ELMs). Extreme Learning Machines are a special type of neural network that enjoy an extremely fast learning speed thanks to the convexity of the training program. This feature makes ELMs particularly useful to tackle online learning tasks. A new probabilistic bound on the accuracy of ELM is prescribed thanks to scenario decision-making theory. Scenario decision-making theory allows equipping the solutions of data-based decision-making problems with formal certificates of generalization. The resulting certificate bounds the probability of constraint violation for future scenarios (samples). The bounds hold non-asymptotically, distribution-free, and therefore quantify the uncertainty resulting from limited availability of training examples. We test the effectiveness of this new method on reliability-based decision-making problem. A data set of samples from the benchmark problem on robust control design is used for the online training of ELMs and empirical validation of the bound on their accuracy.