The quest for the right kernel in Bayesian impulse response identification: The use of OBFs

M.A.H. Darwish, Gianluigi Pillonetto, R. Toth

Research output: Contribution to journalArticleAcademicpeer-review

6 Citations (Scopus)
1 Downloads (Pure)

Abstract

Kernel-based regularization approaches for impulse response estimation of Linear Time-Invariant (LTI) systems have received a lot of attention recently. The reason is that regularized least-squares estimators may achieve a favorable bias/variance trade-off compared with classical Prediction Error Minimization (PEM) methods. To fully exploit this property, the kernel function needs to capture relevant aspects of the data-generating system at hand. Hence, it is important to design automatic procedures for kernel design based on data or prior knowledge. The kernel models, so far introduced, focus on encoding smoothness and BIBO-stability of the expected impulse response while other properties, like oscillatory behavior or the presence of fast and slow poles, have not been successfully implemented in kernel design. Inspired by the representation theory of dynamical systems, we show how to build stable kernels that are able to capture particular aspects of system dynamics via the use of Orthonormal Basis Functions (OBFs). In particular, desired dynamic properties can be easily encoded via the generating poles of OBFs. Such poles are seen as hyperparameters which are tuned via marginal likelihood optimization. Special cases of our kernel construction include Laguerre, Kautz, and Generalized OBFs (GOBFs)-based kernel structures. Monte-Carlo simulations show that the OBFs-based kernels perform well compared with stable spline/TC kernels, especially for slow systems with dominant poles close to the unit circle. Moreover, the capability of Kautz basis to model resonating systems is also shown.
Original languageEnglish
Pages (from-to)318-329
JournalAutomatica
Volume87
DOIs
Publication statusPublished - Jan 2018

Fingerprint

Impulse response
Poles
Dynamical systems
Splines

Cite this

Darwish, M.A.H. ; Pillonetto, Gianluigi ; Toth, R. / The quest for the right kernel in Bayesian impulse response identification: The use of OBFs. In: Automatica. 2018 ; Vol. 87. pp. 318-329.
@article{22e9948fb16f46fdacf8c23b7717e433,
title = "The quest for the right kernel in Bayesian impulse response identification: The use of OBFs",
abstract = "Kernel-based regularization approaches for impulse response estimation of Linear Time-Invariant (LTI) systems have received a lot of attention recently. The reason is that regularized least-squares estimators may achieve a favorable bias/variance trade-off compared with classical Prediction Error Minimization (PEM) methods. To fully exploit this property, the kernel function needs to capture relevant aspects of the data-generating system at hand. Hence, it is important to design automatic procedures for kernel design based on data or prior knowledge. The kernel models, so far introduced, focus on encoding smoothness and BIBO-stability of the expected impulse response while other properties, like oscillatory behavior or the presence of fast and slow poles, have not been successfully implemented in kernel design. Inspired by the representation theory of dynamical systems, we show how to build stable kernels that are able to capture particular aspects of system dynamics via the use of Orthonormal Basis Functions (OBFs). In particular, desired dynamic properties can be easily encoded via the generating poles of OBFs. Such poles are seen as hyperparameters which are tuned via marginal likelihood optimization. Special cases of our kernel construction include Laguerre, Kautz, and Generalized OBFs (GOBFs)-based kernel structures. Monte-Carlo simulations show that the OBFs-based kernels perform well compared with stable spline/TC kernels, especially for slow systems with dominant poles close to the unit circle. Moreover, the capability of Kautz basis to model resonating systems is also shown.",
author = "M.A.H. Darwish and Gianluigi Pillonetto and R. Toth",
year = "2018",
month = "1",
doi = "10.1016/j.automatica.2017.10.007",
language = "English",
volume = "87",
pages = "318--329",
journal = "Automatica",
issn = "0005-1098",
publisher = "Agon Elsevier",

}

The quest for the right kernel in Bayesian impulse response identification: The use of OBFs. / Darwish, M.A.H.; Pillonetto, Gianluigi; Toth, R.

In: Automatica, Vol. 87, 01.2018, p. 318-329.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - The quest for the right kernel in Bayesian impulse response identification: The use of OBFs

AU - Darwish, M.A.H.

AU - Pillonetto, Gianluigi

AU - Toth, R.

PY - 2018/1

Y1 - 2018/1

N2 - Kernel-based regularization approaches for impulse response estimation of Linear Time-Invariant (LTI) systems have received a lot of attention recently. The reason is that regularized least-squares estimators may achieve a favorable bias/variance trade-off compared with classical Prediction Error Minimization (PEM) methods. To fully exploit this property, the kernel function needs to capture relevant aspects of the data-generating system at hand. Hence, it is important to design automatic procedures for kernel design based on data or prior knowledge. The kernel models, so far introduced, focus on encoding smoothness and BIBO-stability of the expected impulse response while other properties, like oscillatory behavior or the presence of fast and slow poles, have not been successfully implemented in kernel design. Inspired by the representation theory of dynamical systems, we show how to build stable kernels that are able to capture particular aspects of system dynamics via the use of Orthonormal Basis Functions (OBFs). In particular, desired dynamic properties can be easily encoded via the generating poles of OBFs. Such poles are seen as hyperparameters which are tuned via marginal likelihood optimization. Special cases of our kernel construction include Laguerre, Kautz, and Generalized OBFs (GOBFs)-based kernel structures. Monte-Carlo simulations show that the OBFs-based kernels perform well compared with stable spline/TC kernels, especially for slow systems with dominant poles close to the unit circle. Moreover, the capability of Kautz basis to model resonating systems is also shown.

AB - Kernel-based regularization approaches for impulse response estimation of Linear Time-Invariant (LTI) systems have received a lot of attention recently. The reason is that regularized least-squares estimators may achieve a favorable bias/variance trade-off compared with classical Prediction Error Minimization (PEM) methods. To fully exploit this property, the kernel function needs to capture relevant aspects of the data-generating system at hand. Hence, it is important to design automatic procedures for kernel design based on data or prior knowledge. The kernel models, so far introduced, focus on encoding smoothness and BIBO-stability of the expected impulse response while other properties, like oscillatory behavior or the presence of fast and slow poles, have not been successfully implemented in kernel design. Inspired by the representation theory of dynamical systems, we show how to build stable kernels that are able to capture particular aspects of system dynamics via the use of Orthonormal Basis Functions (OBFs). In particular, desired dynamic properties can be easily encoded via the generating poles of OBFs. Such poles are seen as hyperparameters which are tuned via marginal likelihood optimization. Special cases of our kernel construction include Laguerre, Kautz, and Generalized OBFs (GOBFs)-based kernel structures. Monte-Carlo simulations show that the OBFs-based kernels perform well compared with stable spline/TC kernels, especially for slow systems with dominant poles close to the unit circle. Moreover, the capability of Kautz basis to model resonating systems is also shown.

U2 - 10.1016/j.automatica.2017.10.007

DO - 10.1016/j.automatica.2017.10.007

M3 - Article

VL - 87

SP - 318

EP - 329

JO - Automatica

JF - Automatica

SN - 0005-1098

ER -