Recursive tracking algorithm for a predictable time-varying parameter of a time series

We propose a recursive algorithm for tracking a multi-dimensional time-varying parameter of a time series, which is also allowed to be a predictable process with respect to the underlying time series. The algorithm is driven by a gain function. For an arbitrary time series model and a gain function satisfying some conditions, we derive a general uniform non-asymptotic accuracy bound for the tracking algorithm in terms of chosen step size for the algorithm and the oscillations of the parameter of interest. We outline how appropriate gain functions can be constructed and give several examples of different variability settings for the parameter process for which our general result can be applied, leading to different convergence rates in different asymptotic regimes. The proposed approach covers many frameworks and models where stochastic approximation algorithms comprise the main inference tool for the data analysis.We treat in some detail a couple of specificmodels.