Temporal processing with neural networks : the development of the Gamma model

This dissertation discusses the problem of processing complex temporal patterns by artificial neural networks. The relatively broad topic of this work is intentional--processing here includes such specialties as system identification, time series prediction, interference canceling and sequence classification. Rather than focusing on a particular application, this research concentrates on the paradigm of time representation in neural network structures. In all temporal processing applications, an essential capacity for a neural net is to store information from the recent past (we refer to this capacity as short term memory). The main contribution of this work is the introduction of a new (neural net) mechanism to store temporal information. This model, the gamma neural model, compares very favorable to competing memory structures, such as the tapped delay line and rirst-order self-recurrent memory units. The gamma memory mechanism is characterized by a cascade of uniform locally self-recurrent delay units. An interesting feature of the gamma memory mechanism is the adaptability of the memory depth and resolution. The gamma model is analyzed and compared with competing neural models. A temporal back propagation training procedure for gamma neural nets is derived. Experiments in time series prediction (electro-encephalogram (EEG) and synthetic chaotic signals), noise removal from a chaotic signal and system identification are discussed. In all experiments, the gamma model outperforms competing network architectures. Interestingly, the application of the gamma memory structure is not limited to neural nets. A chapter is devoted to introduce adaline (u) an adaptive filter with gamma memory. Adaline (U) generalizes Widrow's adaptive linear combiner (adaline), the most widely used structure in adaptive signal processing. The signal characteristics and processing applications where adaline (u) improves on the performance of adaline are identified.