Stochastic analysis of neural network modeling and identification of nonlinear memoryless MIMO systems

Ibnkahla

Description

Neural network (NN) approaches have been widely applied for modeling and identification of nonlinear multiple-input multiple-output (MIMO) systems. This paper proposes a stochastic analysis of a class of these NN algorithms. The class of MIMO systems considered in this paper is composed of a set of single-input nonlinearities followed by a linear combiner. The NN model consists of a set of single-input memoryless NN blocks followed by a linear combiner. A gradient descent algorithm is used for the learning process. Here we give analytical expressions for the mean squared error (MSE), explore the stationary points of the algorithm, evaluate the misadjustment error due to weight fluctuations, and derive recursions for the mean weight transient behavior during the learning process. The paper shows that in the case of independent inputs, the adaptive linear combiner identifies the linear combining matrix of the MIMO system (to within a scaling diagonal matrix) and that each NN block identifies the corresponding unknown nonlinearity to within a scale factor. The paper also investigates the particular case of linear identification of the nonlinear MIMO system. It is shown in this case that, for independent inputs, the adaptive linear combiner identifies a scaled version of the unknown linear combining matrix. The paper is supported with computer simulations which confirm the theoretical results.

Sujets

• Neural network (disambiguation)
• Gradient descent
• Statistics

Informations

• Publié par: biomed
• Publié le: 01 janvier 2012
• Nombre de lectures: 9
• Langue: English
• Poids de l'ouvrage: 3 Mo

Extrait

IbnkahlaEURASIP Journal on Advances in Signal Processing2012,2012:179 http://asp.eurasipjournals.com/content/2012/1/179

R E S E A R C H

Open Access

Stochastic analysis of neural network modeling and identification of nonlinear memoryless MIMO systems Mohamed Ibnkahla *

Abstract Neural network (NN) approaches have been widely applied for modeling and identification of nonlinear multiple-input multiple-output (MIMO) systems. This paper proposes a stochastic analysis of a class of these NN algorithms. The class of MIMO systems considered in this paper is composed of a set of single-input nonlinearities followed by a linear combiner. The NN model consists of a set of single-input memoryless NN blocks followed by a linear combiner. A gradient descent algorithm is used for the learning process. Here we give analytical expressions for the mean squared error (MSE), explore the stationary points of the algorithm, evaluate the misadjustment error due to weight fluctuations, and derive recursions for the mean weight transient behavior during the learning process. The paper shows that in the case of independent inputs, the adaptive linear combiner identifies the linear combining matrix of the MIMO system (to within a scaling diagonal matrix) and that each NN block identifies the corresponding unknown nonlinearity to within a scale factor. The paper also investigates the particular case of linear identification of the nonlinear MIMO system. It is shown in this case that, for independent inputs, the adaptive linear combiner identifies a scaled version of the unknown linear combining matrix. The paper is supported with computer simulations which confirm the theoretical results. Keywords:Nonlinear system identification, Neural networks, Gradient descent, Statistical analysis

Introduction case. In [9] the authors proposed a stochastic analysis of Neural network [1] approaches have been extensively gradient adaptive identification of nonlinear Wiener sys- used in the past few years for nonlinear MIMO system tems composed of a linear filter followed with a Zero- modeling, identification and control where they have memory nonlinearity. The model was composed of a lin- shown very good performances compared to classical ear adaptive filter followed by an adaptive parameterized techniques [2-6]. version of the nonlinearity. This study has been later If these NN approaches are to be used in real systems, generalized [16] for the analysis of stochastic gradient it is important for the algorithm designer and the user to tracking of time-varying polynomial Wiener systems. In understand their learning behavior and performance cap- [12] the author analyzed NN identification of nonlinear abilities. Several authors have analyzed NN algorithms SISO Wiener systems with memory for the case where during the last two decades which considerably helped the adaptive nonlinearity is a memoryless NN with an ar- the neural network community to better understand the bitrary number of neurons. The case of a nonlinear SISO mechanisms of neural networks [1,7-15]. For example, Wiener-Hammerstein system (i.e., an adaptive filter fol- the authors in [13] have studied a simple structure con- lowed by an adaptive Zero-memory NN followed by an sisting of two inputs and a single neuron. The authors in adaptive filter) has been analyzed in [11]. [8] studied a memoryless single-input single-output This paper deals with a typical class of nonlinear (SISO) system identification model for the single neuron MIMO systems (Figure 1) which is composed ofM inputs,Mmemoryless nonlinearities, a linear combiner, Correspondence: [e-mail] andLoutputs. This corresponds, for example, to MIMO Electrical and Computer Engineering Department, Queen’s University, channels used in wireless terrestrial communications Kingston, Ontario K7L 3N6, Canada

© 2012 Ibnkahla; licensee Springer. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

IbnkahlaEURASIP Journal on Advances in Signal Processing2012,2012:179 http://asp.eurasipjournals.com/content/2012/1/179

Figure 1Nonlinear MIMO system.

Page 2 of 22

[17-22], satellite communications [23,24], amplifier mod- structures is detailed in Section 3. Section 4 presents the eling [25], control of nonlinear MIMO systems [6], etc. analysis for the complete structure. Simulation results Recently, a neural network approach has been proposed and illustrations are given in Section 5. Finally, conclu- to adaptively identify the overall input– sionsoutput transfer and future work are given in Section 6. function of this class of MIMO systems and to characterize each component of the system (i.e., the Problem statement memoryless nonlinearities and the linear combiner) [4]. Nonlinear MIMO system The proposed NN model is composed of a set of mem- The class of nonlinear MIMO systems discussed in this oryless NN blocks followed by an adaptive linear com- paper is presented in Figure 1. Each inputx i (n) (i =1,. . ., biner. Each part of the adaptive system aims atMnonlinearly transformed by a memoryless nonlinear-) is identifying the corresponding part in the unknown ityg i (.). The outputs of these nonlinearities are then MIMO system. The algorithm has been successfully ap- linearly combined by anL×MmatrixH =[h ji ] (assumed plied to system modeling, channel tracking, and fault in this paper to be constant). MatrixHis defined by the detection. unknown system to be identified. For example, in wireless The purpose of this paper is to provide a stochastic MIMO communication systems,Mis the propagation analysis of NN modeling of this class of MIMO systems. matrix representing the channel betweenMtransmitting The paper provides a general methodology that may be antennas andLreceiving antennas. used to solve other problems in stochastic NN learning Thej th output of the MIMO system is expressed as: analysis. The methodology consists of splitting the study into simple structures, before studying the complete yðnÞ ¼ M X h ji ðnÞg i ðx i ðnÞÞ þN j ðnÞ ð1Þ structure. Here, as a first step we start by analyzing a j i¼1 simple linear adaptive MIMO scheme (consisting of an adaptive matrix) that identifies the nonlinear MIMO sys-  whereN j is a white Gaussian noise with varianceσ 0 2 . tem (i.e., problem of linear identification of a nonlinear  LetXðnÞ¼  x 1 ðnÞx 2 ðnÞ. . .x M ðnÞ t ;gðXðnÞÞ¼ ½g 1 ðx 1 ðnÞÞ MIMO system). Then we analyze a nonlinear adaptive g 2 ðx 2 ðnÞÞ. . .g M ðx M ðnÞÞ t ;YðnÞ ¼  y 1 ðnÞy 2 ðnÞ. . .y L ðnÞ t system in which the nonlinearities are assumed to be ; known and frozen during the learning process, only the and NðnÞ ¼  N 1 ðnÞN 2 ðnÞ. . .N L ðnÞ t : linearcombinerismadeadaptive.Finally,thecomplete exp T r h e e ssed sy i s n te a m mat i r n ix pu f t–output relationship can be adaptive scheme is analyzed taking into account the  orm as: insights given by the analysis of the simpler structures. In our analytical approach, we derive the general formu- YðnÞ ¼HgðXðnÞÞ þNðnÞ:ð2Þ las and recursions, which we apply to a case study that we believe is illustrative to the reader. Neural Network identification structure and algorithm The paper is organized as follows. The problem state-  The neural network (Figure 2) is composed ofMblocks. ment is given in Section 2. The study of the simple  Each blockkhas a scalar inputx k (n) (k =1,. . .,M), N