Mạng thần kinh thường xuyên cho dự đoán P5

Recurrent Neural Networks ArchitecturesPerspective In this chapter, the use of neural networks, in particular recurrent neural networks, in system identification, signal processing and forecasting is considered. The ability of neural networks to model nonlinear dynamical systems is demonstrated, and the correspondence between neural networks and block-stochastic models is established. Finally, further discussion of recurrent neural network architectures is provided.
Nội dung trích xuất từ tài liệu:
Mạng thần kinh thường xuyên cho dự đoán P5 Recurrent Neural Networks for Prediction Authored by Danilo P. Mandic, Jonathon A. Chambers Copyright c 2001 John Wiley & Sons Ltd ISBNs: 0-471-49517-4 (Hardback); 0-470-84535-X (Electronic)5Recurrent Neural NetworksArchitectures5.1 PerspectiveIn this chapter, the use of neural networks, in particular recurrent neural networks,in system identification, signal processing and forecasting is considered. The abilityof neural networks to model nonlinear dynamical systems is demonstrated, and thecorrespondence between neural networks and block-stochastic models is established.Finally, further discussion of recurrent neural network architectures is provided.5.2 IntroductionThere are numerous situations in which the use of linear filters and models is limited.For instance, when trying to identify a saturation type nonlinearity, linear models willinevitably fail. This is also the case when separating signals with overlapping spectralcomponents. Most real-world signals are generated, to a certain extent, by a nonlinear mech-anism and therefore in many applications the choice of a nonlinear model may benecessary to achieve an acceptable performance from an adaptive predictor. Commu-nications channels, for instance, often need nonlinear equalisers to achieve acceptableperformance. The choice of model has crucial importance 1 and practical applicationshave shown that nonlinear models can offer a better prediction performance than theirlinear counterparts. They also reveal rich dynamical behaviour, such as limit cycles,bifurcations and fixed points, that cannot be captured by linear models (Gershenfeldand Weigend 1993). By system we consider the actual underlying physics 2 that generate the data,whereas by model we consider a mathematical description of the system. Many vari-ations of mathematical models can be postulated on the basis of datasets collectedfrom observations of a system, and their suitability assessed by various performance 1 System identification, for instance, consists of choice of the model, model parameter estimationand model validation. 2 Technically, the notions of system and process are equivalent (Pearson 1995; Sj¨berg et al. 1995). o70 INTRODUCTION 1 1 0.5 0.5 Neuron output y=tanh(v) 0 0 −0.5 −0.5 −1 −1 −5 0 5 −5 0 5 −5 0 5 −5 0 5 Input signal Figure 5.1 Effects of y = tanh(v) nonlinearity in a neuron model upon two example inputsmetrics. Since it is not possible to characterise nonlinear systems by their impulseresponse, one has to resort to less general models, such as homomorphic filters, mor-phological filters and polynomial filters. Some of the most frequently used polynomialfilters are based upon Volterra series (Mathews 1991), a nonlinear analogue of thelinear impulse response, threshold autoregressive models (TAR) (Priestley 1991) andHammerstein and Wiener models. The latter two represent structures that consistof a linear dynamical model and a static zero-memory nonlinearity. An overview ofthese models can be found in Haber and Unbehauen (1990). Notice that for nonlinearsystems, the ordering of the modules within a modular structure 3 plays an importantrole. To illustrate some important features associated with nonlinear neurons, let us con-sider a squashing nonlinear activation function of a neuron, shown in Figure 5.1. Fortwo identical mixed sinusoidal inputs with different offsets, passed through this non-linearity, the output behaviour varies from amplifying and slightly distorting the inputsignal (solid line in Figure 5.1) to attenuating and considerably nonlinearly distortingthe input signal (broken line in Figure 5.1). From the viewpoint of system theory,neural networks represent nonlinear maps, mapping one metric space to another. 3 To depict this, for two modules performing nonlinear functions H1 = sin(x) and H2 = ex , wehave H1 (H2 (x)) = H2 (H1 (x)) since sin(ex ) = esin(x) . This is the reason to use the term nestingrather than cascading in modular neural networks.RECURRENT NEURAL NETWORKS ARCHITECTURES 71 Nonlinear system modelling has traditionally focused on Volterra–Wiener analysis.These models are nonparametric and computationally extremely demanding. TheVolterra series expansion is given by N N N y(k) = h0 + h1 (i)x(k − i) + h2 (i, j)x(k − i)x(k − j) + · · · (5.1) i=0 i=0 j=0for the representation of a causal system. A nonlinear system represented by a Volterraseries is completely characterised by its Volterra kernels hi , i = 0, 1, 2, . . . . TheVolterra modelling of a nonlinear system requires a great deal of computation, andmostly second- or third-order Volterra systems are used in practice. Since the Volterra series expansion is a Taylor serie ...