Thuật toán ICA - 13: Practical Considerations

In the preceding chapters, we presented several approaches for the estimation ofthe independent component analysis (ICA) model. In particular, several algorithmswere proposed for the estimation of the basic version of the model, which has asquare mixing matrix and no noise. Now we are, in principle, ready to apply thosealgorithms on real data sets. Many such applications will be discussed in Part IV.
Nội dung trích xuất từ tài liệu:
Thuật toán ICA - 13: Practical Considerations Independent Component Analysis. Aapo Hyv¨ rinen, Juha Karhunen, Erkki Oja a Copyright  2001 John Wiley & Sons, Inc. ISBNs: 0-471-40540-X (Hardback); 0-471-22131-7 (Electronic) 13 Practical ConsiderationsIn the preceding chapters, we presented several approaches for the estimation ofthe independent component analysis (ICA) model. In particular, several algorithmswere proposed for the estimation of the basic version of the model, which has asquare mixing matrix and no noise. Now we are, in principle, ready to apply thosealgorithms on real data sets. Many such applications will be discussed in Part IV. However, when applying the ICA algorithms to real data, some practical con-siderations arise and need to be taken into account. In this chapter, we discussdifferent problems that may arise, in particular, overlearning and noise in the data.We also propose some preprocessing techniques (dimension reduction by principalcomponent analysis, time filtering) that may be useful and even necessary before theapplication of the ICA algorithms in practice.13.1 PREPROCESSING BY TIME FILTERINGThe success of ICA for a given data set may depend crucially on performing someapplication-dependent preprocessing steps. In the basic methods discussed in theprevious chapters, we always used centering in preprocessing, and often whiteningwas done as well. Here we discuss further preprocessing methods that are notnecessary in theory, but are often very useful in practice. 263264 PRACTICAL CONSIDERATIONS13.1.1 Why time filtering is possibleIn many cases, the observed random variables are, in fact, time signals or time series,which means that they describe the time course of some phenomenon or system.Thus the sample index t in xi (t) is a time index. In such a case, it may be very usefulto filter the signals. In other words, this means taking moving averages of the timeseries. Of course, in the ICA model no time structure is assumed, so filtering is not xalways possible: If the sample points (t) cannot be ordered in any meaningful waywith respect to t, filtering is not meaningful, either. For time series, any linear filtering of the signals is allowed, since it does notchange the ICA model. In fact, if we filter linearly the observed signals xi (t) toobtain new signals, say xi (t), the ICA model still holds for xi (t), with the same Xmixing matrix. This can be seen as follows. Denote by the matrix that contains x x Sthe observations (1) ::: (T ) as its columns, and similarly for . Then the ICAmodel can be expressed as: X = AS (13.1) XNow, time filtering of corresponds to multiplying X from the right by a matrix, let Mus call it . This gives X = XM = ASM = AS (13.2)which shows that the ICA model still remains valid. The independent componentsare filtered by the same filtering that was applied on the mixtures. They are notmixed with each other in S M because the matrix is by definition a component-wisefiltering matrix. Since the mixing matrix remains unchanged, we can use the filtered data in theICA estimating method only. After estimating the mixing matrix, we can apply thesame mixing matrix on the original data to obtain the independent components. The question then arises what kind of filtering could be useful. In the following,we consider three different kinds of filtering: high-pass and low-pass filtering, aswell as their compromise. PREPROCESSING BY TIME FILTERING 26513.1.2 Low-pass filteringBasically, low-pass filtering means that every sample point is replaced by a weighted Maverage of that point and the points immediately before it.1 This is a form ofsmoothing the data. Then the matrix in (13.2) would be something like 0 . 1 B B 1 1 1 0 ...