Adaptive Filtering and Data Compression using Neural Networks in Biomedical Signal Processing

1. Adaptive Filtering and Data Compression using Neural Networks in Biomedical Signal Processing T-61.181 Biomedical Signal Processing 2.12.2004 Sampo Ojala

2. Contents Neural Networks What are neural networks and where they are used? Properties of a neural network Network topologies Learning Adaptive filtering What is adaptive filtering and when is it needed? Common algorithms used in adaptive filtering Adaptive filtering with neural networks Two example cases Data Compression What is data compression and when is it needed? Lossless and lossy compression Data compression with neural networks An example case Sampo Ojala

3. Neural Networks ”A neural network is a massively parallel distributed processor made up of simple processing units…” [1] Why? (Artificial) neural networks (ANNs) offer several useful properties and capabilities e.g. nonlinearity, adaptivity, input-output mapping, VLSI implementability etc. Where? Neural networks can be used in many application areas for example in: Pattern recognition Vehicle and robot control systems Data compression Adaptive filtering Sampo Ojala

4. Neural Networks - Basic Concepts The basic building block of a neural network is a single perceptron, which has many similarities with a biological neuron Activation function weights summation Inputs Output For more complex problems, which are encountered in practice, a multilayer network is required. Sampo Ojala

5. Input Signal and Weights Weights Weights are connected between an input and a summing node. These affect to the summing operation. The quality of network can be seen from weights Bias is a constant input with certain weight. Usually the weights are randomized in the beginning Input signals An input may be either a raw / preprocessed signal or image. Alternatively, some specific features can also be used. If specific features are used as input, their number and selection is crucial and application dependent Sampo Ojala

6. Activation Function and Output Summation This is just a weighted sum (linear combination) Output This depends on the activation function. There may be several outputs Activation Function There are different activation functions used in different applications. The most common ones are: Hard-limiter Piecewise linear Sigmoid Hyperbolic tangent Sampo Ojala

7. Different Neural Networks Neural networks can be characterized by the topology of the network. Single layer feedforward networks Multilayer feedforward networks (hidden layers) Recurrent networks (feedbacks) Networks may be either partially or fully connected A fully connected, multilayer feedforward neural network Sampo Ojala

8. Learning The learning can be supervised (e.g. backpropagation), unsupervised (e.g. competitive learning) or reinforcement learning The backpropagation-algorithm is probably the most common algorithm used for training neuron j is an output node neuron j is a hidden node There is some optimal point where the training should be stopped in order not to overtrain the network. Sampo Ojala

9. Adaptive Filtering Why? If the properties of noise change in time If the frequencies of the signal and the noise overlap How? By changing the system parameters according to some algorithm Where? Few biomedical application examples [3&6]: Fetal ECG extraction (ECG signal) 50 Hz mains (ECG-signal) Eye movement artefacts (EEG-signal) Sampo Ojala

10. Neural Network Adaptive Filtering - Basic Concepts In adaptive filters, there are many different algorithms used for computing the weights. These are the two common ones [6]: Least Mean Square (LMS) Recursive Least Squares (RLS) Most effective in terms of computation & storage requirements Superior convergence properties learning parameter forgetting factor recursive way to compute the inverse matrix Sampo Ojala

11. Adaptive Filtering with Neural Networks Why neural networks? High-speed computational capacity. The weight coefficients can be computed in real-time [5] Any nonlinear filter can be realized with by an MLP-network. Used if noise is non-Gaussian or not additive [5] LMS: Can be implemented with one linear neuron RLS: Computational complexity is reduced [O(N n ) O(N)] and the result of the algorithm can be calculated in real-time [5] 2 2 Sampo Ojala

12. Adaptive Filtering with Neural Networks (2) The weights of a neural network are usually updated with backpropagation algorithmor temporal backpropagation algorithm [1] 1) Propagate input signal through the network and calculate error e(n) 2) Neuron j in the output layer: A = a group of neurons connected to the neuron j 3) Neuron j in a hidden layer: l = hidden layer index (from output layer) The filter coefficients are actually the weights of the neurons The response of the adaptive filter is affected by: weights of the neurons thresholds used activation functions Sampo Ojala

13. (NLMS) Example Case (ECG) – Fetal ECG Extraction Problem: The spectra of mother and fetal signals overlap each other. In addition the amplitude of MECG is much higher [9] Sampo Ojala

14. Example Case (EEG) –Visual Evoked Potential Estimation Problem: The number of VEPs required in ensemble averaging is large due poor SNR. The averaging smoothes differences (amplitude & latency) between VEPs [10] The VEP signals have a non-linear behaviour, which can be matched better with NN’s than conventional adaptive filters. As a result, fewer VEPs are required [10] Sampo Ojala

15. Data Compression “The goal of data compression is to represent an information source … as accurately as possible using the fewest number of bits.“ [1] Why? Measured raw data requires a lot of memory / channel capacity. How? Redundancies in the data are exploited Where? There are many applications for example in image & video signal, speech biomedical signal processing Sampo Ojala

16. Data Compression – Basic Concepts Compression rate, measures how much the signal can be compressed from the original one. Compression methods which are used can be lossless (e.g. RLC), lossy (e.g. JPEG) or both. Lossy compression High compression rates can be achieved (e.g. 28,67) Too high compression ratio results in distortions The exact replica of an original signal cannot be recovered anymore Size: 10,5 MB Size: 375 kB The compression of ECG signal must always be lossy in order to be efficient enough Sampo Ojala

17. Data Compression Methods There are many ways for compressing data, for example: Linear Predictive Coding (LPC): A sample s(n) is approximated with a linear combination of previous samples Principal Component Analysis (PCA): Gives the optimal solution in the field of data compression. It projects the d-dimensional data into a lower dimension in a optimal way (sum-squared error sense). The d eigenvectors have the largest variances Sampo Ojala

18. Data Compression with Neural Networks Why neural networks? High-speed computational capacity. The compression can be done in real-time [5] The nonlinear mapping properties of an MLP can be used to implement nonlinear predictors in prediction coding. This is much more powerful than using linear predictors [5][11] Experimental knowledge of the networks can be exploited [5] LPC: Solution can be provided at much higher speed. PCA: Adaptive PCA extraction, more simple to calculate [5] Sampo Ojala

19. Data Compression with Neural Networks The neural networks used in data compression have massively parallel structures and high-degree of interconnections [5] The compression ratio depends on the ratio of neurons on input layer and on hidden layer. Output layer is as big as input layer [13] The actual compressed data is obtained from the weights and activation levels of the network [12] Sampo Ojala

20. Example Case (ECG) Problem: The long time (Holter) monitoring of ECG requires a lot of memory. fs = 250 Hz ADC = 11 bits time = 24 h ~226,6 Mbits BPNN (Back Propagation Neural Network) PCANN (Principal Component Analysis Neural Network) Sampo Ojala

21. Lower Higher Lower Higher Example Case (ECG) (2) - Results The compression rate of ~33 is achievable with this method PRD (Percent Root Mean Square Difference) Sampo Ojala

22. Summary (1): Neural Networks They offer many advantages e.g. non-linearity, VLSI implementability and adaptivity The basic building block of a network is a perceptron, which resembles a physiological neuron To generate an output, a weighted sum from the inputs of the network is calculated and then an activation function is used. There are many different network topologies for different applications Neural networks can learn (adaptability) either in a supervised or unsupervised manner. Many different learning algorithms have been developed. Sampo Ojala

23. Summary (2): Neural Networks in Adaptive Filtering Adaptive filtering is needed when spectra of signal and noise overlap Neural networks can be used effectively for adaptive filtering, since they adapt to nonlinear and time-varying features of a signal. The filter weights adapt faster with NN’s than with common adaptive filter algorithms. Also signal artefacts are handled better [7]. The exact algorithm knowledge is not required, thus any non-linear adaptive filter is implementable with NN’s Today the research is more focused on wavelets Sampo Ojala

24. Summary (3): Neural Networks in Data Compression Data compression is often needed because of limited storage capacity (memory) or transmission bandwidth Neural networks can be used to data compression and their advantages over ”traditional” methods are very fast, nonlinear operation and learning capability. As disadvantages, they must be teached for each patient separately, which takes time and the resulting compression is lossy. The neural network structure is highly parallel and interconnected in data compression applications. Sampo Ojala

25. References [1] S. Haykin, Neural Networks 2nd Ed.1990 [2] Data Compression.com <http://www.data-compression.com/index.shtml> 11.11.2004 [3] D. Perez, Adaptive Filtering, University of Strathclyde <http://www.spd.eee.strath.ac.uk/~interact/AF/aftutorial/apps/appsindex.html> 11.11.2004 [4] V. Sebesta, The Utilization of Neural Networks in Medical Informatics <http://new.euromise.org/english/material/95102023/sebesta.html> 15.11.2004 [5] F. Luo, Applied Neural Networks for Signal Processing, 1997 [6] E.Ifeachor, Digital Signal Processing – Practical Approach, 1997 Sampo Ojala

26. [7] A.Yilmaz & M.J.English, Adaptive Non-Linear Filtering of ECG Signals: Dynamic Neural Network Approach, Artificial Intelligence Methods for Biomedical Data Processing, IEEE Colloquium on , 26 Apr 1996 [8] M.B.I. Reaz & L.S. Wei, An Approach of Neural Network Based Fetal ECG Extraction. Enterprise Networking and Computing in Healthcare Industry, 2004. HEALTHCOM 2004. Proceedings. 6th International Workshop on , 28-29 June 2004 [9] G. Camps, M. Martinez & E. Soria, Fetal ECG Extraction Using an FIR Neural Network. Computers in Cardiology 2001, 23-26 Sept. 2001 References (2) Sampo Ojala

27. References (3) [10] K.S.M. Fung, F.K. Lam, F.H.Y. Chan, P.W.F. Poon, J.G.Liu, Adaptive Neural Network Filter for Visual Evoked Potential Estimation. Neural Networks, 1995. Proceedings., IEEE International Conference on  ,Volume: 5 , 27 Nov.-1 Dec. 1995 [11] R. Kannan, C. Eswaran, N. Sriraam, Neural Network Based Methods for ECG Data Compression. Neural Information Processing, 2002. ICONIP '02. Proceedings of the 9th International Conference on , Volume: 5 , 18-22 Nov. 2002 [12] A. Iwata, Y. Nagasaka, N. Suzumura, Data Compression of the ECG Using Neural Network for Digital Holter Monitor. Engineering in Medicine and Biology Magazine, IEEE, Volume: 9 , Issue: 3 , Sept. 1990 Sampo Ojala

28. References (4) [13] Y. Nagasaka, A. Iwata, Performance Evaluation of BP and PCA Neural Networks for ECG Data Compression. Neural Networks, 1993. IJCNN '93-Nagoya. Proceedings of 1993 International Joint Conference on , Volume: 1 , 25-29 Oct. 1993 Sampo Ojala