Hierarchical Temporal Memory

1. Hierarchical Temporal Memory “The Application of Hierarchical Temporal Memory to the ECG Waveforms” May 6, 2011 Version 3.0; 05/06/2011 John M. Casarella Proceedings of Student/Faculty Research Day Ivan G. Seidenberg School of CSIS, Pace University

2. Introduction • Human Vision is a Key component of Human Learning • Learning is a process that alters the synaptic strengths between neurons • This allows for pattern memories to form • Now, inference can take place • What appears to be a task so easily accomplished by humans, invariant visual pattern recognition, still eludes computational models of human intelligence or what we refer to as Artificial Intelligence

3. Learning and Memory • Humans learn and form memories • Learning alters the physical structure of the brain, exhibiting plasticity in its ability to change and adapt • Memory is not localized, but distributed across the neocortex • Learning is associated with the construction of cell assemblies related to the formation of pattern association - brain plasticity • Donald Hebb, Hebbian Learning • Briefly, if a neuron receives an input from another neuron and if both are highly active (mathematically have the same sign), the weight between the neurons should be strengthened. • Learning is the modification of these connections • Highly influential in the concepts behind the HTM Model • Pattern recognition learned through by changing the strength of the connections between neurons

4. Intelligence and Machine Intelligence • What is Intelligence? • A uniquely human quality? • The ability to adapt to a changing environment or the ability to change our environment for survival • Machine Intelligence or AI • Define – what is it? • Objectives - Create machines to do something which would require intelligence if done by a human or to solve the problem of how to solve the problem • Varieties – Expert Systems, Genetic Algorithms, Perceptrons, Neural Nets • Key People – Shannon, von Neumann, McCarthy, Rosenblatt, Minsky (MIT AI Lab) and Zadeh (Fuzzy Logic) • Alan Turing

5. Alan Turing • “Can machines Think?” and The Turing Test • Provided the foundation for Connectionism • Model digital computer like child’s mind, then “educate” to obtain “adult” • “Unorganized Machines” : A network of neuron-like Boolean elements randomly connected together • Proposed machines should be able to “learn by experience” Neurons Unorganized Machines

6. Neural Nets • McCulloch and Pitts • Model of Neurons of the brain • Cornerstone of neural computing and neural networks • Boolean nets of simple two-state ‘neurons’ and the concept of ‘threshold’ • No mechanism for learning • Hebb - Pattern recognition learned through by changing the strength of the connections between neurons • Rosenblatt • Perceptron Model - permitted mathematical analysis of neural networks • Based on McCulloch and Pitts • Linear combiner followed by a hard limiter • Activation and weight training • Linear Separation - No XOR

7. Neuroscience • The neocortex in humans is associated with such tasks as visual pattern recognition, understanding language and object recognition and manipulation and inference • These are just some of the basic tasks which we engage in on a daily basis, allowing us to function in our world, yet with all of the advancements in computer hardware, software and computing in general, computers are still incapable of cognitive function. • Locus of: perception, language, planned behavior, declarative memory, imagination, planning • Invariant Pattern Recognition • Repetitive structure • Unique columnar six layer structure - Montcastle • Hierarchy of cortical regions

8. Electro-cardiology • Electrocardiography is the process of recording and interpreting the electrical activity of the action of the heart, providing physicians with the ability to study the heart's normal behavior and allow for the diagnosing of abnormalities • The standard ECG in a clinical setting has 12 leads • Represents different phases of the cardiac cycle • A normal ECG shows a sequence of at least three waves labeled P, QRS, and T. The baseline voltage of the electrocardiogram is known as the isoelectric line.

9. ECG Computational Methods • Currently there are three computerized interpretations techniques: (1) statistical methods (2) deterministic method and (3) the use of artificial neural networks, with the deterministic approach the most common, which mimics the diagnostic criteria tree used by cardiologists • In the multitude of papers written and reviewed concerning the computational evaluation of ECG waveforms, one of the common methods is the application of traditional artificial neural networks. • Artificial Neural Networks have been used to correlate ECG signals to specific pathologies, in the classification of the individual ECG beat waveforms, for the determination of myocardial infarction to identify but a few of the of most significant applications • The majority of these neural networks were designed using well-documented methodologies and mathematical models, but included the need to pre-process the data and to identify a set of extracted features

10. Hierarchical Temporal Memory The Hierarchical Temporal Memory (HTM) model is an attempt to replicate the structural and algorithmic properties of the neocortex based on human neurological function, specifically identified in its pattern of learning and its ability to form pattern memories for pattern classification The HTM model really should be viewed as a memory prediction model The HTM can also be compared to traditional Artificial Neural Networks (ANN) but how they learn, how they resolve the differences between different patterns and the computational algorithms behind the HTM model, removes any connection to ANNs

11. Hierarchical Temporal Memory Overview • Each node performs similar algorithm • Each node learns • 1) Common spatial patterns • 2) Common sequences of spatial patterns • (use time to form groups of patterns with a common cause) • “Names” of groups passed up • - Many to one mapping, bottom to top • - Stable patterns at top of hierarchy • Modeled as an extension of Bayesian network with belief propagation • Creates a hierarchical model (time and space) of the world Hierarchy of memory nodes

12. Hierarchical Temporal Memory • (A) An initial node that has not started its learning process. • (B) The spatial pooler of the node is in its learning phase and has formed 2 quantization enters • (C) the spatial pooler has finished its learning process and is in the inference stage. The temporal pooler is receiving inputs and learning the time-adjacency matrix. • (D) shows a fully learned node where both the spatial pooler and temporal pooler have finished their learning processes

13. Hierarchical Temporal Memory • Structure of an HTM network for learning invariant representations for the binary images world. • This network is organized in 3 levels. Input is fed in at the bottom level. Nodes are shown as squares. • The top level of the network has one node, the middle level has 16 nodes and the bottom level has 64 nodes. • The input image is of size 32 pixels by 32 pixels. This image is divided into adjoining patches of 4 pixels by 4 pixels as shown. Each bottom-level node’s input corresponds to one such 4x4 patch.

14. Hierarchical Temporal Memory Level 2 Temporal Pooler Level 2 Spatial Pooler Level 1 Temporal Pooler Level 1 Spatial Pooler

15. Hierarchical Temporal Memory

16. Hierarchical Temporal Memory This figure illustrates how nodes operate in a hierarchy; we show a two-level network and its associated inputs for three time steps. This network is constructed for illustrative purposes and is not the result of a real learning process. The outputs of the nodes are represented using an array of rectangles. The number of rectangles in the array corresponds to the length of the output vector. Filled rectangles represent ‘1’s and empty rectangles represent ‘0’s.

17. Hierarchical Temporal Memory

18. Research Focus • Application of the HTM model, once correctly designed and configured, will provide a greater success rate in the classification of complex waveforms • Abandon traditional data pre-processing and feature extraction, applying a visual process using the actual images • Task Description • Create an image dataset of each waveform group for classification • Determine, through organized experiments, an optimized HTM • Apply optimized HTM to the classification of waveforms using images, devoid of any pre-processing or feature extraction

19. HTM Design and Implementation • No set methodologies or mathematical correlations to determine HTM structure • Key optimization parameters addressed by observation • The key parameters identified included, but were not limited to, the number of iterations at each level, the Euclidian distance during vector formation, the number of images applied during learning, the maximum number of coincidences (pattern memories) allowed and the maximum number of groups allowed for pattern grouping. • De facto recommendation: • Initial settings should be based on educated guesses • Parameters and framework to be refined by trial and error

20. ECG Data • The Limb Lead II is commonly used in three lead as well as twelve lead ECG recordings. • This lead features a prominent and upright QRS complex, facilitating the choice of this lead. • Of the beat classifications, Normal Sinus, LBBB and RBBB beats were used in this study. • The total number of beats used was 1160 per data set, which consisted of learning and unknown subsets. • Each learning set was comprised on 300 images, 100 from each group classification. • Each unknown data set contained 860 images comprising of 300 LBBB images, 360 normal sinus images and 200 RBBB images.

21. ECG Waveform Images Normal Sinus Right Bundle Branch Block Left Bundle Branch Block

22. Individual ECG Beat Images Left Bundle Branch Block Normal Sinus Right Bundle Branch Block All images were sized to 96 x 120 pixels

23. Results • For the classification of the individual ECG beats, with very few exceptions, the classification rates exceeded 98 percent and approached 100 percent for many of the individual models in combination with the various data sets. • Once a well defined model is established, the dataset applied during learning always returned a classification rate of 100 percent. • For the classification of the same grouped heartbeats as used in this study, Thiagarajan obtained an average classification percentage of 91 percent for the training set and 79 percent for the unknown set using a 1 to 1 training to unknown ratio, each containing approximately 700 entities. • The experimental results indicate the number of images and the number of iterations applied during learning directly influenced the performance of an HTM model in its ability to correctly classify and differentiate the individual ECG beat images. • When all unknown images were presented to a HTM model after learning en masse, it was able to correctly differentiate and classify the vast majority, if not all, of heartbeats to the correct waveform class. • The addition of background noise to the unknown image set did not have a significant effect on the ability to classify the unknown images. Even with a high level of noise added, the model correctly classified an average of 99.4 percent of the images.

24. HTM Model Results Unkn1 = Normal Sinus Unkn2 = Right Bundle Branch Block Unkn3 = Left Bundle Branch Block

25. Results by HTM Model

26. Results by Dataset and Model

27. Results by Dataset

28. Effects of Occlusion on Classification Clean Occlusion0 Occlusion1 Occlusion2

29. HTM Learning Curve

30. Observations The formulation of a mathematical model to determine the number of groups, coincidences per group, the number of iterations required to create a sufficient number of pattern memories or any other parameter, was met with limited success. It was determined the level of complexity of the ECG images used during learning and the relationship (Euclidian proximity) between the graph (which could be classified as noise) and the ECG waveform produced pattern memories of greater complexity, diversity and Euclidian differential than encountered in previous examples. It was the visual inspection of the pattern memories formed, the grouping of the patterns, the groups with many associated coincidences and the number single coincidence groups, which was necessary to validate the results. Creation of an Optimal image learning set can influence the learning ability of the model. Sufficient number of pattern memories formed during learning is necessary, which is directly influenced by the number of iterations at each level.

31. Observations There is a need for the Optimal grouping of pattern memories, based on the number of patterns per group, the number of groups and the number of one-pattern groups. There was a distinct need to determine how often the learning image set needed to be sampled to insure a sufficient number of coincidences or pattern memories are created to insure the correct classification of unknown images when presented to the model.

32. Conclusions and Inference The primary objective of the research presented herein was to determine if the Hierarchical Temporal Memory model when applied to the classification (pattern recognition) of ECG waveform patterns (individual ECG beats), would return performance results equal to or greater than the application of a neural network. The results obtained provided strong evidence the Hierarchical Temporal Memory Model provides a unique and improved approach for the classification of ECG waveforms, with the potential of providing a classification accuracy of 100 percent. Additional experimental results indicate the number of images and the number of iterations applied during learning directly influences the performance of an HTM model in its ability to correctly classify and differentiate the individual ECG beat images, mimicking human learning.

33. References [1] Turing, A. M. “Computing Machinery and Intelligence”. Mind, New Series, Vol. 59, No. 236. (Oct., 1950). pp 443 – 460. [2] B. J. Copeland, "The Essential Turing," Oxford, Great Britain: Oxford University Press, 2004. [3] B. Scassellati, "How social robots will help us to diagnose, treat, and understand autism," in 12th International Symposium of Robotics Research San Francisco, CA. 2005. [4] D. O. Hebb, The Organization of Behavior. New York: John Wiley & Sons, 1949. [5] E. Besterman, and Richard Creese, "Waller - Pioneer of Electrocardiography," British Heart Journal, vol. 42, pp. 61-64, 1979 [6] W. Einthoven, G. Fahr, and A. de Waart, "On the Direction and Manifest Size of the Variations of Potential in Human Heart and on the Influence of the Position of the Heart on the Form of the Electrocardiogram," American Heart Journal, vol. 40, pp. 163-211, 1950. [7] Pipberger, H.V., R.J. Arms and F. W. Stallman, “Automatic Screening of Normal and Abnormal Electrocardiograms by Means of a Digital Computer,”Proc Soc Exp Biol Med, vol. 106, pp. 130-132, 1961. [8] L. Edenbrandt, Brian Devine, and Peter W. Macfarlane, "Neural Networks for Classification of ECG ST-T Segments," Journal of Electrocardiology, vol. 25, pp. 167 - 173, 1992. [9] S. Thiagarajan, "Bottom-up Design of Artificial Neural Network for Single Lead Electrocardiography Beat and Rhythm Classification," Doctoral Dissertation; Baton Rouge: College of Engineering and Science, Louisiana Tech University, 2000 [10] A. A. Sleiman, "Simplified ECG Waveforms Analysis Utilizing Matlab-Based Artificial Neural Network (ANN) Tools," in Department of Electrical Engineering. vol. MSc Long Beach: California State University, Long Beach, 2007. [11] Computational Models of the Neocortex. http://www.cs.brown.edu/people/tld/projects/cortex/ [12] George, Dileep and Jaros, Bobby. The HTM Learning Algorithms. Numenta, Inc. March 1, 2007. [13] Hawkins, Jeff. Learn like a Human. http://spectrum.ieee.org/apr07/4982. [14] Hawkins, Jeff and Dileep, George. Hierarchical Temporal Memory, Concepts, Theory, and Terminology. Numenta, Inc. 2008. [15] F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review, vol. 65, pp. 386 - 408, 1958 [16] F. Rosenblatt, Principles of Neurodynamics. Washington, DC: Spartan Books, 1962. [17] D. E. Rumelhart, and McClelland, J.L. editors, "Parallel Distributed Processing: Explorations in the Microstructures of Cognition." vol. 2 - Psychological and Biological Models Cambridge, MA: MIT Press, 1986. [18] D. E. Rumelhart, and McClelland, J.L. editors, "Parallel Distributed Processing: Explorations in the Microstructures of Cognition." vol. 1 - Foundations Cambridge, MA: MIT Press, 1986 [19] M. L. Minsky, and Papert, Seymour S., Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969.

34. References [20] M. Negnevitsky, Artificial Intelligence, A Guide to Intelligent Systems, 2nd ed. Essex, England: Addison Wesley (Person Education Limited), 2005. [21] PhysioNet, "The research resource for complex physiologic signals," http://www.physionet.org/. [22] A. L. Goldberger, L.A.N. Amaral, L. Glass, J.M. Hausdorff. P.Ch. Ivanov, R.G. Mark, J.E. Mietus, G.B. Moody, C.K. Peng, and H.E. Stanley, "PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals," Circulation, vol. 101, pp. 215-220, June 13 2000. [23] P. Laguna, RG Mark, A Goldberg, and GB Moody, "A Database for Evaluation of Algorithms for Measurement of QT and Other Waveform Intervals in the ECG," Computers in Cardiology, vol. 24, pp. 673-676, 1997. [24] T.-W. Shen, "Biometric Identity Verification Based on Electrocardiogram (ECG)," in Biomedical Engineering. vol. PhD Madison: University of Wisconsin - Madison, 2005, p. 178. [25] C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer, 2006. [26] J. Copeland, and Diane Proudfoot, "Alan Turing's Forgotten Ideas in Computer Science," Scientific American, pp. 99 - 103, 1999. [27] Schey, N. C., Song Identification Using the Numenta Platform for Intelligent Computing. B.S. Thesis, The Ohio State University, 2008. [28] Bonhoff, G. M., Using Hierarchical Temporal Memory for Detecting Anomalous Network Activity. Ph.D. Dissertation, Air Force Institute of Technology, 2008. [29] Perea, A. J., Merono, J. E., and Aguilera, M. J., “Application of Numenta Hierarchical Temporal Memory for land-use classification,” South African Journal of Science, vol. 105, no. 9-10, pp. 370–375, 2009. [30] Squire, Larry R. (Editor)., Encyclopedia of Learning and Memory. New York, NY. Macmillian Publishing Company, 1992. [31] Q. Xue, Y.H. Hu, and W.J. Tompkins, "Neural-network-based adaptive matched filtering for QRS detection," IEEE Transactions on Biomedical Engineering, vol. 39, pp. 317-329, 1992. [32] H. Holst, Mattias Ohlsson, Carsten Peterson, and Lars Edenbrandt, "A confident decision support system for interpreting electrocardiograms," Clinical Physiology, vol. 19, pp. 410 - 418, 1999. [33] B. Heden, Mattias Ohlsson, Ralf Rittner, Olle Pahlm, Wesley K. Haisty, Carsten Petterson and Lars Edenbrandt, "Agreement Between Artificial Neural Networks and Experienced Electrocardiographer on Electrocardiographic Diagnosis of Healed Myocardial Infarction," JACC, vol. 28 No. 4, pp. 1012 - 6, October 1996. [34] Josh Hartung, Jay McCormack and Frank Jacobus, “Support for the use of Hierarchical Temporal Memory Systems in Automated Design Evaluation: A First Experiment,” Proc. ASME 2009 International Design Eng. Tech. Conf & Computers and Information in Eng. Conference. Aug 30 – Sept 2, 2009. San Diego, CA, USA. Pp 1 – 10.