Multilevel Category Structure in the ART-2 Network

1. Multilevel Category Structure in the ART-2 Network Advisor ：Dr. Hsu Graduate： Yu Cheng Chen Author: Michael P. Davenport, Albert H. IEEE Transactions on Neural Network (2004)

2. Outline • Motivation • Objective • Introduction • Background • ART2 Neural Network • Analysis of Multilevel Category Structure • Conclusions

3. Motivation • Real-world problems require that categorization is performed on complex data sets, which points to the need for a modular approach to systems design. • The variations of the ART network have proliferated during the last decade, very few if any of those variations have emphasized psychological or neurobiological principles in their design.

4. Objective • By exchanging a single node category description for a more complex category pattern description, the distribution of information processing is distributed beyond a single ART module, which is more consistent with studies of neurobiological organization.

5. Introduction • These categories are represented by prototype patterns. • In this work, we present a detailed analysis of an ART 2 network that is specifically tuned to extract secondary level features from an input data set. • Thus, ART2 network can be used to show not just what broad category an input belongs to, but also what features an input has that makes it different or similar to other inputs.

6. Background • Review of ART Theory

7. Background • The template STM layer forms a prototype or template category based on the previous experience stored in the LTM weights. • The category output of the network is based on the interpretation of activity across the template layer (also referred to as the output layer or the category layer). • Throughout the paper we use the terms single winning node and multiple winning nodes. These are nodes in the template or output layer.

8. Background • The term single winning node indicates that for every input pattern, the same output node always has the highest activity level. • The term multiple winning nodes indicates that while only one output node will have the highest level of activity for a given input pattern, not all the input patterns will cause the same output node to be most active.

9. Background • MultiLevel Categorization • Several variations of ART networks have been developed to perform multilevel categorization. • SMART、HART • SMART configured each individual ART network to have a different value for the vigilance parameter. • generate a pair of categories • more general category and more specific subcategory.

10. Background • In all the above models the ART networks share a common characteristic: they represent the category output as a single node — the most active node of the network. • In this paper, we demonstrate that the elements of a multilevel categorization can be obtained by using a single ART 2 network, and that this information is obtained by considering the pattern of activity acrossall the output nodes instead of considering only the most active.

11. Background • Psychological and Neurobiological Principles of Categorization • Psychological Categorization: • First, any system performing categorization should do so efficiently; the maximum amount of useful information should be obtained using a minimal amount of cognitive effort. • second, an organism perceives a high correlational structure among the features, attributes, objects, events, etc.

12. Background • Category structures vary across different levels of abstraction. • Categories within the same abstraction level are also characterized by not having clearly defined category boundaries which implies that every pattern is not equally representative of the category to which it belongs. • We demonstrate in our results that category representation as an analog pattern of activity can be described in a similar manner.

13. Background • Neurobiological Categorization: • Neurobiological studies indicate that many different and widely separated neural structures are involved in category learning. • Different neural structures may also exist for learning categories depending upon the type of category learning.

14. ART2 Neural Network

15. ART2 Neural Network • Each individual layer F1 is described by a two-step process (1)–(3). • First signals are combined and enhanced by gain parameters • Second, the overall activity across the layer is normalized to keep the total activity of the entire network bounded.

16. ART2 Neural Network • The template STM layer in Fig. 1 is represented in Fig. 2 by the F2 layer (category layer).

17. ART2 Neural Network • Model Parameter Selection • a=0.5,b=0.5,c=0.1,d=0.5,e=0.0001 • We have used the “zoo” and the “auto-mpg” databases, which are available at the University of California, Irvine, Information and Computing Science. • The dataset used was the “zoo” database consisting of 100 patterns. Each pattern consists of fourteen binary elements representing features associated with the animals: hair, feathers, eggs, milk, airborne, aquatic…

18. Analysis of Multilevel Category Structure • Table 1

19. Analysis of Multilevel Category Structure • The first case studies patterns generated by ART network where the vigilance parameter is set low, enough so that the same node is most active for every input pattern. • The output patterns corresponding to 43 of the 100 input patterns first case are shown in Fig. 3 (the x axis represents the ten output nodes, the y axis represents the analog activity level of a particular output node)

20. Analysis of Multilevel Category Structure Fig. 3

21. Analysis of Multilevel Category Structure • One important characteristic of these activity patterns is that the feature information is compressed, but still evident in the shape of the category pattern. • For example, the aquatic animals, essentially the fish and crustaceans of the second category, all have a characteristic hump around the nodes seven through ten.

22. Analysis of Multilevel Category Structure Fig. 4

23. Analysis of Multilevel Category Structure Fig. 5

24. Analysis of Multilevel Category Structure • Fig 6. shows the individual activity patterns generated when there are four different winning category nodes associated with the input data set.

25. Analysis of Multilevel Category Structure Fig. 6

26. Analysis of Multilevel Category Structure Fig. 7

27. Analysis of Multilevel Category Structure • There is clearly evidence for a hierarchical category description moving from general categories to more specific categories. • The graph in Fig. 7 also shows that it is possible to combine more specific categories into more general categories.

28. Discussion • Traditionally many ART systems have either been modified for use in specific applications, or have undergone design changes to build in ever more sophisticated controls that give the ART network increased ability to do more information processing in one place. • The category patterns have demonstrated such a variety of categories using a single network whose parameters do not need to be tuned or changed during learning and recall in order to make such category information available.

29. Conclusions • We have presented a novel approach to analyzing categories and multilevel category structure in the output patterns of an ART 2 network. • We have shown how general and specific categories can be derived from the same set of category data, which is not possible when defining a category simply by the winning node. • Finally, these findings have been compared favorably with principles of categorization from different disciplines.