Project MLExAI Machine Learning Experiences in AI http://uhaweb.hartford.edu/compsci/ccli

1. Project MLExAIMachine Learning Experiences in AIhttp://uhaweb.hartford.edu/compsci/ccli Ingrid Russell, University of Hartford Zdravko Markov, Central Connecticut State University Todd Neller, Gettysburg College

2. Project Goal • The project goal is to develop a framework for teaching core AI topics with a unifying theme of machine learning. A suite of hands-on term-long projects are developed, each involving the design and implementation of a learning system that enhances a commonly-deployed application. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

3. Project Objectives • Enhance the student learning experience in the AI course by implementing a unifying theme of machine learning to tie together the diverse topics in the AI course. • Increase student interest and motivation to learn AI by providing a framework for the presentation of the major AI topics that emphasizes the strong connection between AI and computer science. • Highlight the bridge that machine learning provides between AI technology and modern software engineering. • Introduce students to an increasingly important research area, thus motivating them to pursue more advanced courses in machine learning and to pursue undergraduate research projects in this area. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

4. Features of MLExAI Projects • Teaching AI with hands-on experiments. • Common features in different AI fields are unified through the theme of machine learning. • Emphasis on application of ideas through implementation. • Varying levels of mathematical sophistication with implementation of concepts being central to the learning process. • Design and implementation of learning systems. • Practical approach that includes real-world applications. • Easily adaptable and customizable. • Various emphases, backgrounds and prerequisites that can serve different goals within the general framework of teaching AI. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

5. MLExAI Projects • Web Document Classification: Investigates the process of tagging web pages using a topic directory structure and applies machine learning techniques for automatic tagging. • Data Mining for Web User Profiling Using Decision Tree Learning: Focuses on the use of decision tree learning to create models of web users. • Character Recognition Using Neural Networks: Involves the development of a character recognition system based on a neural network model. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

6. MLExAI Projects • Explanation-Based Learning and the N-Puzzle Problem: Involves the application of explanation-based learning to improve the performance of uninformed search algorithms when solving the N-Puzzle problem. • Reinforcement Learning for the jeopardy Dice Game “Pig”: Students model the game and several variants, implementing dynamic programming and value iteration algorithms to compute optimal play. • Getting a Clue with Boolean Satisfiability: We use SAT solvers to deduce card locations in the popular board game Clue, illustrating principles of knowledge representation and reasoning, including resolution theorem proving. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

7. Sample Project: Web Document Classification Goal To investigate the process of tagging web pages using the topic directory structure and apply machine learning techniques for automatic tagging. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

8. Web Document Classification Project Phases • Data collection – collecting a set of 100 web documents grouped by topic. Will serve as our training set. • Feature extraction and data preparation – web documents will be represented by feature vectors, which in turn are used to form a training data set for the Machine Learning stage. • Machine learning – applying learning algorithms to create models of the data sets. Using these models the accuracy of the initial topic structure is evaluated and new web documents are classified into existing topics. • Analysis – identifying relations between approaches used in the project and AI areas of search and knowledge representation and reasoning (KR&R) NSF CCLI Showcase, March 1-5, 2006. Houston, TX

9. Phase I: Data Collection Topic 1 Topic 2 Topic 3 Computers: Artificial Intelligence: Machine Learning Topic 1 Topic 2 Topic 4 Computers: Artificial Intelligence: Agents Topic 1 Topic 5 Topic 6 Computers: Algorithms: Sorting and Searching Topic 1 Topic 7 Topic 8 Computers: Multimedia: MPEG Topic 1 Topic 9 Computers: History NSF CCLI Showcase, March 1-5, 2006. Houston, TX

10. Phase II: Feature Extraction and Data Collection NSF CCLI Showcase, March 1-5, 2006. Houston, TX

11. Phase III: Machine Learning NSF CCLI Showcase, March 1-5, 2006. Houston, TX

12. Predicted Page Classification Predicted Page Classification Actual Page Classification Actual Page Classification Phase III: Machine Learning Error Analysis Plot Results ===== Stratified Cross-Validation ===== Correctly Classified Inst.(101) 87.069 % Incorrectly Classified Inst.(15) 12.931 % Kappa statistic 0.8379 Mean absolute error 0.0633 Root mean squared error 0.2245 Relative absolute error 19.7946 % Root relative squared error 56.1278 % Total Number of Instances 116 ===== Stratified Cross-Validation ===== Correctly Classified Inst.(101) 87.069 % Incorrectly Classified Inst.(15) 12.931 % Kappa statistic 0.8379 Mean absolute error 0.0633 Root mean squared error 0.2245 Relative absolute error 19.7946 % Root relative squared error 56.1278 % Total Number of Instances 116 NSF CCLI Showcase, March 1-5, 2006. Houston, TX

13. Preliminary Experiences • Preliminary results were positive and showed that students had good experiences in the classes. • While covering the main AI topics, the course provided students with an introduction to and an appreciation of an increasingly important area in AI, Machine Learning. • Using a unified theme proved to be helpful and motivating for the students. Students saw how simple search programs evolve into more interesting ones, and finally into a learning framework with interesting theoretical and practical properties. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

14. Preliminary Experiences: Student Quotes • Working on the project was a great experience. I was able to see how various AI concepts tie together in developing a machine learning system. I was amazed by the wide range of applications of machine learning in various aspects of our lives. • I liked acquiring knowledge about machine learning techniques and being able to implement a system and see it work. This gave me a concrete understanding of the concepts. • The project was really neat. I was challenged to strengthen my deductive reasoning skills by formalizing the process by which I derive solutions. The problems associated with “satisfiability” are fun to work out, but they also provide me with an intellectual challenge. • I liked the fact that the project I worked on pertained to the Internet and web document classification. It presented a useful real world application of machine learning. Often, examples in the book or other projects lack real world usefulness. NSF CCLI Showcase, March 1-5, 2006. Houston, TX

15. Acknowledgement This work is supported in part by National Science Foundation grant DUE CCLI-A&I Award Number 0409497. NSF CCLI Showcase, March 1-5, 2006. Houston, TX