This project started in August 2004. Preliminary results published in two conference papers:

1. Grant Number: IIS- 0347148Institution of PI: The Pennsylvania State UnivPIs: James Z. WangTitle: Career: Machine Learning Based Intelligent Image Annotation and Retrieval This Career project aims at developing an interdisciplinary research and education program for investigating the underlying theoretical and computational principles of machine learning based image annotation and retrieval. Among other goals, we plan to investigate the following areas: • This project started in August 2004. • Preliminary results published in two conference papers: • Jia Li, Dhiraj Joshi and James Z. Wang, ``Stochastic Modeling of Volume Images with a 3-D Hidden Markov Model,'' Proc. IEEE International Conference on Image Processing (ICIP), Singapore, 4 pages, IEEE, October 2004. • Dhiraj Joshi, James Z. Wang and Jia Li, ``The Story Picturing Engine: Finding Elite Images to Illustrate a Story Using Mutual Reinforcement,'' Proc. 6th International Workshop on Multimedia Information Retrieval, in conjunction with ACM Multimedia, 8 pages, New York, NY, ACM, October 2004. • The journal versions of this work will be submitted later this year. • Multidimensional Hidden Markov Models • Construction of a 3-D HMM • Implementation of an efficient algorithm • Modeling of volume images • Refine the learning process for the ALIP system • (Automatic Linguistic Indexing of Pictures) • Story Picturing Engine • Illustration of story text with pictures • Quantification of image representativeness • Building an online interface • Providing real time solutions to users • Evaluating the approach In this project, we will explore mathematical models and develop intelligent solutions for computerized content analysis of digital images. We will integrate several types of media from various sources in order to enhance users’ abilities to find the information they need. We will work with our collaborators to apply this technology to solving real world problems. On the educational front, we will involve researchers and students of various expertise. The following specific approaches are adopted for the research areas: • Story Picturing Engine • Image selection based on the story • Incorporating both lexical and visual features • Associating an importance measure to images • Illustration of story with machine selected pictures • Multidimensional Hidden Markov Models • Volume image modeled as a 3-D HMM • Spatial dependencies between image voxels • captured by model • Model parameters estimated with an iterative • procedure using maximum likelihood criteria A flow diagram of the Story Picturing Engine • Education: story picturing for school learning • Human development: training both undergraduate and graduate student researchers • Promoting interdisciplinary research: • bridging the gap among humanities, computer science, and mathematics • Tools: developing tools for image management, browsing, and retrieval • International and industrial collaborations: • strengthen and expand collaborations on application areas: biomedicine, homeland security, Web Segmentation of noisy volume images using 3-D HMM