GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT

1. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Research Projects Transforming Communication between Deaf and Hearing people using Pattern Recognition Brain Prosthesis – An Advanced Signal Analysis and Classification of P300 ERP Signal- Component in BCI Measuring and Improving Human Video Surveillance Performance Computational Fluid Dynamics of Pharmaceuticals Processing Marine Security: Towards Maritime Sentry System Red Tide Detection and Tracking Center of Excellence in Pattern Recognition

2. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Principal Investigators Emanuel Donchin, College of Arts and Sciences Dmitry Goldgof, College of Engineering Lawrence Hall, College of Engineering Chad Lembke, College of Marine Science Barbara Loeding, College of Education Rangachar Kasturi, College of Engineering Dmitry Khavinson, College of Arts and Sciences Frank Muller-Karger, College of Marine Science Scott Sampson, SRI (was with College of Marine Science) Ravi Sankar, College of Engineering Thomas Sanocki, College of Arts and Sciences Sudeep Sarkar, College of Engineering Aydin Sunol, College of Engineering Center of Excellence in Pattern Recognition

3. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Graduate Students Weijan Cheng Sergiy Fefilatyev Jayapragas Gnaniah Siri Kamp Valentina Korzhova L. Kuznia Kun Li Vasant Manohar B. Smeltzer Inia Soto Noah Sulman Center of Excellence in Pattern Recognition

4. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Summary of Activities Pattern Recognition Technology Showcase Distinguished Speaker Series International Conference on Pattern Recognition Florida Imaging and Recognition Systems and Technology (FIRST) Center of Excellence External Grant Proposals New Courses Cluster Computing System Center of Excellence in Pattern Recognition

5. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Pattern Recognition Technology Showcase USF has extensive expertise in Video and Image Analysis, Knowledge Discovery & Language Learning, Medical Data Analysis and Bioinformatics technologies. These technologies result in commercial products and services that enhance the Security, Health, and Quality of Life of our citizens. These technologies were demonstrated to the public at the Pattern Recognition Technology Showcase held at the Sam and Martha Gibbons Alumni Center – Traditions Hall on September 28, 2007 Center of Excellence in Pattern Recognition

6. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Distinguished Speaker Series Sharat Chandran, IIIT – Bombay, India, “Modeling Translucency in Image Based Relighting, August 29, 2007 Horst Bunke, University of Bern, Switzerland, A family of novel graph kernels for structural pattern recognition, September 4, 2007 Demetri Terzopoulos, University of California, Los Angeles, Artificial animals and humans – From biomechanics to intelligence, October 3, 2007 Alexander Vasilev, University of Bergen, Norway, Pattern recognition: Energy of the Laplace evolution, December 14, 2007 Mark Mineev-Weinstein, Los Alamos National Labs, New Mexico, The Laplacian growth: Physics, mathematics, and algorithms for shape recovery, January 14, 2008 Center of Excellence in Pattern Recognition

7. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT International Conference on Pattern Recognition In a World-wide Competition, USF was Chosen to Host the 2008 International Conference on Pattern Recognition (www.ICPR2008.org) This Premier International Conference has not been held in the USA since 1990 This is a Great Opportunity for Florida-based Companies to Showcase their Products to some 2,000 Delegates from Around The World Center of Excellence in Pattern Recognition

8. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT Florida Imaging and Recognition Systems and Technology (FIRST) Center of Excellence We have submitted a $20 Million research proposal in partnership with the University of Central Florida to the Florida State Centers of Excellence competition. Much of the research described in this presentation will serve as the foundation to the activities of the proposed center. Center of Excellence in Pattern Recognition

9. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT External Grant Proposals ElasticFace: Characterization of Facial Dynamics in Video (NSF) (PI - Goldgof) Evaluation of Smart Video for Transit Event Pattern Detection, Federal Department of Transportation (FDOT-NCTR), (PI - Goldgof, Sapper) Tracking Fluid Flow Parameters in Video Data ,  (NSF) (PI: Goldgof) iSIMON: Development of an Automated Intelligent Sign Language Monitor to Facilitate On Demand Sign Language Instruction and Learning, US Department of Education, (PIs: Loeding, Sarkar) Gravitational Lensing, Complex Lightning Bolts and Complex Analysis (NSF, PI: Khavinson) Bottom Stationed Ocean Profiler Design Improvements (ONR, continuation funding, Lead: C. Lembke) MERHAB: Eastern Gulf of Mexico Sentinel Program (NOPP/NOAA continuation funding, Lead: C. Lembke) A P300 Brain Computer Interface used to Tele-operate a Robotic Arm Mounted to a Mobile Platform, National Institute of Health (NIH), Lead: E. Donchin) Center of Excellence in Pattern Recognition

10. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT New Courses CIS 6930 Three-Dimensional Data from Images - Spring 2008, Goldgof CIS 6930 Video Processing - Spring 2007, Goldgof CIS 6930 Document Image Analysis, Fall 2007, Kasturi EXP 7099 Tools for Language Discovery, Fall 2007, Sanocki Center of Excellence in Pattern Recognition

11. GRADUATE EDUCATION AND RESEARCH IN PATTERN RECOGNITION INTERIM PROGRESS REPORT • Cluster Computing System • Initiated construction of 64 cores Compute Cluster • Intel Xeon X5355 Clovertown 2.66GHz (Quad-core) • 256 Gbyte of memory distributed on 8 compute nodes • Interconnected via 1000 Mbps Ethernet switch • 12 TB disk array • 340.48 GFlops max theoretical performance • All parts received and assembled • All CPUs are tested • Initiated benchmarking for final testing • Available for users – end of Spring 2008 semester Center of Excellence in Pattern Recognition

12. Red Tide Detection and Tracking L.O. Hall, F. Muller-Karger, C. Hu, D. Goldgof, W. Cheng, I. Soto Computer Science and Engineering and Marine Science Recipe Harmful algae are microscopic, single cell plants Occasionally, the algae grow very fast (bloom) Accumulate into dense, visible patches The can appear red, orange, brown, or greenish depending the organism or the waters. Use Satellite imaging to find blooms in conjunction with Data Mining. Karenia Brevis The following 5 attributes from MODIS satellite images were used: chlorophyll, fluorescence line height, normalized water-leaving radiance at 412 nm, at 443 n and at 488 nm. We utilized Fuzzy C Means clustering to cluster seawater in potential red tide regions into 4 different clusters or classes. Different clusters are labeled with different colors. The cluster centroid with the highest fluorescence line height value was recognized as red tide (colored in bright red). We tested our algorithm on nine different days in 2004 and 2005 and achieve an overall accuracy of 66.13% vs. pixels known to contain red tide through physical measurement. Ground truth red tide spots are marked as blue crosses in the figure while the non-red tide spots as white crosses. Consequences Location and Frequency Center of Excellence in Pattern Recognition

13. Marine Security: Towards Maritime Sentry System Sergiy Fefiliatiev, Dmitry B. Goldgof 1, Chad Lembke, Scott Sampson 2 1 Department of Computer Science and Engineering, College of Engineering, University of South Florida 2 Center for Ocean Technology, University of South Florida, SRI International Experiments In this work a dataset of 100 images of open sea with horizon was used. Most of the images had multiple ships present on the horizon line; maximum number of ships in an image was 6. Some of the images did not include any ships or floating objects. Manual search for parameters of the Canny edge detector was conducted on a separate dataset of 27 images. The next table shows accuracy of ship detection and false alarms depending on threshold value Abstract This project presents a new algorithm for automatic detection of marine vehicles in open sea from a buoy camera system. Users of such system include border guards, military, port safety and flow management, sanctuary protection personnel. To reduce human effort in the venture, it is desirable to build a system that would perform such a task autonomously, detecting the vessels in the sea in a robust manner. The goal of the system is to detect an approximate window around the ship and prepare the small image for transmission and human evaluation. The proposed algorithm combines horizon detection method with edge detection and post-processing. The dataset of 100 images is used to evaluate the performance of proposed algorithms. Overview Algorithm Description We propose a semi-automated computer system that detects ships on the horizon, using a computer vision approach. Such system will be equipped with a digital camera, located on a buoy and will work in autonomous mode taking images of the surrounding area. After processing the image information the system will send only images of the found objects to a human operator for further evaluation and action. One of the constraints for such a system is low communication bandwidth available while reporting the results. Hence, one of the requirements lies in efficient compression of the obtained visual results before they are sent to the control center. The algorithm consisting of six steps. The image acquisition step uses a digital camera installed on the buoy for acquiring images of the surrounding area. For edge detection step we have chosen Canny edge detector. Horizon detection step is the most essential step in the algorithm - the found horizon line is used to eliminate all edges in the edge-image that would not belong to floating objects – we expect that in an image all objects of interest, i.e. ships, lie above the horizon line. The purpose of the post-processing step is to connect closely lying separate edges belonging to a single object and to eliminate some edges that would not belong to that object. The output result consists of a region of the original image inside the bounding box around found objects. Summary and Conclusions High detection rate with a moderate alarm rate for low threshold values is recorded. The visual results for the experiment reflect the fact that a typical failure for ships detection occurs due to a multiple region representation of single objects. One of the proposed future improvements is the introduction of a training step in order to identify parameters of the Canny edge detector. Another enhancement may include a ‘merge’ procedure, which would connect closely lying ship regions. Center of Excellence in Pattern Recognition

14. Image preprocessing (thresholding and local enhancement) Top of the wave point detection Determination the two neighboring maxima correspond to the image intensity and belong to the same radius on the disk (semi-manual) Finding the tangent at the given point to the circle and to the spiral Return pump Top reservoir Control box Calculating the inclination angle between tangents at the given point Calculating the Euclidian distance between two consecutive waves Flow meter Cupper tubing Rotatingdisk Bottom Reservoir andStand a) b) Computational Fluid Dynamics of Pharmaceuticals Processing Dmitry B. Goldgof, Grigori M. Sisoev, Aydin Sunol, Dmitry Khavinson, College of Engineering, College of Arts and Sciences Abstract Theory and Results The novel video-based algorithms for detection and tracking of spiral waves in a spinning disk reactor are presented. They are based on processing of experimental video data consisting of the discrete field of disk point coordinates and its intensities. The experimental results are compared with theoretical results. Also, model-based algorithms to estimate the disk speed and the fluid-flow rate in a spinning disk reactor are developed. The block-scheme of wave detecting and estimating the wave parameters The resulting images Estimation of fluid flow parameters -wavelengths -inclination angles -radial velocity Comparison Evolution equations Flow is governed by the continuity, the Navier-Stokes equations, and an appropriate set of boundary conditions. Derived model is described in1) with the addition of non-axisymmetric terms. A localized version of equations is used for computation of parameters of developing waves. Camera calibration accuracy Considering lens distortion of a camera with coefficients k1 and k2 andthe standard relative deviations for estimates of k1 and k2 do not exceed 3-4%, one has where s is scale factor, x, y are image coordinates and X, Y are world coordinates. Spiral detection algorithm Considering the spirals below as periodic functions with The period ∆Φ in the polar system of coordinates r = rj(φ) : rj(φ + ∆Φ ) = rj(φ), ∆ φ be the angle-step of the calibration, N = ∆Φ / ∆ φ > 1 is integer, and φ = φi = φ0+ i∆ φ, i = 1, 2, ...,N; rj(φ0) = r0 = min rj(φ); r0 < ri1 < … < riS < 100, where Sis the number of spirals for each i, rij are experimental data for φ = φi; the points (i, rij) are on the respective spirals. Then 1st spiral is: (φ1, r11), (φ2, r21),(φ3, r31), ...,(φN, rN1);(φN+1, r12),(φN+2, r22), ...,(φ2N, rN2); ...;(φ(S-1)N+1, r1S), (φ(S-1)N+2, r2S), ...,(φSN, rNS); ………………………………………………………………… Let ∆r be the radius-step of calibration and (φi, k∆r; Iik), i = 1, 2, ...; k = 1, 2, ..., be the calibration net on the contracted disk, then where Iik are the intensities of the points (φi, k∆r) . ------------------------------------------------------------------------------------------------------------------ 1)G.M. Sisoev, O.K. Matar, and C.J. Lawrence, ”Axisymmetric wave regimes in viscous liquid film flow over a spinning disk”, J. Fluid Mech 495, 385-411 2003. b) a) c) The calculated averaged a), b) inclination angles β and c) their changes ∆β compared to the predicted. Data Acquisition Radial velocity component of wave flow. Estimation of inclination angle and radial velocity component is ill-posed problem, so the asymptotically optimal method is used to determine the step of differentiation under the initial data accuracy ε given Respective hardware(Figure above) Physical measurements Series of videos(Figures below) Main applications - chemical (processes of heat or mass transfer between expanded liquid and surrounded gas) - medical ( blood oxygenation) - engineering (cooling devices) • Recovery of controlling parameters • Fluid flow rate • Disk speed Determination of β and φ Conclusion The developed algorithms allow for recovery of both observed flow parameters (including wavelengths, inclination angles and radial velocity components) and controlling parameters. The inclination angles between spirals and the respective circles and radial velocity components are calculated using the asymptotically optimal method. Results computed from video data are compared with number predicted by the theoretical model. c) The calculated averaged wavelengths a) of the sequence of ten frames ,b) over five videos compared with theoretical wavelengths and c) wavelength vs. radius, asterisks and o-s denote experimental and theoretical values. Center of Excellence in Pattern Recognition

15. fs-JOHN CAN BUY HOUSE FUTURE SHE WOMAN HER HOUSE FIRE Automated Detection of Sign Language Patterns Sudeep Sarkar, Barbara Loeding, Sunita Nayak , Alan Yang Department of Computer Science and Engineering, Department of Special Education Goal and Impact Statement Representation that does not require tracking Movement Epenthesis Aware Matching • Goal: • To advance the design of robust computer representations • and algorithms for recognizing American Sign Language from • video. • Broader Impact: • To facilitate the communication between the Deaf and the hearing population. • To bridge the gap in access to next generation Human Computer Inferences. • Intellectual Merit: • We are developing representations and approaches that can • Handle hand and face segmentation (detection) errors, • Learn, without supervision, sign models from examples, • Recognize in the presence of movement epenthesis, i.e. hand movements that appear between two signs. • We have proposed a novel representation that captures the Gestalt configuration of edges and points in an image. • It can work with fragmented noisy low-level outputs such as edges and regions • It captures the statistics of the relations between the low-level primitives • Distance and orientation between edge primitive. • Vertical and horizontal displacement • Relationships between short motion tracks Movement epenthesis is the gesture movement that bridges two consecutive signs. This effect can be over a long duration and involve variations in hand shape, position, and movement, making it hard to model explicitly these intervening segments. This has been a problem when trying to match individual signs to full sentences. We have overcome this with a novel matching methodology that do not require modeling of movement epenthesis segments. • Normalized RD is an estimate of Prob (Any two primitives in the image exhibit a relationship) • The shape of the RD changes as parts of the objects move. • Relational distributions over time model high-level motion patterns. Unsupervised Learning of Sign Models Learn sign model given example sentences with one sign in common. In the following two sentences, the target sign model to be learned is HOUSE (marked in red) The error rates for enhanced Level Building (eLB) (our method), which accounts for movement epenthesis, and classical Level Building (LB) that does not account for movement epenthesis. Segmentation Aware Matching • Frag-Hidden Markov Models: • Groups across frames are linked • Best match is a path in this induced graph over groups • Matching involves optimization over states AND groups for each frame Publications and Acknowledgement • R. Yang; S. Sarkar, B. Loeding, Enhanced Level Building Algorithm for the Movement Epenthesis Problem in • Sign Language Recognition, to be presented at IEEE Conf. on Computer Vision and Pattern Recognition, 2007. • R. Yang; Sarkar, S., “Gesture Recognition using Hidden Markov Models from Fragmented Observations,” IEEE Conference on Computer Vision and Pattern Recognition pp. 766- 773, 17-22 June 2006. • R. Yang and S. Sarkar, “Detecting Coarticulation in Sign Language using Conditional Random Fields,” • International Conference on Pattern Recognition vol.2, pp. 108- 112, 20-24 Aug. 2006 • S. Nayak, S. Sarkar, and B. Loeding, “Unsupervised Modeling of Signs Embedded in Continuous Sentences,” • IEEE Workshop on Vision for Human-Computer Interaction, vol. 3, pp. 81, June 2005. • R. Yang, S. Sarkar, B. L. Loeding, A. I. Karshmer: Efficient Generation of Large Amounts of Training Data for • Sign Language Recognition: A Semi-automatic Tool. ICCHP 2006: 635-642 • B. L. Loeding, S. Sarkar, A. Parashar, A. Karshmer: Progress in Automated Computer Recognition of Sign • Language. ICCHP 2004: 1079-1087 • This work was supported in part by the National Science Foundation under ITR grant IIS 0312993. Center of Excellence in Pattern Recognition

16. What are the Limits of Human Ability in Video Surveillance? Rangachar Kasturi 1, Thomas Sanocki 2, Dmitry Goldgof 1, Sudeep Sarkar 1,Vasant Manohar 1, Noah Sulman 2 1 Department of Computer Science and Engineering, College of Engineering, University of South Florida 2 Department of Psychology, College of Arts and Sciences, University of South Florida • Objective: To conduct a formal psychological study on the efficiency of the human operator in video surveillance and suggest recommendations, if any, based on the research findings. • What are the research questions? • How many simultaneous windows of videos can the analyst effectively monitor? • Given a target information to monitor, what is the ability of the analyst to detect this event and how does this vary as a function of number of windows? • Is the current method of presenting the raw data to the analyst sufficient? • Do the patterns revealed in the earlier experiments vary across expert and non-expert populations? If the results vary, what is an effective and systematic way to train non-experts? Center of Excellence in Pattern Recognition

17. Signal Analysis and Classification of P300 ERP Signal Component in Brain Computer Interface (BCI) Emanuel Donchin, Ravi Sankar, Kun Li, Siri-Maria Kamp, David Seebran and Yael Arbel 1 Department of Psychology, College of Arts and Sciences, University of South Florida 2 Department of Electrical Engineering, College of Engineering, University of South Florida • Objective • Design new P300 based analysis algorithms; Increase P300 detection speed with high accuracy; Therefore achieve effective • communication speed for locked-in people. • What are the research questions? • How to increase the P300 detection speed and accuracy? • What are the complexities of the new algorithms and how to relieve the computation load? • Does the current raw data provide sufficient information to the analyst? • How to increase the data transfer rate for effective communication? Center of Excellence in Pattern Recognition