1 / 33

Multimedia Analytics

Multimedia Analytics. Jianping Fan Department of Computer Science University of North Carolina at Charlotte. Presentation Outlines. What is Multimedia Analytics in my mind? What multimedia analytics can do for Multimedia Computing ? What we (UNCC team) have done so far ?.

gisela
Download Presentation

Multimedia Analytics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimedia Analytics Jianping Fan Department of Computer Science University of North Carolina at Charlotte

  2. Presentation Outlines • What is Multimedia Analytics in my mind? • What multimedia analytics can do for Multimedia Computing? • What we (UNCC team) have done so far?

  3. What is Multimedia Analytics? Multimedia analytics is science of multimedia computing facilitated by visual interface for interactive user’s inputs & assessments! Machine Side Human Side multimedia computing Assess & Inputs 1. Multimedia Rept. 2. Computing Hypos 3. Decision Function 4. Computing Results Visual Analytics 1. Multimedia Rept. 2. Computing Hypos 3. Decision Function 4. Computing Results Semantic Gap

  4. What is Multimedia Analytics? Multimedia Analytics Loop • Machine-Based Multimedia Computing; • Visualization of Multimedia Data, Knowledge, • Hypotheses, Decision Functions & Results; • c. Human-Computer Interaction for assessing & • changing hypotheses for multimedia computing

  5. What Multimedia Analytics can do for Multimedia Computing? Multimedia Computing sampling & projection Multimedia Analytics Decision adjustment Hypothesis updating Decision making Knowledge & Hypotheses Visualization Hypothesis visualization User-System Interaction & Assessment Hypothesis assessment

  6. What Multimedia Analytics can do for Multimedia Computing? • Hypothesis & Decision Function Visualization and Assessment • Computing Results & Knowledge Visualization, Exploration and Assessment • Leveraging the advantages of both human beings on creative thinking and computers on large memory and computing capacity.

  7. What we have done so far a. Interactive Similarity Function Assessment

  8. What we have done so far a. Interactive Similarity Function Assessment

  9. What we have done so far a. Interactive Similarity Function Assessment

  10. What we have done so far a. Interactive Similarity Function Assessment

  11. What we have done so far a. Interactive Similarity Function Assessment Scalability Inter-cluster visual correlation visualization

  12. Feature Quality Evaluation

  13. What we have done so far a. Interactive Similarity Function Assessment • Similarity function is suitable for measuring visual similarity contexts? • Similarity function combination (kernel combination) is good? • HD projection can precisely preserve original visual similarity contexts? • Feature quality is good?

  14. What we have done so far Misleading Effects: Data Uncertainty Junk Images

  15. What we have done so far Misleading Effects: Data Uncertainty Duplicates/Near-Duplicates

  16. What we have done so far Misleading Effects: Data Uncertainty Multiple text terms may share similar semantic meaning! One single text term may have multiple semantic meanings!

  17. What we have done so far Misleading Effects: Data Uncertainty Multi-Modal Information Association

  18. What we have done so far Misleading Effects: Data Uncertainty

  19. What we have done so far b. Interactive Decision Function Evaluation SVM Decision Boundary Visualization & Assessment

  20. What we have done so far b. Interactive Decision Function Evaluation Concept 1 Concept 2 GMM Model Visualization

  21. What we have done so far b. Interactive Decision Function Evaluation

  22. Interactive User Involvements via labeling

  23. What we have done so far b. Interactive Decision Function Evaluation • Updating kernel combinations (kernel weights); • Updating projection optimization criteria to preserve similarity better! • Updating decision function: margin between positive samples and negative samples! • Updating hypotheses for data representation & similarity characterization!

  24. Enlarge the margin between two classes!

  25. Larger margin has good generalization property!

  26. What we have done so far c. Interactive Knowledge Exploration

  27. What we have done so far c. Interactive Knowledge Exploration

  28. What we have done so far c. Interactive Knowledge Exploration

  29. What we have done so far c. Interactive Knowledge Exploration

  30. What we have done so far d. Collaborative Multimedia Analytics People have strong motivations (self or social) to perform collaborate multimedia understanding!

  31. What we have done so far d. Collaborative Multimedia Analytics

  32. What we have done so far • Learning task and training group organization; • Human-computer communication on data and knowledge representation; • Human-human communication on hypotheses and knowledge. d. Collaborative Multimedia Analytics

  33. Thank You for Your Attention! Q & A! More information is available at: http://www.cs.uncc.edu/~jfan

More Related