1 / 16

Detecting False Captioning Using Common Sense Reasoning

Picture editing is prevalent across various fields, risking misinformation. This paper introduces a meticulous mathematical technique to distinguish true from false images by focusing on photomontage and false captioning methods and employing segmentation, classification, and common sense reasoning. Methods discussed include image segmentation, importance maps, ROI calculation, supervised and unsupervised classification, and reasoning to detect anomalies in images.

gnishimura
Download Presentation

Detecting False Captioning Using Common Sense Reasoning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Detecting False Captioning Using Common Sense Reasoning James Byrd

  2. Abstract • Picture editing is popular • Medical, Journalism, Science all at risk • how do we detect what is real from what is not? • We shall introduce a nearly flawless mathematical technique to tell the true from the false

  3. Intro • Pictures are a compelling way to communicate information • Why modify? • To make the picture tell a different story • “Yellow Journalism”

  4. Photo Manipulation • 4 ways • Deletion of Details - deleting details • Insertion of Details - inserting additional details • Photomontage - combining multiple images • False Captioning - misrepresenting image content

  5. This Paper’s Focus • Photomontage • False Captioning

  6. Steps for detection • Segmentation • Classification • Common Sense Reasoning

  7. Segmentation • Segment the image into regions • Make an “importance map” to compare across images in a given corpus

  8. Classification • Perhaps the most important part • Segment based color scheme • compare segments among images

  9. Common Sense Reasoning • 2 approaches • resolve local classification ambiguities within images; we will query a knowledge base to resolve proper relations • reason across a larger corpa of images to find unique or missing elements during an investigation

  10. Methods • Image segmentation • Importance Map • Calculating ROIs • Classification • Reasoning

  11. Image Segmentation • Segment the image • mean-shift image segmentation to decompose an image into homogeneous regions • choosing parameter values is often difficult • therefore we adds tons of colors and do many segmetns

  12. Importance Map • to truly understand the importance of an image requires a thorough understanding of what the image contains and what the viewer needs • recognize objects (faces) and regions are important

  13. Calculating ROI • 2 step process • Identify Candidate ROI • minimal image that identifies key important parts of the image • Grow the ROI • combine the ROI by using a clustering algorithm recursively

  14. Classification • Supervised Classification • Training Sites - predetermined objects of interest in an image • Software uses these to create a “signature analysis” • Unsupervised Classification • Takes a large number of ROI and divides them into classes based on grouping in them • becoming increasingly popular

  15. Reasoning • Resolve local ambiguities within an image • Answer questions the program has • i.e. water versus sky • Then the software determines if the anomalies are consistent across images or show signs of tampering • More pictures to examine are better for consistency

  16. Conclusion • methods are limited by the performance of the components of image segmentation and important object identification

More Related