1 / 49

Mor Naaman, Yee Jiun Song, Andreas Paepcke, Hector Garcia-Molina

Automatic Organization for Digital Photographs with Geographic Coordinates. Mor Naaman, Yee Jiun Song, Andreas Paepcke, Hector Garcia-Molina. Digital Library Project, Database Group Stanford University. April 8 th , 2004 1:20:02pm. Latitude: N34.3121 Longitude: W122.234.

mercer
Download Presentation

Mor Naaman, Yee Jiun Song, Andreas Paepcke, Hector Garcia-Molina

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic Organization for Digital Photographs with Geographic Coordinates Mor Naaman, Yee Jiun Song, Andreas Paepcke, Hector Garcia-Molina Digital Library Project, Database Group Stanford University

  2. April 8th, 20041:20:02pm Latitude: N34.3121 Longitude: W122.234 Geo-Referenced Photos JCDL 2004

  3. Geo-Photography Technology 1) 2) + JCDL 2004

  4. Personal Photo Libraries • Searching/browsing very difficult • Little discernible structure to photo collections JCDL 2004

  5. Managing Personal Photos • Content-based retrieval • Basic, primitive (far from semantic) • Manual labeling • Improved, yet cumbersome • Visual methods for fast scanning (Zoom) • Don’t scale well JCDL 2004

  6. Our Approach • Absolutely no human effort required • Utilize time and location • Automatically captured • Easy to get JCDL 2004

  7. Automatic Organization JCDL 2004

  8. Automatic Organization JCDL 2004

  9. Automatic Organization JCDL 2004

  10. Automatic Organization JCDL 2004

  11. Outline • Requirements and challenges • The algorithms • Sample output • Experiment results JCDL 2004

  12. Browsing by Location/Time • Use a map/calendar • wwmx.org from MSR: • Map issues • Lots of screen space • Sparse • Limited interaction? • Not intuitive for some JCDL 2004

  13. Time: 2003-01-01: Yosemite N.P. (2 Days) 2003-01-18: San Francisco (1 hour) 2003-01-18: San Francisco (1 hour) Using Hierarchies Location: United States Yosemite N.P, Yosemite Valley, CA Around: San Francisco, Berkeley, Sonoma CA Seattle, WA … San Francisco, Golden Gate Park, CA Berkeley, Oakland CA … … … Time

  14. Challenges • Locations should be intuitive • Events are tricky • 3-days trip to NYC • The kid’s soccer game, followed by a birthday party • Good names are important. JCDL 2004

  15. Outline • Requirements and challenges • The algorithms • Sample output • Experiment results JCDL 2004

  16. Process Diagram JCDL 2004

  17. Initial Event Segmentation Discovering Structure Automatic Organization Event Hierarchy Final Event Segmentation Location Hierarchy Initial Event Segmentation Location Clustering JCDL 2004

  18. Initial Event Segmentation • Photos occur in bursts • Identify bursts: semantically “connected” JCDL 2004

  19. Initial Event Segmentation Stream of photos • More details: • Graham et al, JCDL 2002 • Tomorrow • Proceedings JCDL 2004

  20. Location Clustering Discovering Structure Automatic Organization Event Hierarchy Final Event Segmentation Location Hierarchy Initial Event Segmentation JCDL 2004

  21. Location Clusters • Cluster the bursts into locations • A. Gionis and H. Mannila. Finding recurrent sources in sequences. InProceedings, Computational molecular biology 2003. • Minimize: number of clusters • Minimize: error (distance to cluster centers) JCDL 2004

  22. Photo location Location Clusters: 2-D View

  23. 2-D View: with Bursts

  24. Location4- Location3- Location2- Location1- Location Clusters JCDL 2004

  25. Location4- Location3- San Francisco Location2- Location1- Location Clusters (breakdown) • Some clusters may be overloaded: • Many bursts / picture-taking days in one location

  26. Final Event Segmentation Discovering Structure Automatic Organization Event Hierarchy Location Hierarchy Initial Event Segmentation Location Clustering JCDL 2004

  27. Final Event Segmentation • Again scan sequence, new events detected: • Whenever location context changes • In the same location, use adaptive time threshold JCDL 2004

  28. Overnight trip to Yosemite Soccer game and dinner Final Event Segmentation JCDL 2004

  29. Next - names • Detected location and event structure • Need to choose names for each node JCDL 2004

  30. Palo Alto City Park Butano State Park Palo Alto Stanford Photo location Assigning Names

  31. San Francisco, 30 miles San Jose, 20 miles Assigning Names – Nearby? What if photos occur sparsely within cities or parks?

  32. Assigning Names - Nearby Which city has stronger “gravity”? JCDL 2004

  33. Assigning Names - Nearby San Jose is Closer JCDL 2004

  34. Assigning Names - Nearby San Jose is bigger* *larger population JCDL 2004

  35. Assigning Names - Nearby But San Fran is more important!* *greater Google count Final name for location cluster: “Stanford, 30 miles South of SF” JCDL 2004

  36. Assigning Names - Alexandria • Using polygon-based dataset of administrative areas • Alexandria gazetteer can be used for other prominent geographic features JCDL 2004

  37. Outline • The requirement and challenges of automatic organization • The algorithms • Sample output • Experiment results JCDL 2004

  38. Location Hierarchy Photoshop Album (at least 4 man-hours) Our system (about 0 man-seconds) JCDL 2004

  39. Location Hierarchy (US) • San Francisco, Berkeley, Sonoma, CA • -Stanford, Mountain View, Monterey, CA • Monterey (58 miles S of San Jose)  • Mountain View (4 miles NW of San Jose)  • Stanford • -Colorado (219 miles W of Denver) • -Long Beach (35 miles S of Los Angeles, CA) • -Philadelphia, PA • -Seattle, WA • -Sequoia N.P. (153 miles E of Fresno, CA) • -South lake Tahoe; Bear Valley, CA • -Yosemite N.P.; Yosemite Valley, CA 

  40. Events Photoshop Album (at least 4 man-hours) ... 2003-06-28: Long Beach,CA (3 days) 2003-07-04: San Francisco,CA (3 hours) 2003-07-10: Colorado (3 days) 2003-07-15: San Francisco,CA(1 hours) 2003-07-18: Mountain View,CA (5 hours) 2003-07-27: San Francisco,CA (1 hours) 2003-09-28: Philadelphia,PA (1 hours) 2003-10-03: Sequoia NP (3 days) ... about 0 man-seconds:

  41. Event Names • LOCALE: share automatically • Check personal calendar • Event Gazetteer • Easy interface JCDL 2004

  42. Experiment • Tested on 3 real-world geo-referenced photo collections • Our system automatically generated the structure and names • Tested with the owners JCDL 2004

  43. Experiment - Locations • Accepted the automatic hierarchy • Only minor edits requested • Merge/split few of the locations JCDL 2004

  44. Experiment - Events • Compared to events as annotated by users • 80-85% in both recall and precision • Other metrics proposed (see paper) JCDL 2004

  45. Experiment - Naming • Naming location clusters • For 76% of clusters, system and users pick at least one name in common • For the rest, “automatic” name was useful JCDL 2004

  46. Not yet published: • Paid 13 participants to “geo-reference” their photos • Loaded to WWMX and our browser • Most liked the map better, but… • Performed the same for search/browse tasks • Event notion helps overcome location handicap • Organization “made sense” P.S. Some didn’t touch the map, yet used our location hierarchy. P.S.2 This was on a BIG screen!

  47. Thank You! More details: Proceedings Google: Mor Naaman mor@cs.stanford.edu http://www-db.stanford.edu/~mor/ JCDL 2004

  48. Future Work • User interface • PDA • Integrate with map • Global photo libraries JCDL 2004

  49. Remember The Bursts? JCDL 2004

More Related