1 / 95

Video-Based Document Tracking: Unifying Your Physical and Electronic Desktops

Video-Based Document Tracking: Unifying Your Physical and Electronic Desktops. Jiwon Kim Steven M. Seitz Maneesh Agrawala University of Washington Microsoft Research. Motivation. Unifying physical and electronic desktops. Video camera. Record video of paper on physical desktop. Desktop.

Mia_John
Download Presentation

Video-Based Document Tracking: Unifying Your Physical and Electronic Desktops

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video-Based Document Tracking:Unifying Your Physical and Electronic Desktops Jiwon Kim Steven M. Seitz Maneesh Agrawala University of Washington Microsoft Research

  2. Motivation

  3. Unifying physical andelectronic desktops Video camera Record video of paper on physical desktop Desktop

  4. Unifying physical andelectronic desktops Video camera Record video of paper on physical desktop • Tracking Desktop

  5. Unifying physical andelectronic desktops Video camera Record video of paper on physical desktop • Tracking • Recognition Desktop

  6. Unifying physical andelectronic desktops Video camera Record video of paper on physical desktop • Tracking • Recognition • Linking Desktop

  7. Applications Video camera Desktop

  8. Applications Video camera • Find lost document Desktop

  9. Applications Video camera • Find lost document • Browse remote desk Desktop

  10. Applications Video camera • Find lost document • Browse remote desk • Find electronic version Desktop

  11. Applications Video camera • Find lost document • Browse remote desk • Find electronic version • History-based queries Desktop

  12. Example Input Video

  13. Demo – Remote Desktop

  14. Related Work • Interactive desktops DigitalDesk [Wellner 93]

  15. Related Work • Interactive desktops Self-Organizing Desk [Rus et al. 97] DigitalDesk [Wellner 93]

  16. Related Work • Interactive desktops • Augmentedpaper PADD [Guimbretière 03]

  17. Related Work • Interactive desktops • Augmentedpaper CyberCode [Rekimoto et al. 00] PADD [Guimbretière 03]

  18. Related Work • Interactive desktops • Alternative media • Object tracking & recognition SIFT [Lowe 04]

  19. System Overview Video camera Computer User Desk

  20. System Overview Video of desk

  21. System Overview Images from PDF Video of desk

  22. System Overview Images from PDF Video of desk Track & recognize

  23. Desk Desk System Overview Internal representation Images from PDF Video of desk Track & recognize T T+1

  24. Desk Desk System Overview Internal representation Images from PDF Video of desk Track & recognize T T+1 Scene Graph

  25. Desk Desk System Overview Where is my W-2? Internal representation Images from PDF Video of desk Track & recognize T T+1

  26. System Overview Where is my W-2? Answer Internal representation Images from PDF Video of desk Track & recognize Desk Desk T T+1

  27. Desk Desk System Overview Where is my W-2? Internal representation Images from PDF Video of desk Track & recognize T T+1

  28. Tracking & Recognition … …

  29. … … Tracking & Recognition Event

  30. Event Types before after Move

  31. Event Types before after Move Entry

  32. Event Types before after Move Entry Exit

  33. … … Tracking & Recognition Event … Desk

  34. … … Tracking & Recognition Event … … Desk Desk

  35. … … Tracking & Recognition Event sanders01.pdf lowe04sift.pdf tut-article.pdf … … objectspaces.pdf kidd94.pdf Desk Desk

  36. Assumptions • Document • Corresponding electronic copy exists • No duplicates of same document

  37. Assumptions • Document • Corresponding electronic copy exists • No duplicates of same document • Motion • 3 event types: move/entry/exit • One document at a time • Only topmost document can move

  38. Non-Assumptions • Desk need not be initially empty

  39. Non-Assumptions • Desk need not be initially empty • Stacks may overlap

  40. Algorithm Overview Input Frames … …

  41. Algorithm Overview Input Frames … … Event Detection before after

  42. Algorithm Overview Input Frames … … Event Detection before after Event Interpretation “A document moved from (x1,y1) to (x2,y2)”

  43. Algorithm Overview Input Frames … … Event Detection before after Event Interpretation “A document moved from (x1,y1) to (x2,y2)” File1.pdf Document Recognition File2.pdf File3.pdf

  44. Algorithm Overview Input Frames … … Event Detection before after Event Interpretation “A document moved from (x1,y1) to (x2,y2)” File1.pdf Document Recognition File2.pdf File3.pdf Scene Graph Update Desk Desk

  45. Algorithm Overview Input Frames … … Event Detection before after Event Interpretation “A document moved from (x1,y1) to (x2,y2)” File1.pdf Document Recognition File2.pdf File3.pdf Scene Graph Update Desk Desk

  46. … … Event Detection

  47. Event Detection … … Frame differences time

  48. Event Detection … … Frame differences time

  49. Event Detection … … Frame differences time

  50. Event Detection … … Frame differences time

More Related