1 / 19

PhonePoint Pen: Using Mobile Phones to Write in Air

PhonePoint Pen: Using Mobile Phones to Write in Air. Romit Roy Choudhury Assistant Professor. Noting small pieces of information, quickly and effortlessly can be useful So, what are the options. Buy Milk. State of the Art. Sticky notes … organization is a nightmare not handy

aiko-perez
Download Presentation

PhonePoint Pen: Using Mobile Phones to Write in Air

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PhonePoint Pen: Using Mobile Phones to Write in Air Romit Roy Choudhury Assistant Professor

  2. Noting small pieces of information, quickly and effortlessly can be useful So, what are the options ... Buy Milk

  3. State of the Art • Sticky notes … • organization is a nightmare • not handy • Typing on keyboard • tiny keyboard sizes • small inter-key spacing • Audio recording • cannot sketch information • time consuming to browse through voice messages

  4. So, need a solution that is • Easy to use • Always-with-me • Allows sketching • Searchable

  5. Proposed Approach: PhonePoint Pen • User can write messages in air • holding the phone like a pen • Use built-in accelerometer • capture hand movement • Decode text and image • sent to user’s email address for future use

  6. Use Cases • Mashing with cameras • superimposing text on pictures Duke University • Sketching • exchanging figures between caller/callee • One handed use • e.g., carrying a suitcase

  7. Prototype Current prototype implemented on Nokia N95 mobile phones several design challenges emerge …

  8. Design Challenges (1) • Lack of a Gyroscope • Accelerometers only measure linear acceleration • Linear Movements – Rotation Ambiguity • Proposed Approach: • Hold Phone in Non-rotating Grip • Determine Angular Orientation during the pause • P

  9. Design Challenges (2) • Background Vibration (Noise) • Sensitive Accelerometers • Significant Jitter by hand vibrations • Proposed Approach: • Smooth the accelerometer readings with moving average over last n (=7) readings • Suppress acceleration values < 0.5 • P

  10. Design Challenges (3) • Computing Displacement • Erroneous Acceleration Reading • Ambiguity when acceleration is zero • Proposed Approach: • Detect Pause Using Moving Window • Reset Velocity to Zero in Between Strokes • P

  11. Design Challenges (4) • Lifting Pen from the Paper • e.g., dotting the “i” and crossing the “t” • Proposed Approach: • Impulse on the Z axis during the lift • Off the Record – On the Record • P

  12. Drawing a Rectangle RawAccelerometer Reading

  13. Noise Smoothing using Moving Average Background Noise Suppression

  14. Velocity Plot after Avoiding Velocity Drifts Final Rectangle

  15. Raw Accelerometer Data for M and i

  16. More PhonePen words …

  17. Ongoing Work • Smaller hand movements in air • Lowercase character recognition • Increased robustness and reliability

  18. Our vision: Swipe your credit card and sign in air …

  19. Thank You! email: romit@ee.duke.edu Web: http://synrg.ee.duke.edu/

More Related