1 / 28

PhonePoint Pen: Using Mobile Phones to Write in Air

PhonePoint Pen: Using Mobile Phones to Write in Air. Sandip Agrawal, Ionut Constandache, Shravan Gaonkar, Romit Roy Choudhury. ACM MobiHeld 2009. Noting small pieces of information, quickly and effortlessly can be useful But aren’t there several solutions . Buy Milk. State of the Art.

jenny
Download Presentation

PhonePoint Pen: Using Mobile Phones to Write in Air

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PhonePoint Pen: Using Mobile Phones to Write in Air Sandip Agrawal, Ionut Constandache, Shravan Gaonkar, Romit Roy Choudhury ACM MobiHeld 2009

  2. Noting small pieces of information, quickly and effortlessly can be useful But aren’t there several solutions ... Buy Milk

  3. State of the Art • Sticky notes … • Difficult to organize electronically • Not always handy • Typing on phone keyboards • Tiny keyboard sizes • Small inter-key spacing • Audio recording • Cannot sketch diagrams • Time consuming to browse through voice messages

  4. So, need a solution that is • Easy to use • Always-with-me • Allows sketching • Online/Searchable

  5. Proposed Approach: PhonePoint Pen • User holds phone like a pen • Writes messages in air • Built-in accelerometer • Captures hand movement • Converts to image / text • Display text on screen • Also sent to user’s email address

  6. Use Cases • Mashing with cameras • Superimposing text on pictures Duke University • Sketching • Giving directions to your home • Exchanging figures on the fly • One handed use • E.g., carrying a suitcase

  7. Use Cases • Applications in healthcare • Hand movement disabilities • Speech problems after surgery • Military and emergency response • Quick and silent updates

  8. Prototype Current prototype on Nokia N95 Equipped with 3-axes accelerometer Programmed usign Python + MATLAB several design challenges emerge …

  9. Design Challenges (1) • Hands rotate while writing • Accelerometers only measure linear acceleration • Rotation injects ambiguity (Wii uses gyroscope) • Proposed Approach: • Hold phone in non-rotating grip • Determine angular orientation during the pause

  10. Design Challenges (2) • Background Vibration (Noise) • Sensitive accelerometers • Jitter from hand vibrations • Proposed Approach: • Smooth the accelerometer readings with moving average • Suppress acceleration values < threshold

  11. Reset Design Challenges (3) • Computing Displacement • Ambiguity when acceleration is zero • Can be constant velocity or stationary • Worse with noise • Proposed Approach: • Detect change of direction • Reset velocity between strokes

  12. Z-axis spikes Design Challenges (4) • Lifting Pen from the Paper • e.g., dotting the “i” and crossing the “t” • Proposed Approach: • Impulse on the Z axis during the lift • Off the Record – On the RecordP

  13. Design Challenges (5) • Character transition • e.g., B vs 13 • Proposed Approach: • Use delimited such as “.” • 1.3 for 13

  14. Drawing a Rectangle RawAccelerometer Reading

  15. Noise Smoothing using Moving Average Background Noise Suppression Final Rectangle Velocity Plot after Avoiding Velocity Drifts

  16. Raw Accelerometer Data for M and i

  17. Character Recognition

  18. More PhonePen words …

  19. More PhonePen words … 20 words tested with online OCR web service. 78% accuracy

  20. Results

  21. Video Demo

  22. Of Course, Not a Product Yet • Need smaller hand movements in air • Youtube comment: looking crazy waving hands • Slashdot: Nearby people can get injured • Lowercase character recognition • Cursive handwriting more complicated • Need increased robustness • 20% failure when writing in air • 5% failure when writing on a surface

  23. Of Course, Not a Product Yet • Erasing alphabets • Shake your hands like using a eraser • What about longer messages • Sentences? Email replies? Ongoing work is pursuing these directions …

  24. The vision is: Swipe your credit card and sign in air …

  25. Thanks • Visit Systems Networking Research Group (SyNRG) • @ Duke University • Google “synrg duke”

  26. Thank You! email: romit@ee.duke.edu Web: http://synrg.ee.duke.edu/

  27. Noting small pieces of information, quickly and effortlessly can be useful So, what are the options ... Call John @ 6658 Buy Milk Right on 3rd traffic light

  28. Velocity Plot after Avoiding Velocity Drifts Final Rectangle

More Related