1 / 20

Panlingual Mobile Camera: Break Language Barriers Instantly

This project presents a camera phone solution to overcome language barriers, illiteracy, and vision disabilities by allowing users to photograph text and receive fast explanations. Contextual inquiry and ESM were conducted to gather insights on user behavior and environment to optimize user experience. Three representative tasks were identified to demonstrate the device's utility in various scenarios. The design aims for a fast, efficient, and user-friendly solution supporting audio and text translations. Enhanced with UI design ideas, the camera enables on-the-fly translations and understanding, promoting inclusivity and accessibility.

valentine-burris
Download Presentation

Panlingual Mobile Camera: Break Language Barriers Instantly

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Panlingual Mobile Camera CSE 490f Jonathan Pool, Neb Tadesse, Tim Wong, Luke Woods

  2. Overview • Overall problem & solution • Contextual inquiry description & results • ESM description & results • Representative tasks • Task Scenarios • Summary • Q & A

  3. The Overall Problem • Have you ever been in a situation where you can’t understand some text? • Language barriers • Illiteracy • Vision disability

  4. A Solution • A camera phone that: • Allows the customer to photograph text • Efficiently receives a fast explanation of the text

  5. Contextual Inquiry Description • Three subjects in different situations: • (1) subject wearing blurring goggles to hinder vision • (2) subject amidst many non-English signs + labels • (3) subject with low English fluency • In various contexts: locations + activities

  6. Contextual Inquiry Description • Goals: • (1) subjects’ purpose • (2) kinds of text subjects wanted to understand • (3) urgency of understanding • (4) alternative sources of understanding • (5) means of coping without understanding • (6) subjects’ text-selection strategies

  7. Contextual Inquiry Results

  8. Contextual Inquiry Results

  9. ESM Description • Goals: • Determine how well users can take photographs • Clear or blurry? • Size vs distance? • Reveal how users might indicate the specific text to be understood • Determine what sort of environments people are encountering throughout the day

  10. ESM Description • Design: • Prompt approx every hour between 8am and 9:30pm • Asked to take one or more photographs of text • Encouraged to take diverse photos • Subjects were also allowed to take any additional photographs at any time • Subjects could record any audio-visual comments

  11. ESM Results • Not everyone takes good photos

  12. ESM Results • Environment doesn’t always allow for good photos

  13. Some photos contained both legible and illegible text Some users pointed towards text ESM Results • Cannot rely on the user to indicate area

  14. Representative Tasks • (Easy) Customer looks at a road sign in a foreign language and tries to infer whether it defines a speed limit. • (Moderate) The customer wanders around a foreign bookstore looking for the philosophy section. • (Hard) The customer approaches an apartment building and wants to know whether a person is listed in the building's directory, which is written in a script that the customer doesn't know.

  15. (1) A traveler in Germany rents a car. He comes across this sign and wants to know if he can park here legally.He wants to take a photo and annotate it, describing the info he wants to understand. Task Scenarios

  16. (2) A person travels to China, and wants to go to a Chinese restaurant. However, the person does not understand the menu in Chinese.She takes a photo. She wants to circle several random items of her choosing for translation. Task Scenarios

  17. (3) A person understands spoken Japanese, however he cannot read it.He wishes to hear an audio clip of this road sign.He takes a photo and requests to receive an audio translation. Task Scenarios

  18. Task Scenario • UI Design ideas:

  19. Summary • A device that can allow you to translate and/or understand text on the fly • Supports multiple delivery mediums: • Audio • Text • Must be fast, efficient, and easy to use.

  20. Q & A • Questions????

More Related