1 / 20

Somani Patnaik 1 , Emma Brunskill 1 , William Thies 2

Accuracy of Data Collection on Mobile Phones: A Study of Forms, SMS, and Voice. Somani Patnaik 1 , Emma Brunskill 1 , William Thies 2. Massachusetts Institute of Technology. 1. Microsoft Research India. 2. Mobile Data Collection . • Broad applications in . –.

koto
Download Presentation

Somani Patnaik 1 , Emma Brunskill 1 , William Thies 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Accuracy of Data Collection on Mobile Phones: A Study of Forms, SMS, and Voice Somani Patnaik1, Emma Brunskill1, William Thies2 Massachusetts Institute of Technology 1 Microsoft Research India 2

  2. Mobile Data Collection • Broad applications in – Mobile banking – Microfinance – Healthcare – Environmental monitoring • •Benefits • Immediate digitization • Fast and cheap • Environment friendly • Flexible questionnaires

  3. Study of data collection interfaces • •Comparison of three interfaces for health data collection

  4. 1. Electronic Forms Interface

  5. 1. Electronic Forms Interface • • Consist of numeric fields and multiple-choice menus. • Can be implemented in Java or a native phone platform. • • General Strengths • Easy patient identification • Ongoing cost is low (SMS or data plan) • Can store visits when connectivity is poor • • General Weaknesses • Requires programmable phones • Requires basic literacy skills • Hard to alter survey questions • Hard to enter in free-form notes • Application can be deleted by user

  6. 2. SMS Interface

  7. 2. SMS Interface • • Sending a structured SMS messages to a server • Logical fields separated by delimiters in the message • • General Strengths • Can be used with any phone • Ongoing cost is low (SMS or data plan) • Many workers familiar with SMS • • General Weaknesses • Requires basic literacy skills • Changing survey requires new cue card • Quite easy to fake visits (copy old SMS) • Hard to enter in free-form notes

  8. 3. Live Operator Interface

  9. 3. Live Operator Interface • • A normal telephone call • Live human operator that enters the data into a centralized • spreadsheet • • General Strengths • No literacy required of workers • Can be used with any phone • Hard to fake a visit: operator can ask new questions • • General Weaknesses • Ongoing cost of operator salary • Voice plans often higher cost than SMS • Awkward 3-way social interaction

  10. Operations • Setup time - Electronic forms require application, • which requires either an Internet-enabled phone or an • external computer • Training time – Voice interface requires least amount of • education and background • System coverage and reliability - voice most reliable but • require sufficient number of operators. • Flexibility - Ability to modify the data collection interface, • fix an error, improve usability etc

  11. Effectiveness • •Characterized by two factors- • Data should not be intentionally faked by the user • The data should be accurate(not intentionally faked) • Unfortunately, quite easy to fake data in SMS systems, just • requires copying and pasting prior messages • Faking data on electronic systems slightly harder and most • difficult in voice. • Voice also allows correcting previous visits

  12. Cost Comparison • • Electronic phone requires a programmable phone(such as a • Java enable phone or windows phone) • Voice has the ongoing cost of the operator • Cost of live-operator in India proves to be cost-effective • Decreased cost of voice-only handsets, training time and • literacy requirements of health workers compensate the cost

  13. Cost Analysis

  14. ..Cost Analysis

  15. Context: Rural Tuberculosis Treatment • With local partners, working to improve tuberculosis treatment in rural Bihar. • Monitoring the patients symptoms remotely by collecting data remotely.

  16. Study Participants • 13 health workers and hospital staff (Gujarat, India) Age Education Cell Phone Experience (Median) • • Training • Health workers: big groups, 6-8 hours • Hospital staff: small groups, 1-2 hours • Every user made two error-free reports on each interface

  17. Testing and Results • •Testing • Tested in pairs, one patient on data entry, and the other being • the fake patient • Two complete patient–worker interactions for electronic forms • and SMS interfaces • Results

  18. Results • • Error rate higher in health workers as compared to hospital staff • Two possible reasons: • Hospital staff more educated and old • Difference in training

  19. Sources of Error • Types of errors • Misplacement of decimal point in the temperature entry • Non-revision of cue cards for the SMS interface • Putting wrong patient identity when using SMS • Usability barriers • small keys • scrolling/selection • SMS encoding

  20. Conclusions • •Accuracy of mobile data collection demands attention • 5% error rates for those lacking experience • There exist cases where a live operator makes sense • Can be cost effective, esp. for short calls • Study has limitations • Small sample size • Varied education, phone experience, training of participants • Future work • Distinguish factors responsible for error rates • Compare to paper forms, Interactive Voice Response

More Related