Helping Hand

1. Pedro Rivera Torres AmmarAlmani SrrahAlgheithy Thad Guriel Andy Duann Hao Chen Helping Hand

2. CDR Overview • PDR Revision • Functional decomposition (hardware) • Hardware subcomponents/implementation • Functional decomposition (software) • Software subcomponents/implementation • Current project status • Goals (milestone 1, 2, expo) • Technicalities (schedule, budget) Thad Guriel

3. Technical Specialist • Sam Siewart • Google IO interface • Jim Pilkington • Assistive Technology Specialist • Colorado Department of Human Services • Division of Vocational Rehabilitation • Catherine Kilcommons • Assistive technology lab coordinator • Disability services - University of Colorado Boulder Thad Guriel

4. Audio Output CPU RF Comm IR Navigation RF Receiver User Info Android Maps Info IR Receiver Keypad PDR Revision User Thad Guriel

5. Not Our Project… Thad Guriel

6. Our Project… Thad Guriel

7. Functional Decomposition Hardware Thad Guriel

8. Entrance Wake Sequence Query signal sent from device Xbee No Is there a signal? Xbee checks for signal Yes Xbee establishes communication with device Xbee sends interrupt signal to processor Andy Duann

9. After interrupt has been sent to processor Xbees establish communication Processor wakes up Processor kicks out sign data Processor turns on IR Talk to device XBee Start clock Send data Turn off IR Xbee cyclic sleep Processor goes to sleep Andy Duann

10. Door Transmitter Andy Duann

11. XBee Device Cyclic sleep Cycle wake up Interrupt from processor Was there a signal? Xbee establishes communication Send signal to wake door XBee No Yes Transmit data Andy Duann

12. Audio Circuit Andy Duann

13. Device Audio: No signal No audio signal uP outputs logic low to switches Turns off synthesizer switch Turns on Android switch Andy Duann

14. Audio Circuit Andy Duann

15. Phone Audio Phone audio Comparator outputs logic high uP turns off synthesizer interrupts Outputs logic low to switches Comparator goes low when phone audio gone Andy Duann

16. Audio Circuit Andy Duann

17. Synthesizer Audio Synthesizer audio Turns android off Turns synthesizer on Synthesizer sends interrupt request when data present uP sets logic high to switches uP checks android interrupt Is it low? uP sends interrupt enable to synthesizer No Yes Andy Duann

18. Audio Circuit Andy Duann

19. Overall Device Schematic Andy Duann

20. Audio Circuit Andy Duann

21. Keypad Andy Duann

22. GPS Transition • Locate building entrance after hand-off from GPS • Read signs from a distance and determine its relative direction Hao Chen

23. Example Hao Chen

24. Sign Transmitter • 3 infrared LED’s are mounted on the entrance, to send directional information at a wide angle • The information of the building is transmitted with XBee Hao Chen

25. Sign Receiver • 5 IR receivers are mounted on the device • IR receivers are sensitive to 950 nm wavelengths and 38 kHz carrier only • The direction of the entrance can be determined based on which IR receiver(s) received the signal Hao Chen

26. Device Receiver Example • In the above picture, receiver 4 picks up the signal while the rest don’t, therefore the microprocessor determines the sign is at northeast direction 3 2 4 Reader 1 5 Hao Chen

27. Power Saving • Motivation: battery runs out quickly if IR LED’s are on all the time • Solution: only turn transmitter on when user close • Device uses XBee to trigger transmitter • The transmitter turns off after set amount of time Hao Chen

28. Building Information Reading Hao Chen

29. Software Engineering SrrahAlgheithy

30. Functional Decomposition Software SrrahAlgheithy

31. Software Components Android app to access GPS and Compass USB interface Android phone 2.3.4 gingerbread Android ADK SrrahAlgheithy

32. Android App SrrahAlgheithy

33. Android App SrrahAlgheithy

34. Android App Code • requestLocationUpdates() • Location provider • Time between updates • Distance between updates • LocationListener() • ItemizedOverlay<Overlay Item> • OverlayItem() • Specify and name points of interest • Use built in compass for direction SrrahAlgheithy

35. Data Handling System Overview • Communicate effectively with the keypad system. • Coordinate Keypad output with Android apps. • Control internal functionalities of the Android phone • Manage operation and information coming form Xbee • Coordinate Xbee info to turn on IR sensors. • Manage the audio system. AmmarAlmani

36. Development • Using PIC24F j256 GB110 • Features: • 16-bit: similar to DSP • Still low power (3.3V) • Programming Memory: 256 KB • Full Speed USB operation in Device • USB on-The-Go • 3 ADCs: 500K conversion / second. AmmarAlmani

37. Microcontroller Functionality Primary ---- Secondary ---- AmmarAlmani

38. Keypad • Coordination Module • Using the keypad encoder to generate interrupts when a button is pressed. AmmarAlmani

39. Android/GPS • Input/output module AmmarAlmani

40. Sensors • Composite module AmmarAlmani

41. Xbee communication • Input Module Pedro Rivera

42. Audio • A system that outputs audio signals from both Android device and text-to-speech chip Pedro Rivera

43. Temporal Diagram Primary ---- Secondary ---- Pedro Rivera

44. PIC USB – Android interface • Requirements for Open Accessory Hardware • Making PIC the host • Being host entails: -Initiating communication -Knowing what all devices are doing Pedro Rivera

45. PIC USB – Android interface • Open accessory Handshake (When accessory is connected, the accessory will perform these steps): • Send “Get Protocol” command • Send device info to identify the accessory to the Android Device • Send “Start” Command • The Android device should re-enumerate in accessory mode and launch an app Pedro Rivera

46. PIC USB – Android interface • USB Accessory Handshake, part 2 • Accessory mode if: • Vendor ID is 0x18D1 (Google) • Product ID is 0x2D00 or 0x2D01 • Read configuration descriptors • Look for first bulk IN and first bulk OUT endpoints • Set configuration to 1 • Endpoints are now ready for communication Pedro Rivera

47. Software Pedro Rivera

48. Current Status Thad Guriel

49. Milestone I Thad Guriel

50. Milestone II Thad Guriel