BroBot The Study Companion Group 25

1. Richard Landau Sarah Patten Jacob Stewart BroBotThe Study CompanionGroup 25

2. BroBot assists in easing the minds of students longing for a worry-free break. • Watches study materials while the user is away from the table. • Sends updated images and alerts to user through an Android app on user’s phone. • Will provide real-time updates and will notify user if something is stolen. Project Overview

3. Detect theft of an object. • Alert user when a theft occurs. • Be reliable and simple to use. • Replaceable or rechargeable power source • Be lightweight and affordable. Goals and Objectives

4. Design Specifications

5. Overall Block Diagram

6. Cameras

7. 5 volts needed • Data is serial data • Support for fast baud rate • Sends 8-bit greyscale RAW Images • Need to use RAWs to analyze pixel values uCAM-II

8. All of the computations are on the microcontroller side • A good amount of serial communication protocols (SPI, I2C, UART) • Low power modes including sleep mode • Good community backing • Cheap development board with all the pin outs on the microcontroller Processor Considerations

9. ARM Microcontrollers

10. 3.3V Voltage • Low power modes • Digital Camera Interface(DCMI) • Good amount of SRAM • Fast operation speed • A lot of serial communication lines • Development board comes with ST-Link capabilities • Good community backing it with pre-written libraries for most features STM32F407VG

11. Wi-Fi direct and Bluetooth are comparable • Wi-Fi direct has longer range than Bluetooth • However, Bluetooth has more support and is easier to implement • Bluetooth is also more common in everyday life so we wanted experience with it Bluetooth vs WiFi-Direct

12. Bluetooth Module Comparison • Chose RN41 due to low cost, high range, and black box functionality • WT41 strong contender but expensive and drains battery quickly

13. Alarm • Uses STM32’s on board DAC to create a sine wave • Speed of output of DAC is varied to create ascending and descending tones • TPA2005D1 – Class-D Amplifier • 3.3V • Low power consumption via Pulse Width Modulation • 8-ohm speaker inside BroBot

14. All components need 3.3-5V • Around 750mA at max • Power source needs to be small and lightweight • Rechargeable or easily replaceable power source • Two UA78 linear regulators, one for 3.3V and one for 5V • 4 AA Batteries held in a battery holder Power System Overview

15. Camera Communication • Camera MUST Sync before further communications • After 15 seconds of no communication, a Sync call must be given to wake up the camera module • To generate an 8-bit grayscale raw picture file: • Initial request – Sets resolution and format • Receive data – Starts data stream • Final ACK – Finish transmission

16. Camera Communication

17. Project box containing PCB and batteries • Bluetooth accessible • Sturdy and adjustable camera arm • Removable cover BroBot’s Platform

18. Use computer vision techniques. • Compare 2 images from different points in time • Determine if there has been a significant amount of change Item watching overview ≠

19. Store an “original” image • Each cycle, take a new picture • Compare new picture with “original” • If the images differ by too much, sound alarm • Every 20 pictures, replace “original” with the newest picture taken Algorithm explanation

20. Item Watching Flowchart

21. Android application’s main focus is to alert the user if something has been stolen and to act as the user interface • Picture of BroBot’s current view will be refreshed for the user to view at any time • App will vibrate to alert the user if something is taken • Image causing the alert will be sent App Summary

22. Android represents a bigger market share • Does not require a Mac to code • Good APIs to interface with Bluetooth Android vs iOS VS

23. App Design • Initial screen shown upon startup • Click Connect, and select the device to connect to • If connection fails, user will see an error • If it succeeds…

24. User will be taken to screen showing BroBot’s field of vision • When his sight is aligned properly, press the button to start the item watching software • This button will be replaced with a “Stop Watching” button, or a “Silence Alarm” button if it is tripped

25. Bluetooth connection • Once Bluetooth connection is established, app opens an input thread and output thread • Output thread stays idle and waits for signals to be sent to BroBot • Input thread constantly looks for new commands from BroBot • Each command received is handled accordingly by the app and then acknowledged by a return command • Bluetooth’s input stream buffer reads in an array of bytes • When the app sends a command to request a picture (every 5 seconds), BroBot responds with a command, followed by the picture size and then the picture data • Combine these data bytes and restore the original picture

26. Testing • Each subsystem was tested as soon as it was possible to do so • Bluetooth module was tested from mobile terminal app to test communication • When app and camera were complete, picture testing between the camera, microcontroller, and app was done • Item watching code was tested on the computer before being extended to the embedded environment • After being entirely integrated, general tests were performed

27. Testing Data No Change Average change = 12,000 Small Change: Average change = 32,140 Medium Change: Average change = 81,486 Large Change: Average change = 142,987

28. Testing Data • Used a counter to time the number of comparisons done in 30 seconds • After 5 consistent tests, it was determined as 26/min • Slowly paced away from BroBot to determine range • After a few tests in multiple locations, it was determined as 60ft

29. Budgeting

30. Completion of Goals