LoCaF : Detecting Real-World States with Lousy Wireless Cameras

1. LoCaF:Detecting Real-World States with Lousy Wireless Cameras Benjamin Meyer, Richard Mietz, Kay Römer

2. Structure • Introduction • Motivation • Challenges • System Architecture • Evaluation

3. Motivation SFpark project: http://sfpark.org/ • Towards the Internet of Things • High-level state of things on the internet • Scalar/specialized sensors are often limited to one scenario • Cameras are more flexible

4. Low-cost hardware • Sensor nodes • Constrained resources • Low-cost cameras • Low resolution • Poor image quality • Low frame rate • Processing is shifted to the gateway

5. Scenarios Occupancy of a room Free seats in a room Individual occupancy of parking spots Picture Objects to detect People People Cars States Free/occupied Number of persons Free/occupied for each parking spot Challenges Possibly lots of movement Possibly lots of movement Outdoor  Changing lighting conditions Flexible Framework to infer and publish states for divers scenarios

6. HTML System Architecture: Overview RDF Tweet SQL Customizable workflow 0 • Image capture • Compression • Wireless transmission • Image processing • Regions of interest • Enhancing filters • Object detection • Face detection • Mobile object detection • State inference • Rule-based language • State publication • Text templates • Different media

7. HTML System Architecture: Sensor Node RDF Tweet SQL Customizable workflow 0 • Image capture • Compression • Wireless transmission • Image processing • Regions of interest • Enhancing filters • Object detection • Face detection • Mobile object detection • State inference • Rule-based language • State publication • Text templates • Different media • Camera equipped sensor node • Two capture modes • Time-triggered • Event-triggered (by PIR) • JPEG-compression in hardware • Fragmented transmission to gateway

8. HTML INSTITUTE OF COMPUTER ENGINEERING System Architecture: Processing RDF Tweet • Image processing • Regions of interest • Enhancing filters SQL Parking spot a Parking spot b Customizable workflow 0 • Image capture • Compression • Wireless transmission • Image processing • Regions of interest • Enhancing filters • Object detection • Face detection • Mobile object detection • State inference • Rule-based language • State publication • Text templates • Different media Region selection Lighting compensation Texture enhancement Contrast enhancement Orchestration and parameterization of enhancements

9. INSTITUTE OF COMPUTER ENGINEERING System Architecture: Processing • Object detection • Face detection • Mobile object detection • Image processing • Regions of interest • Enhancing filters • Object detection • Face detection • Mobile object detection • State inference • Rule-based language • Face detection • Adaptive background subtraction • Classification into fore- and background • Can adapt to small changes • Blob detection • Each blob is an object Number of & area covered by objects

10. HTML INSTITUTE OF COMPUTER ENGINEERING System Architecture: Processing RDF Tweet • State inference • Rule-based language SQL count:map:0:1:free count:map:1:-1:occupied State-based free count:map:0:1:All seats free count:map:10:45:Enough seats count:map:45:70:Almost full count:map:70:-1:No seats left Customizable workflow 80% coverage 80% coverage 0 • Image capture • Compression • Wireless transmission • Image processing • Regions of interest • Enhancing filters • Object detection • Face detection • Mobile object detection • State inference • Rule-based language • State publication • Text templates • Different media occupied area:switch:free:80:occupied area:switch:occupied:80:free area:map:0:80:free area:map:80:100:occupied count:map:0:1:free count:map:1:-1:occupied Event-based Rule-based state inference

11. HTML System Architecture: Publishing RDF Tweet SQL Customizable workflow 0 • Image capture • Compression • Wireless transmission • Image processing • Regions of interest • Enhancing filters • Object detection • Face detection • Mobile object detection • State inference • Rule-based language • State publication • Text templates • Different media • Every text format (HTML, RDF, TXT, …) • Template-based • Publishing via • FTP • Twitter • SQL-Database

12. Evaluation Setup • Camera in front of lecture hall during lecture • Estimate number of students • Also looking at binary state (free/occupied) • One region, background subtraction & no filter • Three phases: • Beginning: Entering persons in dribs and drabs • During: Not many movements • End: Abrupt leaving of students

13. Evaluation: Under- and Overestimation • Underestimation • Several persons identified as one • Persons not recognized because of no movement • Overestimation • Legs recognized as individual

14. Evaluation: Entry phase OE: 130% UE: 70% • Avg: 48% • Binary state always correct

15. Evaluation: Lecture phase UE: 105% • Avg: 54% • Binary state always correct

16. Evaluation: Exit phase UE: 222% OE: ∞ • Avg: 95% • Binary state not correct for picture 11-13

17. Evaluation: Entry phase revisited Image filters can significantly change the estimation

18. Evaluation: Entry phase revisited Parameters can significantly change the estimation Improved avg error: 12%

19. Conclusion Flexible framework Use of cameras to be applicable in divers scenarios Fully customizable by the user in each step Accuracy quite high

20. Questions? Thank you for your attention. Time for questions.

21. Setup Camera node Gateway Netbook with software

22. The Framework: Connection Configuration

23. The Framework: Data Exchange

24. The Framework: Image Processing

25. The Framework: Region Selection / State Inference

26. The Framework: Publishing

27. Filter

28. Evaluation: Parking Spot Scenario area:switch:free:80:occupied area:switch:occupied:80:free Select single spot State switches from free to occupied when car enters (b) and c)) State will switch back when car leaves