SensorBlocks: The Wood-and-Nails of the Electronic Sensor World

1. SensorBlocks: The Wood-and-Nails of the Electronic Sensor World Frank Vahid* Department of Computer Science and Engineering University of California, Riverside vahid@cs.ucr.edu http://www.cs.ucr.edu/~vahid * Also with the Center for Embedded Computer Systems at UC Irvine This work is being supported by the National Science Foundation SensorBlocks are also known as "eBlocks" – http://www.cs.ucr.edu/eblocks

2. Seen this Problem? • Technology everywhere – Why no good solution? Available Not!

3. transmit AND LED receive light sensor contact switch Introduction • 1998: Simple Problem • Garage door open at night • No simple solution • Off-the-shelf solutions costly, hard to find, and/or not customizable • Alarm system cost – overkill • Connecting existing sensors, logic, transmit/receive, LEDs is hard • Electronics, programming • 2-week project: 70% EE/CS unable • Countless similar applications go unrealized • Why can't I just connect those components like "Lego" blocks?

4. Shrinking Processor Size/Cost Enables New Solution • Make sensors smarter • By adding processor+battery • Becomes a "block" easily connected to other blocks Courtesy of Joe Kahn http://www.templehealth.org

5. yes/no yes/no Magnetic Contact Switch yes/no LED yes/no Light Sensor Beeper yes/no Button yes/no Electric Relay Shrinking Processor Size/Cost Enables New Solution – SensorBlocks Existing component view New "SensorBlock" view

6. yes/no yes/no yes/no Light Sensor Beeper Button SensorBlocks • Just connect blocks, and they work • No programming knowledge, no electronics knowledge LED yes/no

7. Magnetic Contact Switch When A is yes no AND OR B is yes no LED then the output is yes Combine Light Sensor SensorBlocks Example • "Garage Open at Night" detector • <10 minutes to build Need to indicate garage open at night – use LED block Detect night-time – use Light Sensor block Use Combine block to combine light sensor and contact switch into one Detect garage door open – use Contact Switch block Plug pieces together and the system is done!

8. Too few – Overwhelming (too much configuration) Too many – Overwhelming (too much choice) When A is yes no AND OR B is yes no Splitter then the output is yes Combine Toggle Splitter 1 1 2 3 4 5 6 7 8 9 2 No detector 2 Yes detector 2-Input Logic 3-Input Logic 4-Input Logic When A is yes no AND OR B is 2 Tripper 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 then the output is yes Prolong (short) Prolong (long) 4 3 5: Splitter 6: ... Splitter 1 2 3 4 5 6 7 8 9 SuperBlock What's Hard (The Research Part) • (1) Finding right set of building blocks

9. yes no A B B A The output should be yes when: yes no: A is yes, B is yes A B Output A is yes, B is yes When the input is the output should be A is yes, B is no A is yes, B is no Combine A is no, B is yes A is no, B is yes A is no, B is no A is no, B is no out Combine Phrased truth table Phrased truth table embedded in sentence configurable DIP switch Logic Block yes no yes no When A is AND OR B is then the output is yes Combine Logic Sentence yes no A B A B When the input is The output should be A B A B A B out Combine Colored truth table embedded in sentence no no no yes no yes no yes yes no no yes yes yes no yes What's Hard (The Research Part) • (2) Making the blocks understandable • People NOT likely to read directions • Those that do are unlikely to understand Performed extensive user testing (over 500 students, kids, and adults) over two years Example: Combine block Most success

10. (a) time Connect Responsiveness Disconnect Responsiveness f t f Reliability Latency (b) 1 1 1 1 interpreted as Freliability Fresponsiveness Fresponsiveness Flatency < < 0 0 0 0 (c) 0.25 4 30 0.50 10 2 5 600 1800 1 3 60 300 365 1 0 730 0.50 1 4 5 10 300 600 0.14 0.10 0.25 2 3 30 60 0.05 error error Mean time between corrupted packets (days) Connect response (seconds) Disconnect response (seconds) Block latency (seconds) < < (d) f f t f f What's Hard (The Research Part) • (3) Batteries must last years, yet performance should appear continuous • Blocks are off 99.9% of the time Developed theory to map SensorBlock events to continuous time Developed custom CAD tool to automatically find the best block parameter settings out of the billions of possibilities

11. Built >100 Prototypes • Size of deck of cards (eventually smaller) • 2-3 years on 2 AA batteries (eventually longer) • Can communicate via wire >1.5 miles, 150 ft wireless • Hundreds of trial users

12. Output Output Output Compute/Communications Compute/Communications Compute/Communications Available eBlocks Available eBlocks Beeper Beeper Beeper When A is When A is When A is yes no yes no yes no AND OR AND OR AND OR B is B is B is yes no yes no yes no then the output is yes then the output is yes then the output is yes Combine Combine Combine Green/Red Light Green/Red Light Green/Red Light rst rst rst in in in Once Yes, Stays Yes Once Yes, Stays Yes Once Yes, Stays Yes Available eBlocks Toggle Toggle Toggle 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Sensors Sensors Sensors seconds seconds seconds Prolonger Prolonger Prolonger Button Button Button Hide this panel Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Motion Sensor Hide this panel Advanced Mode Light Sensor Light Sensor Light Sensor Yes/No Yes/No Yes/No Hide this panel Advanced Mode Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets

13. Output Output Compute/Communications Compute/Communications Available eBlocks Available eBlocks Beeper Beeper When A is When A is yes no yes no AND OR AND OR B is B is yes no yes no then the output is yes then the output is yes Combine Combine Green/Red Light Green/Red Light Button rst rst in in Once Yes, Stays Yes Once Yes, Stays Yes Toggle Toggle 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Sensors Sensors seconds seconds Prolonger Prolonger Button Button Hide this panel Advanced Mode Hide this panel Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Light Sensor Light Sensor Yes/No Yes/No Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging

14. Output Compute/Communications Available eBlocks Beeper When A is yes no AND OR B is yes no then the output is yes Combine Green/Red Light Button rst in Once Yes, Stays Yes Toggle 1 2 3 4 5 6 7 8 9 Sensors Light Sensor seconds Prolonger Button Hide this panel Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Light Sensor Yes/No Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging

15. Output Output Compute/Communications Compute/Communications Available eBlocks Beeper Beeper When A is When A is yes no yes no AND OR AND OR B is B is yes no yes no then the output is yes then the output is yes Combine Combine Green/Red Light Green/Red Light Button rst rst in in Once Yes, Stays Yes Once Yes, Stays Yes Available eBlocks Toggle Toggle When A is yes no AND OR B is yes no 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 then the output is yes Sensors Sensors Light Sensor seconds seconds Combine Prolonger Prolonger Button Button Hide this panel Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Light Sensor Light Sensor Yes/No Yes/No Hide this panel Advanced Mode Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging

16. Output Output Output Compute/Communications Compute/Communications Compute/Communications Available eBlocks Available eBlocks Beeper Beeper When A is When A is When A is yes no yes no yes no AND OR AND OR AND OR B is B is B is yes no yes no yes no Beeper then the output is yes then the output is yes then the output is yes Combine Combine Combine Green/Red Light Green/Red Light Beeper Button rst rst rst When A is yes no AND OR B is yes no Green/Red Light in in in Once Yes, Stays Yes Once Yes, Stays Yes Once Yes, Stays Yes Available eBlocks then the output is yes Combine Toggle Toggle Toggle 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Sensors Sensors Sensors Light Sensor seconds seconds seconds Prolonger Prolonger Prolonger Button Button Button Hide this panel Hide this panel Advanced Mode Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Motion Sensor Light Sensor Light Sensor Light Sensor Yes/No Yes/No Yes/No Hide this panel Advanced Mode Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging

17. Output Output Output Compute/Communications Compute/Communications Compute/Communications Available eBlocks Available eBlocks Beeper Beeper When A is When A is When A is yes no yes no yes no AND OR AND OR AND OR B is B is B is yes no yes no yes no Beeper then the output is yes then the output is yes then the output is yes Combine Combine Combine Green/Red Light Green/Red Light Beeper Button rst rst rst When A is yes no AND OR B is yes no Green/Red Light in in in Once Yes, Stays Yes Once Yes, Stays Yes Once Yes, Stays Yes Available eBlocks then the output is yes Combine Toggle Toggle Toggle 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Sensors Sensors Sensors Light Sensor seconds seconds seconds Prolonger Prolonger Prolonger Button Button Button Hide this panel Hide this panel Advanced Mode Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Motion Sensor Light Sensor Light Sensor Light Sensor Yes/No Yes/No Yes/No Hide this panel Advanced Mode Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging • User is able to configure various blocks by clicking on switches

18. Output Output Output Compute/Communications Compute/Communications Compute/Communications Available eBlocks Available eBlocks Beeper Beeper When A is When A is When A is yes no yes no yes no AND OR AND OR AND OR B is B is B is yes no yes no yes no Beeper then the output is yes then the output is yes then the output is yes Combine Combine Combine Green/Red Light Green/Red Light Beeper Button rst rst rst When A is yes no AND OR B is yes no Green/Red Light in in in Once Yes, Stays Yes Once Yes, Stays Yes Once Yes, Stays Yes Available eBlocks then the output is yes Combine Toggle Toggle Toggle 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Sensors Sensors Sensors Light Sensor seconds seconds seconds Prolonger Prolonger Prolonger Button Button Button Hide this panel Hide this panel Advanced Mode Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Motion Sensor Light Sensor Light Sensor Light Sensor Yes/No Yes/No Yes/No Hide this panel Advanced Mode Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging • User is able to configure various blocks by clicking on switches • Connections created by drawing lines between blocks

19. Output Output Output Compute/Communications Compute/Communications Compute/Communications Available eBlocks Available eBlocks Beeper Beeper When A is When A is When A is yes no yes no yes no AND OR AND OR AND OR B is B is B is yes no yes no yes no Beeper then the output is yes then the output is yes then the output is yes Combine Combine Combine Green/Red Light Green/Red Light Beeper rst rst rst When A is yes no AND OR B is yes no Green/Red Light in in in Once Yes, Stays Yes Once Yes, Stays Yes Once Yes, Stays Yes Available eBlocks then the output is yes Light Sensor Combine Toggle Toggle Toggle 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Sensors Sensors Sensors Light Sensor seconds seconds seconds Prolonger Prolonger Prolonger Button Button Button Hide this panel Hide this panel Advanced Mode Advanced Mode Welcome to the eBlocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an eBlock off of the “Available eBlocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Motion Sensor Motion Sensor Motion Sensor Light Sensor Light Sensor Light Sensor Yes/No Yes/No Yes/No Hide this panel Advanced Mode Graphical Simulator • User specifies and tests block design • Java-based simulator • User chooses between pallets • Blocks added by dragging • User is able to configure various blocks by clicking on switches • Connections created by drawing lines between blocks • User can create, experiment, test and configure design Button

20. Partitions Programmable SensorBlocks SensorBlocks as a Programming Paradigm • Use virtual blocks in graphical simulator to describe desired sensor system behavior • Intuitive due to spatial emphasis, not temporal emphasis • Automatically compile to code on programmable SensorBlocks

21. Programmable SensorBlock #include <pic.h>#include “sci.h” #include “io.h” #include “constants.h” Unsigned char data_val = ERROR; ... main(void) { unsigned I, j; TRISB = 0; ... } main(void) { ORTA = 0xff; CMCON = 0x07; TRISA = 0x00; TRISB = 0x02; asm("CLRWDT"); ... } SensorBlock Tool Generates Code • Tool generates C code automatically C Code • Download code to block with click of a button • Ordinary users can “write” programs in minutes • Spatial vs. temporally-oriented language • 20 high school graduates: eBlocks (spatial) vs. LEGO Mindstorms (temporal), 6 example systems, 40 minutes to build

22. Conclusions • SensorBlock technology nearly mature • Possible early-adoption applications • Hearing impaired home monitoring • Aging parent non-intrusive monitoring • Middle-school early engineering experience kits • Trends work in favor each year • Smaller, cheaper (<$5), longer battery life (or no battery?) • Eventual applications • General blocks for home, stores, office • Blocks as front-end devices to "smart home" • New blocks and tools for intermediate/advanced users • Collaborators • Ph.D students: Susan Lysecky, Ryan Mannion, David Sheldon; Profs. Harry Hsieh, Walid Najjar, Crista Lopes (UC Irvine); UG students: Andrea Coba, Margaret Ukwu, Caleb Leak, and several others