1 / 27

Semantic Streams: a Framework for Composable Semantic Interpretation of Sensor Data

Semantic Streams: a Framework for Composable Semantic Interpretation of Sensor Data. Kamin Whitehouse UC Berkeley EWSN, Feb 13, 2006 Joint with Feng Zhao and Jie Liu, Microsoft Research. fact. fact. fact. fact. Data Processing. Data Processing. fact. Data Processing. Data

roscoe
Download Presentation

Semantic Streams: a Framework for Composable Semantic Interpretation of Sensor Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Semantic Streams: a Framework for Composable Semantic Interpretation of Sensor Data Kamin Whitehouse UC Berkeley EWSN, Feb 13, 2006 Joint with Feng Zhao and Jie Liu, Microsoft Research

  2. fact fact fact fact Data Processing Data Processing fact Data Processing Data Processing Data Processing Motivation • Imaging that sensors cover the globe • Can we automatically query for world facts? • No: each fact is an interpretation of data • Processing must consider • Sensor context • Sensor fusion • Data formats • Calibration • Signal transforms • Etc. World Sensor DB

  3. Fact: empty space Space Inference Vehicle Inference Size Inference Semantic Streams Fact: vehicle • Goal: to allow direct querying of facts • Example query: “Is a vehicle in the parking lot” • Instantiates a graph of inference units to derive the desired fact • Output is a stream of all vehicles detected Sensor.data=72 Sensor.data=56 Sensor.data=98 obj.size=23 obj.size=23 Sensor.data=49 obj.type=vehicle

  4. Usage Model • Fixed Sensor infrastructure • Many simultaneous, unrelated users • Short-term queries repeatedly over long periods • Queries are often similar (though not identical) • When programmer poses new query, system will use existing sensors and inference units, if possible. • Otherwise, system may give actionable error messages, • need to add 2 more sensors in area XYZ • need to add new inference unit to the system • Semantic values produced by infrastructure grows organically as it is used for new purposes

  5. Complete System Planning time Mobile Application Query Processor Inference Graph Query User Run time Execution Engine MSTML Embedding Engine Sensors Inference Unit Instances Output Stream

  6. george, jane, carA X, Y, Z, StudentA female(jane), parent(X,Y) parent(george,jane). male(george). father(X) :- parent(X,Y),male(X). male(X). > X = george father(X). > X = george father(jane). > false Constants Variables Predicates Facts Rules Queries Query processing Background: Prolog and FOL

  7. sensor(magnetometer, [[60,0,0][70,10,10]]). sensor(camera, [[40,0,0][55,15,15]]). sensor(breakBeam, [[10,0,0][12,10,2]]). inference( speedDetector, needs( sensor(magnetometer, R) ), creates( stream(X), isa(X,object), property(X,T,time), property(X,R,region), property(X,S,speed) ) ). Sensors are declared as logical facts All regions are appx’d by 3D cubes Inference units are logical rules Antecedents: the fact streams that it needs Consequents: the fact streams that it creates Spatial relationship encoded in variable name Markup Language

  8. subregion( A, B ) inference(speedDetector, needs( sensor(magnetometer, R) ), creates( stream(X), isa(X,object), property(X,T,time), property(X,R2,region), subregion(R2,R), property(X,S,speed) ) ). More sophisticated spatial relationships Subregions Sensor coverage (R) Queried area (R2) Stream Relationships

  9. inference( speedDetector, needs( sensor(breakBeam, R1), sensor(breakBeam, R2), sensor(breakBeam, R3), subregion(R1,R), subregion(R2,R), subregion(R3,R), \+ intersecttion(_,R1,R2), \+ intersecttion(_,R1,R3), \+ intersecttion(_,R2,R3) ), creates( stream(X), isa(X,object), property(X,T,time), property(X,R,region) ) ). More sophisticated spatial relationships Subregions Non-intersection Queried area Sensor coverage Stream Relationships

  10. inference( vehicleDetector, needs( sensor(magnetometer, R), stream(X), isa(X,object), property(X,S,speed) ), creates( stream(X), isa(X,vehicle) ) ). More sophisticated spatial relationships Subregions Non-intersection Stream identity Stream Relationships

  11. Stream(X), Isa(X,vehicle) Vehicle Inference stream(X), property(X,speed) Sensor(mag,…) stream(Y), property(Y,speed) Speed Inference Sensor1 Sensor2 Sensor3 Query processing isa(A,vehicle) Region(A,[[10…]]) • Choose first predicate in query • Search KB for sensors that match • Search output of inference units for streams that match • Add new input streams to query • Back to step 1

  12. Query processing isa(A,vehicle) Region(A,[[10…]]) • Actionable Error Messages • When missing inference unit • When missing sensor Stream(X), Isa(vehicle,X) Vehicle Inference stream(X), property(X,speed) Sensor(mag,…) stream(Y), property(Y,speed) Speed Inference Sensor1 Sensor2 Sensor3

  13. stream(Y), property(Y,speed) Speed Inference Camera Query processing isa(A,vehicle) Region(A,[[10…]]) • When too many choices • Multiple graphs provide logically equivalent results Stream(X), Isa(vehicle,X) Vehicle Inference stream(X), property(X,speed) Sensor(mag,…) stream(Y), property(Y,speed) Speed Inference Sensor1 Sensor2 Sensor3

  14. subregion( A, B ) inference(speedDetector, needs( sensor(camera, R) ), creates( stream(X), isa(X,object), property(X,T,time), property(X,R2,region), property(X,S,speed), property(X,C,confidence), {C > 99}, property(X,L,latency), {L < 450} ) ) . Inference unit may specify QoS of output Confidence Latency Quality of Service Constraints

  15. subregion( A, B ) inference(speedDetector, needs( sensor(breakBeam, R1) ), sensor(breakBeam, R2) ), sensor(breakBeam, R3) ), creates( stream(X), isa(X,object), property(X,T,time), property(X,R2,region), property(X,S,speed) ) ). property(X,C,confidence), {C > 80}, property(X,L,latency), {L < 75} ) ) . Inference unit may specify QoS of output Confidence Latency Quality of Service Constraints

  16. subregion( A, B ) inference(vehicleDetector, needs( sensor(magnetometer, R1), stream(X), property(X,S,speed), property(X,C2,confidence), property(X,L2,latency), creates( stream(X), isa(X,vehicle), {C > 0.9 * C2}, {L = L2 + 125} ) ). Inference unit may specify QoS of output Confidence Latency May be function of input QoS Quality of Service Constraints

  17. Query processing isa(A,vehicle) Region(A,[[10…]]) {C>70, L <300} • Query can declare QoS parameters to create ordering of inference graphs Stream(X), Isa(vehicle,X) Vehicle Inference stream(X), property(X,speed) Sensor(mag,…) stream(Y), property(Y,speed) stream(Y), property(Y,speed) Size Inference Size Inference Sensor1 Sensor2 Sensor3 Camera

  18. Example Infrastructure

  19. Goal Application: Vehicle Detection

  20. Query and Inference Graph isa(Y,histogram),property(Y,T,value),property(X,T,time), isa(X,vehicle),region(X,R,[[10,10,0][40,30,12]]), Isa(Z,photo),property(Z,X,triggerStream) Histogram Service Photo Service Vehicle Inference Size Inference Speed Inference Direction Inference Camera Break Beams Magnetometers

  21. Screenshot

  22. Final Application Break beam plot Camera image Magnetometer plot Micro Server Camera Server Report (histogram)

  23. Conclusion • Semantic Streams • Infers facts from data • Intended for sensor infrastructure • Can select inference graph based on QoS constraints • Limitations • Must be a large supply of inference units • Stream operators are difficult to program • Different programming language/execution model • Programmer must deal with temporal values, implement appropriate buffering, etc.

  24. Future Work: Streaming KB’s Relational Database : Knowledge Base :: Streaming Database : Streaming KB Logical rules: c(X) :- a(x),b(x). Operate on Facts SQL operators: Select, Join, Max Operate on Tables Infinite tables Infinite facts

  25. Video: Vehicle Detection

  26. Video: Human Filtering

  27. Traffic Histogram

More Related