1 / 45

Pre-Workshop Discussion

Pre-Workshop Discussion. 2006-05-18 10:00 – 14:00 @102A1 Esaki Laboratory. Chapter. Introduction (22 pages) Overview Evaluation Characteristics Accident (3 pages) Accidents already occurred Accidents we must assume Discussion (18 pages). INTRODUCTION.

bruis
Download Presentation

Pre-Workshop Discussion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pre-WorkshopDiscussion 2006-05-18 10:00 – 14:00 @102A1 Esaki Laboratory

  2. Chapter • Introduction (22 pages) • Overview • Evaluation • Characteristics • Accident (3 pages) • Accidents already occurred • Accidents we must assume • Discussion (18 pages)

  3. INTRODUCTION • Historical System (weather.hongo) • Overview • Evaluation • Characteristics • New Approach (live-e.hongo) • Concept (Sensor Abstraction) • System Overview • Characteristics

  4. Live E! Project Overview • A Sensor Network Infrastructure • E stands for three meanings • Earth • Environment • Electronics • Application • Public Service (such as for Natural Disaster) • Business • Education

  5. INTRODUCTION • Historical System (weather.hongo) • Overview • Evaluation • Characteristics • New Approach (live-e.hongo) • Concept (Sensor Abstraction) • System Overview • Characteristics

  6. Historical System APP APP APP Web Interface Web Service(SOAP/XML) Windows 2003 Server MSSQL weather.hongo.wide.ad.jp WD Upload with SOAP WDDBLog Vaisala Sensors

  7. Measure Item Temperature Humidity Pressure (Barometer) RainFall WindDir WindSpeed (CO2) Vaisala Sensor(WXT510) WXT510 RS232C Serial Interface weather.hongo Internet iLon100 Lonworks SOAP WXT510s are connected to weather.hongo through Lonworks.

  8. Behavior of Data Upload Inside weather.hongo TABLE WD Current Data Table neuron_id WNIID Time Temperature Humidity Pressure RainFall WindDir WindSpeed CO2 Archive Data TABLE WDDBLog neuron_id Time Temperature Humidity Pressure RainFall WindDir WindSpeed CO2 DATA MODEL Each sensor has multiple sensor type Update WD Insert WDDBLog WeatherDataWrite Web Service Data Vaisala Sensor

  9. API (weather.hongo) Archived Sensor Data The Latest Sensor Data XML XML Web Service WeatherDataReadAll WDReadLog Select WD Select WDDBLog TABLE WD Current Data Table neuron_id WNIID Time Temperature Humidity Pressure RainFall WindDir WindSpeed CO2 Archive Data TABLE WDDBLog neuron_id Time Temperature Humidity Pressure RainFall WindDir WindSpeed CO2

  10. Evaluation (weather.hongo) • The Number of Sensor • About 50. • Upload Frequency • Temperature, Humidity, Pressure, RainFall • Every 5 minutes • WindDir, WindSpeed • Every 20 seconds • WDDBLog Table Size • Increases about 250,000 rows per day. • Current Row Size is 39,863,725 (2006-05-16 10:12:00) • A Query takes about 40 seconds.

  11. Characteristics (weather.hongo) • Designed by Echelon. • What the Sensor is: • Vaisala sensor (at least, sensor on Lonworks) • Each sensor has multiple sensor type. • Sensors can be identified by NeuronID. • No sensor meta-information • Sensor meta-information (sensor profile) must be provided by other systems. • Centralized • No scalability, No robustness

  12. INTRODUCTION • Historical System (weather.hongo) • Overview • Evaluation • Characteristics • New Approach (live-e.hongo) • Concept (Sensor Abstraction) • System Overview • Characteristics

  13. New Approach (live-e.hongo) • What the Sensor is: (Sensor Abstraction) • Sensor ID of Live E! • NeuronID is base on Lonworks system. • We must make a new ID system. • Globally-unique • A single sensor type. • Each sensor must have only one value. • A Multiple sensor must be divided to single sensors. • Profile • Meta-information that describes sensor.

  14. Sensor ID • sensor_id • Globally-unique ID • Format (Syntax Rule) • Domain Name + /SensorModel/Location/SensorType • Example • live-e.org/WXT510/0300000431b2/Temperature • live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Temperature • live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Humidity • live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Pressure • mew.co.jp/FS-Va-01/aoyama_elementary_school/Temperature • Semantics • No meaning • Just a String

  15. Sensor Data • Each sensor has a single value. • The value can be determined by “sensor_id” and sampled “time”. value = F( sensor_id , time) Ex. The result of GetCurrentData(sensor_id); <d:Data xmlns:d=“http://live-e.org/Ver200603/Data”> <value time=“2006-05-15T02:34:53.000+09:00”>23.5</value> <d:Data>

  16. (Major) Sensor Profile • sensor_id (Ex. live-e.org/WXT5 ---/Temperature) • sensor_model (Ex. WXT510) • sensor_type (Ex. Temperature) • latitude (Ex. 35.712194) • longitude (Ex. 139.76775) • location (Ex. 東大情報基盤センター) • address (Ex.東京都文京区弥生2-11-16)

  17. Sensor Profile Example • GetProfile(“live-e.org/WXT510/ --- /Temperature”); <p:Profile xmlns:p="http://live-e.org/Ver200603/Profile"> <sensor_id>live-e.org/WXT510/03000005c3a2/Temperature</sensor_id> <sensor_vendor>vaisala</sensor_vendor> <sensor_model>WXT510</sensor_model> <sensor_type>Temperature</sensor_type> <longitude>139.76775</longitude> <latitude>35.712194</latitude> <location>東大情報基盤センター</location> <address>東京都文京区弥生2-11-16</address> </p:Profile>

  18. API (live-e.hongo) • String GetCurrentDataAll(); • String GetCurrentData(String sensor_id); • String GetCurrentDataByAreaRect(double,double,double,double); • String GetCurrentDataByType(String sensor_type); • String GetCurrentDataOf(String sensor_id_like); • String GetDataByTimespan(String,Date,Date); • String GetProfileAll(); • String GetProfile(String sensor_id); • String GetProfileByAreaRect(double,double,double,double); • String GetProfileByType(String sensor_type);

  19. live-e.hongo APP APP APP Web Service(SOAP/XML) Debian Etch PostgreSQL8.1 Java+Axis live-e.hongo.wide.ad.jp weather.hongo WXT510 FS-Va-01 WM918

  20. Weather Station(WM918) • Measure Item • Temperature • Humidity • Pressure • RainFall • WindDir • WindSpeed WM918 live-e.hongo Internet PC RS232C SOAP

  21. Field Server (FS-Va-01) • Measure Item • Temperature • Humidity • Pressure • RainFall • WindDir • WindSpeed • Solar_Radiation • Soil_Moisture • Soil_Temperature • Photo FS-Va-01 DataCenter Matsushita-Denko live-e.hongo Internet SOAP

  22. Data Upload Protocol • Protocol • SOAP • Data Format • XML-like format • <Data> • <LiveE-ID>live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Temperature</LiveE-ID> • <Time>2006-05-12T04:34:42.0000+09:00</Time> • <Value>23.1</Value> • </Data> • ・・・ <Data></Data> continues

  23. Behavior of Data Upload TABLE sensor_data_current TABLE sensor_data_log data_index sensor_id time value sensor_id time value Update sensor_data_current Insert sensor_data_log DB_INSERT Web Service Data Sensors

  24. Characteristics (live-e.hongo) • Improved • Sensor Abstraction • Profile Management • The same as weather.hongo • Directly connected to the Database • Upload Message Flow • Query Processing • Centralized • All the sensors are managed in one database • No scalability

  25. Accident • Accidents Already Occurred • The Accident of New Measure Item • The Accident of Software Update • Accidents we must assume. • Server Crush • Rigid System • Difficulty of introducing new architecture or new model. • Database-oriented • Sensors are deployed without managing

  26. The Accident of New Measure Item • There is another data archive system. http://www.cnl.k.u-tokyo.ac.jp/~koba/live_e/index.php • This system collects data by parsing HTML of web interface of weather.hongo. http://weather.hongo.wide.ad.jp/WDShow/WDShow.aspx • One day, a new measure item “DayRainFall” was added. • Of course, the HTML structure was changed. • The archive system went into out of order.

  27. The Accident of Software Update • Sensor data upload software had a bug. • It had been already published. • Isi removed the bug. • Published again • Announced • One of us deployed the old version. • The fact was found out by human check. • live-e.hongo could not detect it. WM918 live-e.hongo Internet PC RS232C SOAP

  28. Discussion

  29. Discussion Agenda • Evaluate the system in many aspects • Scalability, Robustness, ... , etc. • Introduce related work. • Determine what we focus on ... • pub-sub? or range query? or any other? • Design prototype system • Determine what to do at the workshop

  30. Evaluate Point • Scalability • Robustness • Flexibility • Reliability • Efficiency • Functionality • Availability • Feasibility • Heterogeneity Management • Automation • Simplicity

  31. Scalability • What is O(N)? N: The Number of Node. • Resource Capacity VS O(N). • R>>O(N) at N=1,000,000 → Scale • R<O(N) at N=1,000,000 → Does Not Scale • Current System (live-e.hongo) • Does not scale!! • What do we need to introduce to obtain scalability?

  32. Robustness (Tolerance) • What would happen if ... ? • A server crushed. • Disconnected. • Current System • It must be a disaster. • Lose the data archived • Out of Service • Cannot collect sensor data • Take time to comeback • What do we have to do to obtain tolerance?

  33. Flexibility • Is it easy to extend ... ? • easy to add some functions ? • easy to add resources ? (such as disk space or CPU) • Current System • Rigid. • Each parts are tightly dependent. • We need lots of energy to remodel. • How do we design a flexible system?

  34. Reliability • Is the data correct? • Can we trust the value of sensors? • How deep can we detect irregular behavior? • Who has the responsibility of the data? • Certification • Current System • Trust all the sensor even if one of them said -324℃ • What do we need to improve reliability?

  35. Efficiency • Low cost? • Time, Money, Power, Space ... • It is said... The cost is: • big central > small distributed • Current System • Unknown • It is important to think of efficiency when we design system.

  36. Functionality • Complex Query OK? • Range query • Current system • Good • How do we obtain functionality within the system?

  37. Availability • Is Live E! system opened? • Example of open system • Internet • DHT • Current System • Not opend • Only the system manager can control the system. • How do we provide availability?

  38. Feasibility • Can we execute the plan? • Human Resource • Money • Law • We have to consider whether we can do it or not.

  39. Heterogeneity Management • How do we manage sensors uniformly? • Abstraction? • Namespace Management?

  40. Automation • What do we need to design system to work automatically? • When software updated. • When resource added.

  41. Simplicity • Simpler, More User • Few would like to learn complex access methods.

  42. RELATED WORK • Content-Based Network • Publish-Subscribe Service • DHT (Distributed Hash Table) • i3 (publish-subscribe on DHT) • PHT (range query on DHT) • Caching • Live Update • Sensor Abstraction

  43. What we focus on ... • Publish - Subscribe Model • Range Query • Data Mining • The Abstraction of Sensor • Live Update

  44. Prototype System • Depends on what we focus on ... • Talk about these: • Architecture • Interface and Protocol • Evaluation • Who

  45. At the Workshop • Evaluate the system. • What do you want to do? • I want to do something aggressive. Ex. Increase virtual sensors until the system fails.

More Related