real time databases and data services
Download
Skip this Video
Download Presentation
Real-Time Databases and Data Services

Loading in 2 Seconds...

play fullscreen
1 / 23

Real-Time Databases and Data Services - PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on

Real-Time Databases and Data Services. Krithi Ramamritham, Sang Son, Lissa Dipippo. Satisfying QoS/QoD requirements. Transaction exec times & data access patterns are not known a priori, but vary dynamically Transaction timeliness & data freshness may pose conflicting requirements. Motivation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Real-Time Databases and Data Services' - norman


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
real time databases and data services

Real-Time Databases and Data Services

Krithi Ramamritham, Sang Son, Lissa Dipippo

satisfying qos qod requirements
Satisfying QoS/QoD requirements
  • Transaction exec times & data access patterns are not known a priori, but vary dynamically
  • Transaction timeliness & data freshness may pose conflicting requirements
motivation
Motivation
  • Increasing amount of sensor data
    • Agile manufacturing, target tracking, surveillance, structual monitoring, weather forecast, traffic control, ...
    • Wireless sensors push the limit
  • Desired real-time data service
    • Timely query/transaction processing using fresh data
  • Existing databases based on closed-world assumption
    • No notion of sensor data service
    • Timing guarantee or data freshness not considered
real time data service
Real-time data Service

Update Streams

QoS Guarantees?

User Requests

Real-Time

Transaction

Processing

Poor QoS

.

.

.

Overload

Data Conflicts

standford real time information processor strip
Standford real-time information processor (STRIP)
  • Addressed the problem of balancing between the freshness & timeliness requirements
  • Soft real-time: Maximize #transactions finishing within the deadlines using fresh data
  • Apreriodic updates
  • Freshness metrics
    • MA (Max Age): Similar to avi
    • Unapplied Updates: Be optimistic & assume a data is fresh unless an update has been received but not installed yet
strip
STRIP
  • Four scheduling algorithms
    • Update first
    • Transaction first
    • Split Updates
      • Update to high importance data will be installed upon arrival; otherwise, scheduled after transactions
    • On Demand
      • Transaction has precedence, but searches update queue upon accessing a stale data
qmf qos aware miss ratio and freshness guarantees
QMF (QoS-aware Miss ratio and Freshness guarantees)
  • Guaranteed Quality of real-time data service
    • Timing guarantee
      • Support desired deadline miss ratio or response time
      • Home-land security problem, traffic jam, ...
    • Quality of Data
      • Data freshness (temporal consistency)
      • Reflect the time-varying real-world status
challenges
Challenges
  • Unpredictable workloads, access patterns
    • Sudden increase of service requests
    • Severe data contention
  • Conflicts between timing & freshness requirements
    • Some deadline misses/freshness violations are inevitable
key ideas
Key Ideas
  • Feedback control
    • Robustness against unpredictable workloads
    • Widely applied to software peformance management
      • Feedback Control of Computing Systems, Joseph L. Hellerstein et., Wiley Interscience
  • Cost-effective freshness management
    • Novel freshness metrics
    • Cost-benefit model
    • Adaptive update policy
  • Admission control
    • Tardy commits are worse than dropping
qmf architecture
QMF Architecture

U. Thresh.

Manager

MR/Util.

Controllers

MR, Util.

U

Unew

QoD Manager

User

Trans.

Trans.

Handler

Dispatch

Abort/Restart

Admission

Control

Update

Streams

Ready Queue

Block Queue

real time database model
Real-Time Database Model
  • Main memory database model
    • TimesTen, STRIP, DataBlitz, Polyhedra
  • Firm deadlines
  • Sensor data updates
    • Periodic
    • Jitter?
  • User transactions
    • Arithmetic/logical operations considering the current real-world state
qos metrics miss ratio
QoS Metrics: Miss Ratio
  • Admitted transactions
  • Average
  • Transient
    • Overshoot (V)
    • Settling time (Ts)

V

Ts

qos metrics data freshness
QoS Metrics: Data Freshness

RTDB

Database Freshness:

Set of temporal data

Perceived Freshness:

Set of temporal data

accessed by timely

transactions

feedback based miss ratio utilization control
Feedback-Based Miss Ratio/Utilization Control

Transactions

MR

Threshold

Current

MR

Error

W

MR

Controller

RTDB

+

_

feedback control details
Feedback Control Details
  • PI controllers
  • Sampling period
    • Settling time, overshoot
    • 5sec
      • Overhead
      • QoD fluctuations
      • Think time
  • Tuning
    • Root Locus method in Matlab
cost effective updates
Cost-Effective Updates
  • Access Update Ratio (AUR)
    • AUR[i] = Access Freq[i] / Update Freq[i]
    • Access Frequency
      • Benefit
    • Update Frequency
      • Cost
cost benefit model
Cost-Benefit Model

Hot Data

AUR = 1

Cold Data

adaptive update policy
Adaptive Update Policy

D = Dimm

Dimm

Dimm

AUR =1

AUR < 1

Dod

Dod

Underutilized State

Moderately loaded State

Overloaded State

future work
Future Work
  • More flexible QoD metrics and adaptive QoD management schemes
  • More effective feedback control & QoS/QoD management approaches
  • RTDB testbed
  • Apply RTDB techniques to e-commerce applications
    • 3-tier systems: Web server, application server & back-end database
  • Mobile, hand-held RTDB
real time data service in embedded applications
Real-time data service in embedded applications
  • Data needs of embedded applications become more complicated
    • Traffic control
    • Weather forecast
    • Put RT data server in the frontline
      • Collect info rather than raw data and disseminate in a timely manner
      • Key issues such as power/energy management should be reconsidered considering real-time data service semantics
wireless sensor networks
Wireless Sensor Networks
  • View WSN as a distributed DB (???)
    • TinyDB, Cougar, SINA, ...
    • I don’t agree, but you can have different opinions...
  • Real-time event detection
  • Real-time routing
  • Data aggregation
  • QoD in WSNs – anything more than freshness? data values? accuracy? SNR?
  • Security & Trustworthiness
    • Scalability of depandability
    • Resilience of availability
vision example
Vision (Example)

Real-time traffic control using sensor data and weather information

ad