Real time databases and data services
Download
1 / 23

Real-Time Databases and Data Services - PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on

Real-Time Databases and Data Services. Krithi Ramamritham, Sang Son, Lissa Dipippo. Satisfying QoS/QoD requirements. Transaction exec times & data access patterns are not known a priori, but vary dynamically Transaction timeliness & data freshness may pose conflicting requirements. Motivation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Real-Time Databases and Data Services' - norman


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Real time databases and data services

Real-Time Databases and Data Services

Krithi Ramamritham, Sang Son, Lissa Dipippo


Satisfying qos qod requirements
Satisfying QoS/QoD requirements

  • Transaction exec times & data access patterns are not known a priori, but vary dynamically

  • Transaction timeliness & data freshness may pose conflicting requirements


Motivation
Motivation

  • Increasing amount of sensor data

    • Agile manufacturing, target tracking, surveillance, structual monitoring, weather forecast, traffic control, ...

    • Wireless sensors push the limit

  • Desired real-time data service

    • Timely query/transaction processing using fresh data

  • Existing databases based on closed-world assumption

    • No notion of sensor data service

    • Timing guarantee or data freshness not considered


Real time data service
Real-time data Service

Update Streams

QoS Guarantees?

User Requests

Real-Time

Transaction

Processing

Poor QoS

.

.

.

Overload

Data Conflicts


Standford real time information processor strip
Standford real-time information processor (STRIP)

  • Addressed the problem of balancing between the freshness & timeliness requirements

  • Soft real-time: Maximize #transactions finishing within the deadlines using fresh data

  • Apreriodic updates

  • Freshness metrics

    • MA (Max Age): Similar to avi

    • Unapplied Updates: Be optimistic & assume a data is fresh unless an update has been received but not installed yet


Strip
STRIP

  • Four scheduling algorithms

    • Update first

    • Transaction first

    • Split Updates

      • Update to high importance data will be installed upon arrival; otherwise, scheduled after transactions

    • On Demand

      • Transaction has precedence, but searches update queue upon accessing a stale data


Qmf qos aware miss ratio and freshness guarantees
QMF (QoS-aware Miss ratio and Freshness guarantees)

  • Guaranteed Quality of real-time data service

    • Timing guarantee

      • Support desired deadline miss ratio or response time

      • Home-land security problem, traffic jam, ...

    • Quality of Data

      • Data freshness (temporal consistency)

      • Reflect the time-varying real-world status


Challenges
Challenges

  • Unpredictable workloads, access patterns

    • Sudden increase of service requests

    • Severe data contention

  • Conflicts between timing & freshness requirements

    • Some deadline misses/freshness violations are inevitable


Key ideas
Key Ideas

  • Feedback control

    • Robustness against unpredictable workloads

    • Widely applied to software peformance management

      • Feedback Control of Computing Systems, Joseph L. Hellerstein et., Wiley Interscience

  • Cost-effective freshness management

    • Novel freshness metrics

    • Cost-benefit model

    • Adaptive update policy

  • Admission control

    • Tardy commits are worse than dropping


Qmf architecture
QMF Architecture

U. Thresh.

Manager

MR/Util.

Controllers

MR, Util.

U

Unew

QoD Manager

User

Trans.

Trans.

Handler

Dispatch

Abort/Restart

Admission

Control

Update

Streams

Ready Queue

Block Queue


Real time database model
Real-Time Database Model

  • Main memory database model

    • TimesTen, STRIP, DataBlitz, Polyhedra

  • Firm deadlines

  • Sensor data updates

    • Periodic

    • Jitter?

  • User transactions

    • Arithmetic/logical operations considering the current real-world state


Qos metrics miss ratio
QoS Metrics: Miss Ratio

  • Admitted transactions

  • Average

  • Transient

    • Overshoot (V)

    • Settling time (Ts)

V

Ts


Qos metrics data freshness
QoS Metrics: Data Freshness

RTDB

Database Freshness:

Set of temporal data

Perceived Freshness:

Set of temporal data

accessed by timely

transactions


Feedback based miss ratio utilization control
Feedback-Based Miss Ratio/Utilization Control

Transactions

MR

Threshold

Current

MR

Error

W

MR

Controller

RTDB

+

_


Feedback control details
Feedback Control Details

  • PI controllers

  • Sampling period

    • Settling time, overshoot

    • 5sec

      • Overhead

      • QoD fluctuations

      • Think time

  • Tuning

    • Root Locus method in Matlab


Cost effective updates
Cost-Effective Updates

  • Access Update Ratio (AUR)

    • AUR[i] = Access Freq[i] / Update Freq[i]

    • Access Frequency

      • Benefit

    • Update Frequency

      • Cost


Cost benefit model
Cost-Benefit Model

Hot Data

AUR = 1

Cold Data


Adaptive update policy
Adaptive Update Policy

D = Dimm

Dimm

Dimm

AUR =1

AUR < 1

Dod

Dod

Underutilized State

Moderately loaded State

Overloaded State


Future work
Future Work

  • More flexible QoD metrics and adaptive QoD management schemes

  • More effective feedback control & QoS/QoD management approaches

  • RTDB testbed

  • Apply RTDB techniques to e-commerce applications

    • 3-tier systems: Web server, application server & back-end database

  • Mobile, hand-held RTDB


Real time data service in embedded applications
Real-time data service in embedded applications

  • Data needs of embedded applications become more complicated

    • Traffic control

    • Weather forecast

    • Put RT data server in the frontline

      • Collect info rather than raw data and disseminate in a timely manner

      • Key issues such as power/energy management should be reconsidered considering real-time data service semantics


Wireless sensor networks
Wireless Sensor Networks

  • View WSN as a distributed DB (???)

    • TinyDB, Cougar, SINA, ...

    • I don’t agree, but you can have different opinions...

  • Real-time event detection

  • Real-time routing

  • Data aggregation

  • QoD in WSNs – anything more than freshness? data values? accuracy? SNR?

  • Security & Trustworthiness

    • Scalability of depandability

    • Resilience of availability


Vision example
Vision (Example)

Real-time traffic control using sensor data and weather information



ad