My database system is the only thing i can
This presentation is the property of its rightful owner.
Sponsored Links
1 / 47

MY DATABASE SYSTEM IS THE ONLY THING I CAN PowerPoint PPT Presentation


  • 45 Views
  • Uploaded on
  • Presentation posted in: General

MY DATABASE SYSTEM IS THE ONLY THING I CAN. TRUST. @ ANDY_PAVLO. Thirty Years Ago…. I NTERACTIVE T ransactions. S mall # of CPU C ores. S mall M emory S izes. Voter Setup. TP C-C BENCHMARK. Warehouse Order Processing. Application. Transaction:. NewOrder Transaction.

Download Presentation

MY DATABASE SYSTEM IS THE ONLY THING I CAN

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


My database system is the only thing i can

MY DATABASE SYSTEM ISTHE ONLY THING I CAN

TRUST

@ANDY_PAVLO


My database system is the only thing i can

Thirty Years Ago…


My database system is the only thing i can

INTERACTIVE Transactions

Small # of CPUCores

Small MemorySizes


Voter setup

Voter Setup

TPC-C BENCHMARK

Warehouse Order Processing

Application

Transaction:

NewOrder Transaction

Check whether user has already voted.

Insert new vote.

Update vote count for contestant.

Check item stock level.

Create new order information.

Update item stock levels.


Voter mysql postgres

Voter MySQL/Postgres

TPC-C BENCHMARK

Warehouse Order Processing

MySQL

Postgres

txn/sec

CPU CORES


Cpu overhead

CPU Overhead

TRADITIONAL DBMS

Measured CPU Cycles

30%

28%

30%

12%

OLTP Through the Looking Glass,

and What We Found There

SIGMOD, pp. 981-992, 2008.


My database system is the only thing i can

Hardware Upgrade

Replication

Distributed Cache

ShardingMiddleWare

NoSQL


How to scale up without giving up transactions

HOW TO SCALE UP WITHOUT GIVING UP TRANSACTIONS?


My database system is the only thing i can

Distributed Main Memory

Transaction Processing System

H-Store: A High-Performance, DistributedMain Memory Transaction Processing SystemProc. VLDB Endow., vol. 1, iss. 2, pp. 1496-1499, 2008.


My database system is the only thing i can

x

Disk Oriented

i

Main Memory Storage

Concurrent Execution

Serial Execution

/

Heavyweight Recovery

CompactLogging


H store architecture diagram

H-Store Architecture Diagram

Stored Procedure

Partitions

Transaction

Execution

VoteCount:

InsertVote:

Procedure Name

Input Parameters

Transaction

Result

INSERT INTO votes

VALUES (?, ?, ?);

SELECT COUNT(*)

FROM votes

WHEREphone_num = ?;

Application

run(phoneNum, contestantId, currentTime) {

result = execute(VoteCount, phoneNum);

if (result > MAX_VOTES) {

return (ERROR);

}

execute(InsertVote, phoneNum,

contestantId,

currentTime);

return (SUCCESS);

}

Single-Threaded

Execution Engines

CMdLog

Snapshots


Voter mysql postgres1

Voter MySQL/Postgres

TPC-C BENCHMARK

Warehouse Order Processing

MySQL

Postgres

H-Store

40x

txn/sec

CPU CORES


My database system is the only thing i can

Distributed Transactions


Distributed tpc c

Distributed TPC-C

TPC-C BENCHMARK

8 Cores per Node

10% Distributed Transactions

H-Store

txn/sec

Nodes


Distributed txn example

Distributed Txn Example

DISTRIBUTED TRANSACTIONS

Query Count

P1

P2

P3

P4

Application


Know what transactions will do before they start

KNOW WHAT TRANSACTIONSWILL DO BEFORE THEY START


But people always give me bad advice

BUT PEOPLE ALWAYS GIVE ME BAD ADVICE


My database system is the only thing i can

DON’T GET INVOLVED WITH COMPUTERS.

YOU’LL NEVER MAKE ANY MONEY.


My database system is the only thing i can

DON’T GET A PHD.

EVERYONE WILL THINK YOU ARE A JERK.


The database system always has more

THE DATABASE SYSTEMALWAYS HAS MORE

INFORMATION


Do use machine learning to predict transaction behavior

DO USE MACHINE LEARNINGTO PREDICTTRANSACTION BEHAVIOR.

On predictive modeling for optimizing

Transaction execution in parallel oltp systems

Proc. VLDB Endow., Vol 5, Iss. 2, pp. 85-96, 2011


Houdini overview

Houdini Overview

PREDICTIVE MODELS


Creating models

Creating Models

Model Generator

Feature

Clusterer

Classifier

WORKLOAD

SELECT * FROM WAREHOUSEWHERE W_ID = 10;

SELECT * FROM DISTRICT WHERE D_W_ID = 10 AND D_ID =9;

INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID,…) VALUES(10, 9, 12345,…);

______________________________________________________

______________________________________________________

SELECT * FROM WAREHOUSE WHERE W_ID = 10;

SELECT * FROM DISTRICT D_W_ID = 10 AND D_ID =9;

INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);

SELECT * FROM WAREHOUSE WHERE W_ID = 10;

INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);

SELECT * FROM WAREHOUSE WHERE W_ID = 10;

INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);

______________________________________________________

______________________________________________________

Decision Tree

Feature1

Markov Models

Feature2

Feature2


Houdini example

Houdini Example

DISTRIBUTED TRANSACTIONS

Application


Houdini tpc c

Houdini TPC-C

TPC-C BENCHMARK

8 Cores per Node

10% Distributed Transactions

Naïve

Houdini

Optimal

2x

txn/sec

Nodes


Houdini example1

Houdini Example

DISTRIBUTED TRANSACTIONS

Application

Zzzz…

Zzzz…

Zzzz…

Zzzz…

SP1- Waiting for Query Result

SP2- Waiting for Query Request

SP3- Two-Phase Commit


My database system is the only thing i can

TRANSACTION STALL POINTS

Base Partition

Remote Partition

18%

5%

73%

45%

37%

22%

SP1- Waiting for Query Result

SP3- Two-Phase Commit

SP2- Waiting for Query Request

Real Work


Do something useful when stalled

DO SOMETHING USEFULWHEN STALLED


My database system is the only thing i can

DON’T BE SURPRISED IF YOU & KB DON’T LAST THROUGH

GRAD SCHOOL.


My database system is the only thing i can

DON’T BE STAN’S STUDENT

IF YOU GO TOBROWN.


Do use machine learning to schedule speculative tasks

DO USE MACHINE LEARNINGTO SCHEDULESPECULATIVE TASKS.

The Art of Speculative Execution

In Progress (August 2013)


Serializable schedule

Serializable Schedule

SERIALIZABLE SCHEDULE

Distributed Transaction

Verify

C

Zzzz…

Single-Partition Transaction

Speculative Transaction

C

C

Speculative Transaction

Single-Partition Transaction

C

C


Hermes overview

Hermes Overview

SPECULATIVE TRANSACTIONS

Transaction Queue

Speculation Candidate:

Zzzz…

WRITE X

Distributed Transaction:

READ X

READ X


Hermes overview1

Hermes Overview

SPECULATIVE QUERIES

QueryY:

SELECT S_QTY FROM STOCK WHERE S_W_ID = ? AND S_I_ID = ?;

Distributed Transaction:


Mapping example

Mapping: Example

Transaction Parameters:

w_id=0

i_w_ids=[1,0] i_ids=[1001,1002]

GetWarehouse:

CheckStock:

SELECT * FROM WAREHOUSE WHERE W_ID = ?

SELECT S_QTY FROM STOCK WHERE S_W_ID = ? AND S_I_ID = ?;


Hermes overview2

Hermes Overview

VERIFICATION

Distributed Transaction

Speculative Transactions

Query1

Query2

Query3

Query1

Query3

Query3

Query3

Query1

Query2

Query3


Hermes tpc c

Hermes TPC-C

TPC-C BENCHMARK

8 Cores per Node

10% Distributed Transactions

Spec Queries

None

All

Spec Txns

txn/sec

Nodes


My database system is the only thing i can

Optimize Single-Partition Execution

H-Store: A High-Performance, Distributed Main Memory Transaction Processing System

Proc. VLDB Endow., vol. 1, iss. 2, pp. 1496-1499, 2008.

Minimize Distributed Transactions

Skew-Aware Automatic Database Partitioning in Shared-Nothing, Parallel OLTP Systems

Proceedings of SIGMOD, 2012.

Identify Distributed Transactions

On Predictive Modeling for Optimizing Transaction Execution in Parallel OLTP Systems

Proc. VLDB Endow., vol. 5, pp. 85-96, 2011.

Utilize Transaction Stalls

The Art of Speculative Execution

In Progress (August 2013)


My database system is the only thing i can

Future

Work


My database system is the only thing i can

One Size

Almost

Fits All


My database system is the only thing i can

N

H-STORE

S-STORE

N-STORE


My database system is the only thing i can

One Size

Almost

Fits All


My database system is the only thing i can

N

H-STORE

S-STORE

N-STORE


My database system is the only thing i can

Escape From Planet

Zdonik

(i.e., Andy Needs to Get Tenure)


Beyond the stores

Beyond the ‘Stores

  • Non-Partitionable Workloads.

  • The Poor Man’s Spanner.

  • Scientific Databases.


My database system is the only thing i can

DON’T MESS IT UP

WITH KB.


My database system is the only thing i can

StanZdonik

“The Thrill”Stonebraker

Sam

Madden

Ugur

Cetintemel

DavidDeWitt

DanAbadi

EvanJones

Saurya

Velagapudi

Xin

Jia

CarloCurino

JustinDeBrabant

YangZou

Visawee

Angkana.

NingShi

John

Meehan


  • Login