h store a specialized architecture for high throughput oltp applications n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
H-Store: A Specialized Architecture for High-throughput OLTP Applications PowerPoint Presentation
Download Presentation
H-Store: A Specialized Architecture for High-throughput OLTP Applications

Loading in 2 Seconds...

play fullscreen
1 / 18

H-Store: A Specialized Architecture for High-throughput OLTP Applications - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

H-Store: A Specialized Architecture for High-throughput OLTP Applications. Evan Jones (MIT) Andrew Pavlo (Brown) 13 th Intl. Workshop on High Performance Transaction Systems October 26, 2009. Intel Xeon E5540 (Nehalem/Core i7).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

H-Store: A Specialized Architecture for High-throughput OLTP Applications


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
h store a specialized architecture for high throughput oltp applications
H-Store: A Specialized Architecture for High-throughputOLTP Applications

Evan Jones (MIT)Andrew Pavlo (Brown)

13th Intl. Workshop on High Performance Transaction Systems

October 26, 2009

intel xeon e5540 nehalem core i7
Intel Xeon E5540 (Nehalem/Core i7)

Source: Intel 64 and IA-32 Architectures Optimization Reference Manual

scaling oltp on multi core
Scaling OLTP on Multi-Core?

Use a distributed shared-nothing design

how to make a faster oltp dbms
How to Make a Faster OLTP DBMS
  • Main memory storage
    • Replication for durability
  • Explicitly partition the data
  • Specialized concurrency control
    • Stored procedures only
    • Single partition: execute one transaction at time
    • Multiple partitions: supported but slow
oltp where does the time go
OLTP: Where does the time go?

Source: S. Harizopoulos, D. J. Abadi, S. Madden, M. Stonebraker, “OLTP Under the Looking Glass”, SIGMOD 2008.

users rely on partitioning
Users Rely on Partitioning

Source: R. Shoup, D. Pritchett, “The eBay Architecture,” SD Forum, Nov. 2006.

what about multi core
What about multi-core?
  • Traditional approach:
    • One database process
    • Thread per connection
    • Shared-memory, locks and latches
  • H-Store approach:
    • Thread per partition
    • Distributed transactions
example microbenchmark
Example Microbenchmark
  • One table per client

Table(id INTEGER, counter INTEGER)

  • Each client executes the following query:

UPDATE TableSET counter = counter + 1WHERE id = 0;

  • Add clients to find maximum throughput
  • Data on RAM disk
partitions versus threads
Partitions versus Threads

Ideal

Partitions

Threads

scalability analysis
Scalability Analysis

Partitions scale better than threads.

  • Threads: contention for shared resources [1]
  • Partitions: memory bottleneck causes sublinear scaling

H-Store: Not just for distributed shared-nothing clusters

[1] R. Johnson et al., "Shore-MT: A Scalable Storage Manager for the Multicore Era," EDBT 2009.

multi core design problem
Multi-core Design Problem
  • How to automatically create a data placement scheme to improve multi-core throughput?
  • Data Partitioning:
    • Maximize the number of single-partition transactions.
  • Data Placement:
    • Maximize the number of single-node transactions.
database partitioning
Database Partitioning
  • Select partitioning keys and construct schema tree.

TPC-C Schema

Schema Tree

WAREHOUSE

WAREHOUSE

ITEM

DISTRICT

STOCK

DISTRICT

STOCK

CUSTOMER

CUSTOMER

ORDERS

ITEM

Replicated

ORDERS

ORDER_ITEM

ORDER_ITEM

database partitioning1
Database Partitioning
  • Combine table fragments into partitions.

Schema Tree

Partitions

P1

P2

P3

P4

P5

WAREHOUSE

P3

P4

P2

P5

P1

P1

P2

P3

P4

P5

P1

P2

P3

P4

P5

ITEM

ITEM

DISTRICT

STOCK

P1

P2

P3

P4

P5

CUSTOMER

ITEM

ITEM

P1

P2

P3

P4

P5

ORDERS

ITEM

ITEM

ITEMj

ITEM

ITEM

ITEM

Replicated

P1

P2

P3

P4

P5

ORDER_ITEM

ITEM

data placement
Data Placement
  • Assign partitions to cores on each node.

Partitions

Cluster Node

P1

P2

P4

P5

P3

P1

P2

ITEM

ITEM

P3

HT1

HT1

P4

P5

HT2

HT2

Partition Affinity

ITEM

ITEM

Core1

Core1

Core2

Core2

Node 1

Node n

ITEM

h store s future
H-Store’s Future
  • New Name. New Company.
  • Six full-time developers.
  • Open-source project (GPL)
  • Beta by end of 2009
    • Multiple deployments in financial service areas.
more information
More Information
  • H-Store Info + Papers:
    • http://db.cs.yale.edu/hstore/
  • VoltDB Project Information:
    • http://www.voltdb.org/