Cs 347 parallel and distributed data management notes 13 bigtable hbase cassandra
This presentation is the property of its rightful owner.
Sponsored Links
1 / 17

CS 347: Parallel and Distributed Data Management Notes 13: BigTable, HBASE, Cassandra PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on
  • Presentation posted in: General

CS 347: Parallel and Distributed Data Management Notes 13: BigTable, HBASE, Cassandra. Hector Garcia-Molina. Sources. HBASE: The Definitive Guide, Lars George, O’Reilly Publishers, 2011. Cassandra: The Definitive Guide, Eben Hewitt, O’Reilly Publishers, 2011.

Download Presentation

CS 347: Parallel and Distributed Data Management Notes 13: BigTable, HBASE, Cassandra

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cs 347 parallel and distributed data management notes 13 bigtable hbase cassandra

CS 347: Parallel and DistributedData ManagementNotes 13:BigTable, HBASE, Cassandra

Hector Garcia-Molina

Lecture 9B


Sources

Sources

  • HBASE: The Definitive Guide, Lars George, O’Reilly Publishers, 2011.

  • Cassandra: The Definitive Guide, Eben Hewitt, O’Reilly Publishers, 2011.

  • BigTable: A Distributed Storage System for Structured Data, F. Chang et al, ACM Transactions on Computer Systems, Vol. 26, No. 2, June 2008.

Lecture 9B


Lots of buzz words

Lots of Buzz Words!

  • “Apache Cassandra is an open-source, distributed, decentralized, elastically scalable, highly available, fault-tolerant, tunably consistent, column-oriented database that bases its distribution design on Amazon’s dynamo and its data model on Google’s Big Table.”

  • Clearly, it is buzz-word compliant!!

Lecture 9B


Basic idea key value store

Basic Idea: Key-Value Store

Table T:

Lecture 9B


Basic idea key value store1

Basic Idea: Key-Value Store

  • API:

    • lookup(key)  value

    • lookup(key range)  values

    • getNext  value

    • insert(key, value)

    • delete(key)

  • Each row has timestemp

  • Single row actions atomic(but not persistent in some systems?)

  • No multi-key transactions

  • No query language!

Table T:

keys are sorted

Lecture 9B


Fragmentation sharding

Fragmentation (Sharding)

server 1

server 2

server 3

tablet

  • use a partition vector

  • “auto-sharding”: vector selected automatically

Lecture 9B


Tablet replication

Tablet Replication

server 3

server 4

server 5

primary

backup

backup

  • Cassandra:Replication Factor (# copies)R/W Rule: One, Quorum, AllPolicy (e.g., Rack Unaware, Rack Aware, ...)Read all copies (return fastest reply, do repairs if necessary)

  • HBase: Does not manage replication, relies on HDFS

Lecture 9B


Need a directory

Need a “directory”

  • Table Name: Key  Server that stores key Backup servers

  • Can be implemented as a special table.

Lecture 9B


Tablet internals

Tablet Internals

memory

disk

Design Philosophy (?): Primary scenario is where all data is in memory.

Disk storage added as an afterthought

Lecture 9B


Tablet internals1

Tablet Internals

tombstone

memory

flush periodically

disk

  • tablet is merge of all segments (files)

  • disk segments imutable

  • writes efficient; reads only efficient when all data in memory

  • periodically reorganize into single segment

Lecture 9B


Column family

Column Family

Lecture 9B


Column family1

Column Family

  • for storage, treat each row as a single “super value”

  • API provides access to sub-values(use family:qualifier to refer to sub-values e.g., price:euros, price:dollars )

  • Cassandra allows “super-column”: two level nesting of columns (e.g., Column A can have sub-columns X & Y )

Lecture 9B


Vertical partitions

Vertical Partitions

can be manually implemented as

Lecture 9B


Vertical partitions1

Vertical Partitions

column family

  • good for sparse data;

  • good for column scans

  • not so good for tuple reads

  • are atomic updates to row still supported?

  • API supports actions on full table; mapped to actions on column tables

  • API supports column “project”

  • To decide on vertical partition, need to know access patterns

Lecture 9B


Failure recovery bigtable hbase

Failure Recovery (BigTable, HBase)

ping

master node

tablet server

sparetablet server

memory

write ahead logging

log

GFS or HFS

Lecture 9B


Failure recovery cassandra

Failure recovery (Cassandra)

  • No master node, all nodes in “cluster” equal

server 1

server 3

server 2

Lecture 9B


Failure recovery cassandra1

Failure recovery (Cassandra)

  • No master node, all nodes in “cluster” equal

access any table in clusterat any server

server 1

server 3

server 2

that server sends requeststo other servers

Lecture 9B


  • Login