molecular transactions l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Molecular Transactions PowerPoint Presentation
Download Presentation
Molecular Transactions

Loading in 2 Seconds...

play fullscreen
1 / 27

Molecular Transactions - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Molecular Transactions. G. Ramalingam Kapil Vaswani Rigorous Software Engineering, MSRI. Simple banking application. ACID Transaction. let transfer from to amount = atomic { match hasBalance from amount with | true -> debit from amount credit to amount SUCCESS

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Molecular Transactions' - brilliant


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
molecular transactions

Molecular Transactions

G. Ramalingam

Kapil Vaswani

Rigorous Software Engineering, MSRI

simple banking application
Simple banking application

ACID Transaction

let transfer from to amount =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

credit to amount

SUCCESS

| false -> INSUFFICIENT_BALANCE

}

Database

Application

server

Rigorous Software Engineering

Microsoft Research, India

slide3

?

Rigorous Software Engineering

Microsoft Research, India

scalable applications
Scalable applications
  • Support lots of users
    • TBs of data!
    • Many concurrent transactions
    • Unpredictable workload
  • Key requirements
    • Available
    • Fault tolerant
    • Data consistency
  • Achieving all at the same time non-trivial!

Application

server

Relational

Database

Rigorous Software Engineering

Microsoft Research, India

designing for scale

Designing for scale

Rigorous Software Engineering Microsoft Research, India

scaling application tier
Scaling application tier

Application

server

Database

Rigorous Software Engineering Microsoft Research, India

replication
Replication

Application

server

Replica

Replica

Replica

Rigorous Software Engineering Microsoft Research, India

data partitioning
Data partitioning

Application

server

A first-class citizen in cloud based storage systems

Data

partition

Data

partition

Rigorous Software Engineering Microsoft Research, India

horizontal data partitioning
Horizontal data partitioning

let transfer from to amount =

let status =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

DEBIT_COMPLETE

| false -> INSUFFICIENT_BALANCE

}

Distributed vs. local transactions

Database

partition

Distributed transactions do not scale 

Azure tables, BigTable, Dynamo do not support distributed transactions!

atomic{

matchstatus with

| DEBIT_COMPLETE ->

credit to amount

SUCESSS

| _ -> INSUFFICIENT_BALANCE

}

Database

partition

Rigorous Software Engineering

Microsoft Research, India

consistency
Consistency

let transfer from to amount =

let status =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

DEBIT_COMPLETE

| false -> INSUFFICIENT_BALANCE

}

Database

partition

atomic {

match status with

| DEBIT_COMPLETE ->

credit to amount

SUCESSS

| _ -> INSUFFICIENT_BALANCE

}

Database

partition

Rigorous Software Engineering

Microsoft Research, India

slide11
Rigorous Software Engineering

Microsoft Research, India

compute failures
Compute failures

let transfer from to amount =

let status =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

DEBIT_COMPLETE

| false -> INSUFFICIENT_BALANCE

}

Database

partition

Server

failures

Database

partition

Rigorous Software Engineering

Microsoft Research, India

handling compute failures
Handling compute failures

let transfer from to amount =

let status =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

DEBIT_COMPLETE

| false -> INSUFFICIENT_BALANCE

}

Database

partition

Checkpoint

storage

atomic{

matchstatus with

| DEBIT_COMPLETE ->

credit to amount

SUCESSS

| _ -> INSUFFICIENT_BALANCE

}

Database

partition

Rigorous Software Engineering

Microsoft Research, India

there s more logical failures
There’s more…logical failures

let transfer from to amount =

let status =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

DEBIT_COMPLETE

| false -> INSUFFICIENT_BALANCE

}

Database

partition

What if credit fails?

e.g. account closed

Database

partition

Rigorous Software Engineering

Microsoft Research, India

handling logical failures with compensating actions
Handling logical failures withcompensating actions

let transfer from to amount =

let status =

atomic {

matchhasBalance from amount with

| true ->

debit from amount

DEBIT_COMPLETE

| false -> INSUFFICIENT_BALANCE

}

let compensate from amount =

atomic {

credit from amount

}

Credit fails

Rigorous Software Engineering

Microsoft Research, India

molecular transactions16
Molecular transactions

Fault-tolerant composition of atomic transactions with compensating actions

molecule {

atomic {

}

atomic {

}

atomic {

}

atomic {

}

}

Rigorous Software Engineering

Microsoft Research, India

account transfer with mt
Account transfer with MT

let transfer from to amount =

molecule{

do! atomic {

match hasBalancefrom amount with

| true -> debit from amount

| false -> abort

}

|> compensateWith<|

atomic { credit from amount }

return! atomic {

match exists to with

| true -> credit to amount

| false -> abort

}

}

  • Hides
  • Serialization
  • Messaging
  • Fault tolerance

First atomic step

Compensation

  • No isolation
  • Compensating actions must semantically undo

Second atomic step

Rigorous Software Engineering

Microsoft Research, India

implementation

Implementation

Rigorous Software Engineering

Microsoft Research, India

system model
System model
  • Compute agents
    • May fail at any time
    • System detects failure and restarts agents
    • All agent-local state is lost
  • Partitioned storage
    • Partitions are units of serializability
  • Persistent queue
    • Communication between agents

Rigorous Software Engineering

Microsoft Research, India

persistent queue
Persistent queue
  • Not really a queue!
    • Out-of-order
    • At-least once
  • Operations
    • Enqueue(m)
    • Get()
      • Gets a message, makes message invisible
      • Message re-appears if not dequeued
    • Dequeue(m)
      • Permenantly deletes the message

Rigorous Software Engineering

Microsoft Research, India

tolerating agent failures
Tolerating agent failures
  • Agent can fail between transactions
  • Check-pointing
    • Check-point state after every transaction

Rigorous Software Engineering

Microsoft Research, India

check pointing
Check-pointing

Enqueue

Read

Dequeue

Database

partition

Database

partition

Rigorous Software Engineering

Microsoft Research, India

compute failures23
Compute failures

Enqueue

Read

  • Transactions may be evaluated more than once!
  • Messages may be sent more than once!

Dequeue

Database

partition

Rigorous Software Engineering

Microsoft Research, India

idempotence
Idempotence
  • Instrument transactions for idempotence
    • Assumes key-value store with a unique key constraint
    • Assign each atomic step in each MT a unique id
    • Add (unique id, return value) to write set
    • Idempotence log in transaction’s partition
  • If transaction evaluates more than once
    • Unique key violation causes transaction failure
    • Return stored value

Rigorous Software Engineering

Microsoft Research, India

summary
Summary
  • Molecular transactions
    • Language abstraction for programming weakly consistent applications
    • Hides messaging, queuing, fault tolerance
    • Expressed using a monad, lends to declarative, bug-free code
  • Efficient implementation on cloud based storage systems
    • Does not require 2PC!
    • Built a number of applications on Azure
    • < 10% overheads over hand-written code

Rigorous Software Engineering

Microsoft Research, India

open questions
Open questions
  • Specifying and reasoning about correctness of molecular transactions
  • Declarative ways of making consistency/performance trade-offs
  • Do cloud-based systems provide right primitives?

Rigorous Software Engineering

Microsoft Research, India

thank you

Thank you

Questions?

Rigorous Software Engineering

Microsoft Research, India