A Simple Optimistic skip-list Algorithm

A Simple Optimistic skip-listAlgorithm Maurice Herlihy Brown University & Sun Microsystems Laboratories Yossi Lev Brown University & Sun Microsystems Laboratories Victor Luchangco Sun Microsystems Laboratories Nir Shavit Tel-Aviv University & Sun Microsystems Laboratories Assaf Shemesh

Talk outline • Introduction. • What is skip-list? • The Algorithm. • Add operation. • Remove operation. • Contains operation. • Correctness. • Performance. • Conclusions.

Why use skip-list on Concurrent Programming? • Why not use trees? • like Red-Black trees, AVL trees, etc… • Because trees need to be rebalanced after • adding and removing items. • This operations are GLOBAL.

Introduction • Important data structure for storing and retrieving ordered in-memory data. • This algorithm VS Implementation by Pugh and by Doug Lea. • Main keys of this algorithm: • Optimistic - The methods traverse the list without acquiring locks. • In addition Only when a method discovers the items it is seeking, does it lock the item and its predecessors. • Lazy - removing an item involves logically deleting it by marking it before it is physically removed. • A typical pattern is 90% search operations, 9% inserts, and only 1% deletes • Works as well as the Lea’s algorithm under this common usage pattern, • and much better only under uncommon conditions of extreme contention in • multi programmed environments. • This algorithm provide a viable alternative to Lea algorithm.

What is skip-list? • Linked list sorted by keys. • Each node is assigned a random height, up to maximum for • some fixed probability P. • The number of nodes at any height decreases exponentially • with that height. • Each node has one successor at each level of height. • INVARIENT: Each list is a sub list of the list at the layer beneath it. • Usually there is a left sentinel and right sentinel at the beginning • and the end of the lists with maximum height.

- + What is skip-list? Operations on skip-list • Searching – search 78 in the set S3 S2 - 31 + S1 - 23 31 34 64 + S0 - 12 23 26 31 34 44 56 64 78 + O(logn)

The algorithm • Lazy list at each layer. • The key of each node is strictly greater than the key of its predecessor. Class Node{ int key ; int topLayer ; Node** nexts ; bool marked ; bool fullyLinked; Lock lock ; }; Boolean value which is used to make remove operations appear atomic. Boolean value which is set to true after a node has been linked in at all its layers The value of the height of the node. The value of the node. Array in size of top layer. For each layer the node contain his successor.

The algorithm • A key is in the abstract set if and only if there is an unmarked, • fully linkednode with that key in the list. • changes are made to the list structure only when locks are acquired for all • nodes that need to be modified. • Use LSentinel and RSentinel with the values MININT-1 and MAXINT+1. • MININT - The lowest value that can enter the set. • MAXINT - The highest value that can enter the set.

The algorithm “findNode” helper function • Searching the set us accomplished by the findNode helper function. • findNode – • Input: A key v and two arrays of node pointers, “preds” and “succs”. • Output: Returns the height if the node if it was found, otherwise, • it returns -1. • sets “preds” and “succs” accordantly. • findNode searches exactly as in a sequential skip-list. • findNode does not acquire any locks. • findNode don’t retry in case of conflicting access with some other thread.

The algorithm “findNode” helper function int findNode ( int v ,Node* preds [ ] ,Node* succs [ ] ) { int lFound = -1; Node* pred = &LSentinel ; for (int layer = MaxHeight -1;layer > 0 ;layer--) { Node* curr = pred->nexts [ layer ] ; while ( v > curr->key ) { pred = curr ; curr = pred->nexts [ layer ] ; } if ( lFound == -1 && v == curr->key ) { lFound = layer ; } preds [ layer ] = pred ; succs [ layer ] = curr ; } return lFound ; }

The algorithm - add add(v) adds v to the set and returns true iff v was not already in the set. General Description • Calls findNode to determine whether a node with the key • is already in the list. STEP 1 int topLayer = randomLevel ( MaxHeight ) ; Node* preds [ MaxHeight ] , succs [MaxHeight ] ; while ( true ) { int lFound = findNode ( v , preds , succs ) ; if ( lFound != -1) { Node* nodeFound = succs [ lFound ] ; if ( ! nodeFound->marked ) { while ( ! nodeFound->fullyLinked ) { ; } return false ; } continue ; }

The algorithm - add Example – add 6 to the set - Marked flag 6 - Fullylinked flag 0 9 7 0 5 8 0 2

The algorithm - add STEP 1 Example – add 6 to the set - Marked flag 6 6 - Fullylinked flag 0 9 0 7 0 5 8 0 2

The algorithm - add add(v) adds v to the set and returns true iff v was not already in the set. General Description • locks and validates all the predecessors • returned by findNode up to the height of the new node. STEP 2 Int highestLocked = -1; try { Node *pred , *succ , *prevPred = null ; bool valid = true ; for ( int layer = 0 ; valid && ( layer <= topLayer ) ;layer++) { pred = preds [ layer ] ; succ = succs [ layer ] ; if ( pred != prevPred ) { pred->lock . lock ( ) ; highestLocked = layer ; prevPred = pred ; } valid = ! Pred->marked && ! Succ->marked && pred¡>nexts [ layer]==succ ; } If ( ! valid ) continue;

The algorithm - add STEP 2 Example – add 6 to the set 6 6 - Marked flag - Fullylinked flag 0 9 0 7 0 0 5 8 0 2

The algorithm - add add(v) adds v to the set and returns true iff v was not already in the set. General Description • allocates a new node with the appropriate key • and height and links it in. STEP 3 Node* newNode = new Node ( v , topLayer ) ; for ( int layer = 0 ;layer <= topLayer ;layer++) { newNode->nexts [ layer ] = succs [ layer ] ; preds [ layer]->nexts [ layer ] = newNode ; } newNode->fullyLinked = true; return true ; }//end of try finally { unlock ( preds , highestLocked ) ; } }//end of function

The algorithm - add STEP 3 Example – add 6 to the set 6 6 - Marked flag - Fullylinked flag 0 9 0 7 0 5 0 8 0 2 Fullylinked = true

The algorithm - add Example – add 6 to the set - Marked flag - Fullylinked flag 0 9 0 7 6 0 5 0 8 0 2 0 Success!

The algorithm - remove removes v from the set and returns true iff v was in the set. General Description • okToDelete function bool okToDelete ( Node¤ candidate , int lFound ) { return ( candidate->fullyLinked && candidate->topLayer==lFound && !Candidate->marked ) ; }

The algorithm - remove removes v from the set and returns true iff v was in the set. General Description • determine whether a node with the appropriate key is in the list Node* nodeToDelete = null ; bool isMarked = false ; int topLayer = -1; Node* preds [ MaxHeight ] , succs [ MaxHeight ] ; while ( true ) { int lFound = findNode ( v , preds , succs ) ; if ( isMarked || ( lFound != -1 && okToDelete ( succs [ lFound ] , lFound ) ) ) { if ( ! isMarked ) { nodeToDelete = succs [ lFound ] ; topLayer = nodeToDelete->topLayer ; nodeToDelete->lock . lock ( ) ; if ( nodeToDelete¡>marked ) { nodeToDelete->lock . unlock ( ) ; return false ; } nodeToDelete->marked = true ; isMarked = true ; } STEP 1

The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag 0 9 0 7 6 0 0 5 0 8 0 2 0

The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 1 0 9 0 7 6 0 5 0 8 0 2 0

The algorithm - remove removes v from the set and returns true iff v was in the set. General Description • locking its predecessors at all layers up to the height of the deleted node int highestLocked = -1; try { Node *pred , *succ , *prevPred = null ; bool valid = true ; for ( int layer = 0 ;valid && ( layer <= topLayer ) ;layer++) { pred = preds [ layer ] ; succ = succs [ layer ] ; if ( pred != prevPred ) { pred->lock . lock ( ) ; highestLocked = layer ; prevPred = pred ; } valid = ! Pred->marked && pred->nexts [ layer]==succ ; } if ( ! valid ) continue ; STEP 2

The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag 0 9 0 7 6 1 0 5 0 8 0 2 0

The algorithm - remove removes v from the set and returns true iff v was in the set. General Description • splicing the node out one layer at a time STEP 3 for (int layer = topLayer ; layer >=0 ; layer--) { preds [ layer]->nexts [ layer ] = nodeToDelete->nexts [ layer ] ; } nodeToDelete->lock . unlock ( ) ; return true ; }//end of try finally { unlock ( preds , highestLocked ) ; } }//end of if(true) else return false ; }//end of while }//end of function

The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3 0 9 0 7 6 1 0 5 0 8 0 2 0

The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3 0 9 0 7 0 5 0 8 0 2 0

The algorithm - contains returns true iff v is in the set General Description • calls findNode and returns true if and only if • it finds a unmarked, fully linked node with the • appropriate key.

q.enq(x) q.deq(y) q.enq(y) q.deq(x) time Correctness Linearzability an operation (or set of operations) is linearizable ifit appears to the rest of the system to occur instantaneously. Linearizability:(an intuitive definition)Can find a point within the time-interval of each operation, where the operation took place, such that the operations order is legal. q.enq(x) q.deq(y) q.enq(y) q.deq(x) time

time Correctness Linearzability • Linearzability point of REMOVE is when MARKED bit is set. • Linearzability point of ADD is when FULLYLINKED bit is set. newNode->fullyLinked = true ; Add(v) Remove(v) nodeToDelete->marked = true ; time

Correctness Maintaining the skip-list invariant • skip-list invariant - the list at each layer is a sub list of the lists at lower layers. • Add operation - linking new nodes into the skip list always • proceeds from bottom to top. • Remove operation - the higher layers are unlinked before • the lower layers.

Correctness Dead lock freedom • a thread always acquires locks on nodes with larger keys first. • if a thread holds a lock on a node with key v then it will not attempt • to acquire a lock on a node with key greater than or equal to v. • acquiring locks on the predecessor nodes from the bottom layer up • Contains operation is: • Wait-freedom • Lock-freedom • Obstruction-freedom (if every operation has a bound on the number of steps the algorithm will take before the operation completes) (if it satisfies that when the program threads are run sufficiently long at least one of the threads make progress) (if at any point, a single thread executed in isolation)

Performance • This algorithm VS Concurrent skip-list written by Doug Lea • Throughput in operations per millisecond of 1,000,000 operations • 90% search operations, 9% inserts, and only 1% deletes. • Different versions of Experiments

Conclusions • A scalable highly concurrent skip-list using a • simple algorithm. • Support operations - Add, Remove, Contains. • Simple correctness prove • For most uses, it offers an interesting viable • alternative to the Concurrent Skip List by • Doug Lea.

A Simple Optimistic skip-list Algorithm