concurrent queues l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Concurrent Queues PowerPoint Presentation
Download Presentation
Concurrent Queues

Loading in 2 Seconds...

play fullscreen
1 / 65

Concurrent Queues - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

Concurrent Queues. שירן חליבה. Outline:. Some definitions 3 queue implementations : A Bounded Partial Queue An Unbounded Total Queue An Unbounded Lock-Free Queue. Introduction and some definitions: .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Concurrent Queues' - lance


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
concurrent queues

Concurrent Queues

שירן חליבה

outline
Outline:
  • Some definitions
  • 3 queue implementations :
    • A Bounded Partial Queue
    • An Unbounded Total Queue
    • An Unbounded Lock-Free Queue
introduction and some definitions
Introduction and some definitions:
  • Pools show up in many places in concurrent systems. Forexample, in many applications, one or more producer threads produce items to beconsumed by one or more consumer threads.
  • To allow consumers to keep up, we can place a bufferbetween the producers and the consumers.
  • Often, pools act as producer–consumer buffers.
  • A pool allows the same item to appear more than once.
introduction and some definitions cont
Introduction and some definitions cont.
  • A queue is a special kind of pool with FIFO fairness.
  • Itprovides an enq(x) method that puts item x at one end of the queue, called thetail, and a deq() method that removes and returns the item at the other end of thequeue, called the head.
bounded vs unbounded
Bounded vs. Unbounded
  • A pool can be bounded or unbounded.
  • Bounded
    • Fixed capacity
    • Good when resources an issue
  • Unbounded
    • Holds any number of objects
blocking vs non blocking
Blocking vs. Non-Blocking
  • Problem cases:
    • Removing from empty pool
    • Adding to full (bounded) pool
  • Blocking
    • Caller waits until state changes
  • Non-Blocking
    • Method throws exception
total vs partial
Total vs. Partial
  • Pool methods may be total or partial.
  • A method is total if calls do not wait for certain conditions to become true. For example, a get() call that tries to remove an item from an empty pool immediately returns a failure code or throws an exception. A total interface makes sense whenthe producer (or consumer) thread has something better to do than waitfor the method call to take effect.
total vs partial8
Total vs. Partial
  • A method is partial if calls may wait for conditions to hold. For example, a partial get() call that tries to remove an item from an empty pool blocks until an item is available to return. A partial interface makes sense when the producer (or consumer) has nothing better to do than to wait for the pool to become nonempty (or non full).
queue concurrency
Queue: Concurrency

y=deq()

enq(x)

tail

head

enq() and deq() work at different ends of the object

concurrency
Concurrency

y=deq()

enq(x)

head

tail

Challenge: what if the queue is empty or full?

a bounded partial queue
A Bounded Partial Queue

Sentinel

First actual item

head

tail

deqLock

Lock out other deq() calls

enqLock

Lock out other enq() calls

permits

8

Permission to enqueue 8 items

enqueuer
Enqueuer

head

tail

deqLock

No need to lock tail?

Read permits

Lock enqLock

enqLock

permits

8

OK

enqueuer13
Enqueuer

head

tail

deqLock

enqLock

Enqueue Node

permits

8

7

getAndDecrement()

(Why atomic?)

slide14

Enqueuer

head

tail

deqLock

If queue was empty, notify waiting dequeuers

enqLock

permits

Release lock

8

7

unsuccesful enqueuer
Unsuccesful Enqueuer

head

tail

deqLock

Read permits

enqLock

permits

0

Uh-oh

dequeuer
Dequeuer

head

tail

deqLock

Read sentinel’s next field

enqLock

Lock deqLock

permits

7

OK

dequeuer17
Dequeuer

head

tail

deqLock

Read value

enqLock

permits

7

dequeuer18

Make first Node new sentinel

Dequeuer

head

tail

deqLock

enqLock

permits

7

dequeuer19
Dequeuer

head

tail

deqLock

enqLock

permits

7

Release deqLock

dequeuer20
Dequeuer

head

tail

deqLock

enqLock

Increment permits

(no need lock?)

permits

8

Answer: we had to hold the lock while enqueuing to prevent lots of enqueuers from proceeding without noticing that the capacity had been exceeded. Dequeuers will notice the queue is empty when they observe that the sentinel’s next field is null

unsuccesful dequeuer
Unsuccesful Dequeuer

head

tail

deqLock

Read sentinel’s next field

enqLock

permits

8

uh-oh

slide22

Bounded Queue

public class BoundedQueue<T> {

ReentrantLock enqLock, deqLock;

Condition notEmptyCondition, notFullCondition;

AtomicInteger permits;

Node head;

Node tail;

int capacity;

enqLock = new ReentrantLock();

notFullCondition = enqLock.newCondition();

deqLock = new ReentrantLock();

notEmptyCondition = deqLock.newCondition();

}

slide23

The ReentrantLock is a monitor (The mechanism that Java uses to support synchronization ). Allows blocking on a condition rather than spinning.How do we use it?(*More on monitors: http://www.artima.com/insidejvm/ed2/threadsynch.html)

lock conditions
Lock Conditions

public interface Condition {

void await();

boolean await(long time, TimeUnit unit);

void signal();

void signalAll();

}

await
Await
  • Releases lock associated with q
  • Sleeps (gives up processor)
  • Awakens (resumes running)
  • Reacquires lock & returns

q.await()

signal
Signal
  • Awakens one waiting thread
    • Which will reacquire lock

q.signal();

a monitor lock
A Monitor Lock

Lock()

waiting room

unLock()

Critical Section

unsuccessful deq
Unsuccessful Deq

Lock()

waiting room

Deq()

await()

Critical Section

Oh no, Empty!

another one
Another One

Lock()

waiting room

Deq()

await()

Critical Section

Oh no, Empty!

enqueur to the rescue
Enqueur to the Rescue

Yawn!

Yawn!

Lock()

waiting room

Enq( )

signalAll()

Critical Section

unLock()

monitor signalling
Monitor Signalling

Yawn!

Yawn!

waiting room

Critical Section

Awakend thread

might still lose lock to

outside contender…

dequeurs signalled
Dequeurs Signalled

Yawn!

waiting room

Found it

Critical Section

dequeurs signalled33
Dequeurs Signalled

Yawn!

waiting room

Critical Section

Still empty!

dollar short day late
Dollar Short + Day Late

waiting room

Critical Section

lost wake up
Lost Wake-Up

Yawn!

Lock()

waiting room

Enq( )

signal ()

Critical Section

unLock()

lost wake up37
Lost Wake-Up

Yawn!

Lock()

waiting room

Enq( )

Critical Section

unLock()

lost wake up38
Lost Wake-Up

Yawn!

waiting room

Critical Section

lost wake up39
Lost Wake-Up

waiting room

Found it

Critical Section

what s wrong here
What’s Wrong Here?

zzzz….!

waiting room

Critical Section

enq method
Enq Method

public void enq(T x) {

boolean mustWakeDequeuers = false;

enqLock.lock();

try {

while (permits.get() == 0)

notFullCondition.await();

Node e = new Node(x);

tail.next = e;

tail = e;

if (permits.getAndDecrement() == capacity)

mustWakeDequeuers = true;

} finally {

enqLock.unlock();

}

}

slide42
Cont…

public void enq(T x) {

if (mustWakeDequeuers) {

deqLock.lock();

try {

notEmptyCondition.signalAll();

} finally {

deqLock.unlock();

}

}

}

the enq deq methods
The Enq() & Deq() Methods
  • Share no locks
    • That’s good
  • But do share an atomic counter
    • Accessed on every method call
    • That’s not so good
  • Can we alleviate this bottleneck?

What is the problem?

split the counter
Split the Counter
  • The enq() method
    • Decrements only
    • Cares only if value is zero
  • The deq() method
    • Increments only
    • Cares only if value is capacity
split counter
Split Counter
  • Enqueuer decrements enqSidePermits
  • Dequeuer increments deqSidePermits
  • When enqueuer runs out
    • Locks deqLock
    • Transfers permits
    • (dequeuer doesn't need permits- check head.next)
  • Intermittent(תקופתי) synchronization
    • Not with each method call
    • Need both locks! (careful …)
an unbounded total queue
An Unbounded Total Queue
  • Queue can hold an unbounded number of items.
  • The enq() method always enqueues its item.
  • The deq() throws EmptyException if there is no item to dequeue.
  • No deadlock- each method acquires only one lock.
  • Both the enq() and deq() methods are total as they do not wait for the queue to become empty or full.
a lock free queue
A Lock-Free Queue

Sentinel

head

tail

  • Extension of the unbounded total queue
  • Quicker threads help the slower threads
  • Each node’s next field is an:
  • AtomicReference<Node>
  • The queue itself consists of two
  • AtomicReference<Node> fields:
  • head and tail
enqueue
Enqueue

head

tail

Enq( )

enqueue52
Enqueue

head

tail

logical enqueue
Logical Enqueue

CAS

head

tail

physical enqueue
Physical Enqueue

head

CAS

tail

Enqueue Node

enqueue55
Enqueue
  • These two steps are not atomic
  • The tail field refers to either
    • Actual last Node (good)
    • Penultimate* Node (not so good)
  • Be prepared!

(*Penultimate :next to the last)

enqueue56
Enqueue
  • What do you do if you find
    • A trailing tail?
  • Stop and fix it
    • If tail node has non-null next field
    • CAS the queue’s tail field to tail.next
when cass fail
When CASs Fail
  • During logical enqueue
    • Abandon hope, restart
  • During physical enqueue
    • Ignore it (why?)
slide59
Enq()
  • Creates a new node with the new value to be enqueued
  • reads tail, and finds the node that appears to be last
  • checks whether that node has a successor
  • If not - appends the new node by calling compareAndSet()
  • If the compareAndSet() succeeds, the thread uses a second compareAndSet() to advance tail to the new node
  • second compareAndSet() call fails, the thread can still return successfully
  • If the tail node has a successor , then the method tries to “help”other threads by advancing tail to refer directly to the successor before trying again to insert its own node.
dequeuer60
Dequeuer

head

tail

Read value

slide64
Deq()
  • If the queue is nonempty(the next field of the head node is not null), the dequeuer calls compareAndSet() to change head from the sentinel node to its successor
  • before advancing head one must make sure that tail is not left referring to the sentinel node which is about to be removed from the queue
  • test: if head equals tail and the (sentinel) node they refer to has a non-null next field, then the tail is deemed to be lagging behind.
  • deq() then attempts to help make tail consistent by swinging it to the sentinel node’s successor , and only then updates head to remove the sentinel
summary
Summary
  • A thread fails to enqueue or dequeue a node only if another thread’s method call succeeds in changing the reference, so some method call always completes.
  • As it turns out, being lock-free substantially enhances the performance of queue implementations, and the lock-free algorithms tend to outperform the most efficient blocking ones.