CPE 619 Introduction to Queuing Theory

1 / 54

# CPE 619 Introduction to Queuing Theory - PowerPoint PPT Presentation

CPE 619 Introduction to Queuing Theory. Aleksandar Milenković The LaCASA Laboratory Electrical and Computer Engineering Department The University of Alabama in Huntsville http://www.ece.uah.edu/~milenka http://www.ece.uah.edu/~lacasa. Overview. Queueing Notation Rules for All Queues

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'CPE 619 Introduction to Queuing Theory' - iola

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### CPE 619Introduction to Queuing Theory

Aleksandar Milenković

The LaCASA Laboratory

Electrical and Computer Engineering Department

The University of Alabama in Huntsville

http://www.ece.uah.edu/~milenka

http://www.ece.uah.edu/~lacasa

Overview
• Queueing Notation
• Rules for All Queues
• Little's Law
• Types of Stochastic Processes
Queueing Models: What Will You Learn?
• What are various types of queues?
• What is meant by an M/M/m/B/K queue?
• How to obtain response time, queue lengths, and server utilizations?
• How to represent a system using a network of several queues?
• How to analyze simple queueing networks?
• How to obtain bounds on the system performance using queueing models?
• How to obtain variance and other statistics on system performance?
• How to subdivide a large queueing network model and solve it?
Basic Components of a Queue

2. Service time

distribution

1. Arrival

process

6. Service

discipline

5. Customer Population

4. Waiting

positions

3. Number of

servers

Example: students at a typical computer terminal roomwith a number of terminals. If all terminals are busy,the arriving students wait in a queue.

Kendall Notation A/S/m/B/K/SD
• A: Arrival process
• S: Service time distribution
• m: Number of servers
• B: Number of buffers (system capacity)
• K: Population size, and
• SD: Service discipline
Arrival Process
• Arrival times:
• Interarrival times:
• tj form a sequence of Independent and Identically Distributed (IID) random variables
• The most common arrival process: Poisson arrivals
• Inter-arrival times are exponential + IID  Poisson arrivals
• Notation:
• M = Memoryless = Poisson
• E = Erlang
• H = Hyper-exponential
• G = General  Results valid for all distributions
Service Time Distribution
• Time each student spends at the terminal
• Service times are IID
• Distribution: M, E, H, or G
• Device = Service center = Queue
• Buffer = Waiting positions
Service Disciplines
• First-Come-First-Served (FCFS)
• Last-Come-First-Served (LCFS)
• Last-Come-First-Served with Preempt and Resume (LCFS-PR)
• Round-Robin (RR) with a fixed quantum.
• Small Quantum  Processor Sharing (PS)
• Infinite Server: (IS) = fixed delay
• Shortest Processing Time first (SPT)
• Shortest Remaining Processing Time first (SRPT)
• Shortest Expected Processing Time first (SEPT)
• Shortest Expected Remaining Processing Time first (SERPT).
• Biggest-In-First-Served (BIFS)
• Loudest-Voice-First-Served (LVFS)
Common Distributions
• M: Exponential
• Ek: Erlang with parameter k
• Hk: Hyper-exponential with parameter k
• D: Deterministic  constant
• G: General  All
• Memoryless:
• Expected time to the next arrival is always 1/l regardless of the time since the last arrival
• Remembering the past history does not help
Example: M/M/3/20/1500/FCFS
• Time between successive arrivals is exponentially distributed
• Service times are exponentially distributed
• Three servers
• 20 Buffers = 3 service + 17 waiting
• After 20, all arriving jobs are lost
• Total of 1500 jobs that can be serviced
• Service discipline is first-come-first-served
• Defaults:
• (Only the first 3 parameters are sufficient to indicate the type)
• Infinite buffer capacity
• Infinite population size
• FCFS service discipline
• G/G/1 = G/G/1/1/1/FCFS
Group Arrivals/Service
• Bulk arrivals/service
• M[x]:x represents the group size
• G[x]: a bulk arrival or service process with general inter-group times
• Examples:
• M[x]/M/1 : Single server queue with bulk Poisson arrivals and exponential service times
• M/G[x]/m: Poisson arrival process, bulk service with general service time distribution, and m servers

1

2

l

m

m

ns

nq

n

PreviousArrival

BeginService

EndService

Arrival

Time

t

w

s

r

Key Variables
Key Variables (cont’d)
• t = Inter-arrival time = time between two successive arrivals
• l = Mean arrival rate = 1/E[t]May be a function of the state of the system, e.g., number of jobs already in the system
• s = Service time per job
• m = Mean service rate per server = 1/E[s]
• Total service rate for m servers is mm
• n = Number of jobs in the system. This is also called queue length
• Note: Queue length includes jobs currently receiving service as well as those waiting in the queue
Key Variables (cont’d)
• nq = Number of jobs waiting
• ns = Number of jobs receiving service
• r = Response time or the time in the system = time waiting + time receiving service
• w = Waiting time = Time between arrival and beginning of service
Rules for All Queues

Rules: The following apply to G/G/m queues

1. Stability Condition:l < mmFinite-population and the finite-buffer systems are always stable

2. Number in System versus Number in Queue:n = nq+ nsNotice that n, nq, and ns are random variables. E[n]=E[nq]+E[ns]If the service rate is independent of the number in the queue,

Cov(nq,ns) = 0

Rules for All Queues (cont’d)

3. Number versus Time: If jobs are not lost due to insufficient buffers, Mean number of jobs in the system = Arrival rate  Mean response time

4. Similarly, Mean number of jobs in the queue = Arrival rate  Mean waiting time

This is known as Little's law.

5. Time in System versus Time in Queuer = w + sr, w, and s are random variables

E[r] = E[w] + E[s]

Rules for All Queues (cont’d)

6. If the service rate is independent

of the number of jobs in the queue, Cov(w,s)=0

BlackBox

Departures

Arrivals

Little's Law
• Mean number in the system = Arrival rate  Mean response time
• This relationship applies to all systems or parts of systems in which the number of jobs entering the system is equal to those completing service
• Named after Little (1961)
• Based on a black-box view of the system:
• In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positions

4

Arrival

Jobnumber

3

2

Departure

1

1

2

3

4

5

6

7

8

4

TimeinSystem

3

2

1

1

2

3

Job number

Proof of Little's Law

4

• If T is large, arrivals = departures = N
• Arrival rate = Total arrivals/Total time= N/T
• Hatched areas = total time spent inside the system by all jobs = J
• Mean time in the system= J/N
• Mean Number in the system = J/T = (N/T) (J/N)= Arrival rate  Mean time in the system

NumberinSystem

3

2

1

1

2

3

4

5

6

7

8

Time

Time

Application of Little's Law
• Applying to just the waiting facility of a service center
• Mean number in the queue = Arrival rate  Mean waiting time
• Similarly, for those currently receiving the service, we have:
• Mean number in service = Arrival rate  Mean service time

Arrivals

Departures

Example 30.3
• A monitor on a disk server showed that the average time to satisfy an I/O request was 100 milliseconds. The I/O rate was about 100 requests per second. What was the mean number of requests at the disk server?
• Using Little's law:

Mean number in the disk server

= Arrival rate  Response time

= 100 (requests/second) (0.1 seconds)

= 10 requests

Stochastic Processes
• Process: Function of time
• Stochastic Process: Random variables, which are functions of time
• Example 1:
• n(t) = number of jobs at the CPU of a computer system
• Take several identical systems and observe n(t)
• The number n(t) is a random variable
• Can find the probability distribution functions for n(t) at each possible value of t
• Example 2:
• w(t) = waiting time in a queue
Types of Stochastic Processes
• Discrete or Continuous State Processes
• Markov Processes
• Birth-death Processes
• Poisson Processes
Discrete/Continuous State Processes
• Discrete = Finite or Countable
• Number of jobs in a system n(t) = 0, 1, 2, ....
• n(t) is a discrete state process
• The waiting time w(t) is a continuous state process
• Stochastic Chain: discrete state stochastic process
Markov Processes
• Future states are independent of the past and depend only on the present
• Named after A. A. Markov who defined and analyzed them in 1907
• Markov Chain: discrete state Markov process
• Markov  It is not necessary to know how long the process has been in the current state  State time has a memoryless (exponential) distribution
• M/M/m queues can be modeled using Markov processes
• The time spent by a job in such a queue is a Markov process and the number of jobs in the queue is a Markov chain

l0

l1

l2

lj-1

lj

lj+1

0

1

2

j-1

j

j+1

m1

m2

m3

mj

j+1

j+2

Birth-Death Processes
• The discrete space Markov processes in which the transitions are restricted to neighboring states
• Process in state n can change only to state n+1 or n-1.
• Example: the number of jobs in a queue with a single server and individual arrivals (not bulk arrivals)
Poisson Processes
• Interarrival time s = IID and exponential  number of arrivals n over a given interval (t, t+x)has a Poisson distribution  arrival = Poisson process or Poisson stream
• Properties:
• 1.Merging:
• 2.Splitting: If the probability of a job going to ith substream is pi, each substream is also Poisson with a mean rate of pil
Poisson Processes (cont’d)
• 3.If the arrivals to a single server with exponential service time are Poisson with mean rate l, the departures are also Poisson with the same rate l provided l < m
Poisson Process (cont’d)
• 4. If the arrivals to a service facility with m service centers are Poisson with a mean rate l, the departures also constitute a Poisson stream with the same rate l, provided l < åimi. Here, the servers are assumed to have exponentially distributed service times.
Summary
• Kendall Notation: A/S/m/B/k/SD, M/M/1
• Number in system, queue, waiting, serviceService rate, arrival rate, response time, waiting time, service time
• Little’s Law: Mean number in system = Arrival rate X Mean time spent in the system
• Processes: Markov  Memoryless, Birth-death  Adjacent states Poisson  IID and exponential inter-arrival

### Analysis of A Single Queue

Overview
• Birth Death Processes
• M/M/1 Queue
• M/M/m Queue
• M/M/m/B Queue with Finite Buffers
• Results for other Queueing systems

l0

l1

l2

lj-1

lj

lj+1

0

1

2

j-1

j

j+1

j+2

m1

m2

m3

mj

j+1

Birth-Death Processes
• Jobs arrive one at a time (and not as a batch)
• State = Number of jobs n in the system
• Arrival of a new job changes the state to n+1 birth
• Departure of a job changes the system state to n-1 death
• State-transition diagram:
Birth-Death Processes (cont’d)
• When the system is in state n, it has n jobs in it.
• The new arrivals take place at a rate ln
• The service rate is mn
• We assume that both the inter-arrival times and service times are exponentially distributed
Theorem: State Probability
• The steady-state probability pn of a birth-death process being in state n is given by:
• Here, p0 is the probability of being in the zero state
Proof
• Suppose the system is in state j at time t. There are j jobs in the system. In the next time interval of a very small duration Dt, the system can move to state j-1 or j+1 with the following probabilities:
Proof (cont’d)
• If there are no arrivals or departures, the system will stay in state j and, thus:
• Dt = small  zero probability of two events (two arrivals, two departure, or a arrival and a departure) occurring during this interval
• pj(t) = probability of being in state j at time t
Proof (cont’d)
• The jth equation above can be written as follows:
• Under steady state, pj(t) approaches a fixed value pj, that is:
Proof (cont’d)
• Substituting these in the jth equation, we get:
• The solution to this set of equations is:
Proof (cont’d)
• The sum of all probabilities must be equal to one:

l

l

l

l

l

l

0

1

2

J-1

J

J+1

m

m

m

m

M/M/1 Queue
• M/M/1 queue is the most commonly used type of queue
• Used to model single processor systems or to model individual devices in a computer system
• Assumes that the interarrival times and the service times are exponentially distributed and there is only one server
• No buffer or population size limitations and the service discipline is FCFS
• Need to know only the mean arrival rate l and the mean service rate m
• State = number of jobs in the system
Results for M/M/1 Queue
• Birth-death processes with
• Probability of n jobs in the system:
Results for M/M/1 Queue (cont’d)
• The quantity l/m is called traffic intensity and is usually denoted by symbol r. Thus:
• n is geometrically distributed. Utilization of the server = Probability of having one or more jobs in the system:
Results for M/M/1 Queue (cont’d)
• Mean number of jobs in the system:
• Variance of the number of jobs in the system:
Results for M/M/1 Queue (cont’d)
• Probability of n or more jobs in the system:
• Mean response time (using the Little's law):Mean number in the system = Arrival rate ×Mean response timeThat is:
Results for M/M/1 Queue (cont’d)
• Cumulative distribution function of the response time:
• The response time is exponentially distributed. q-percentile of the response time
Results for M/M/1 Queue (cont’d)
• Cumulative distribution function of the waiting time:
• This is a truncated exponential distribution. Its q-percentile is given by:
• The above formula applies only if q is greater than 100(1-r). All lower percentiles are zero.
Results for M/M/1 Queue (cont’d)
• Mean number of jobs in the queue:
• Idle  there are no jobs in the system
• The time interval between two successive idle intervals
• All results for M/M/1 queues including some for the busy period are summarized in Box 31.1 in the book.
Example 31.2
• On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about two milliseconds to forward them. Using an M/M/1 model, analyze the gateway. What is the probability of buffer overflow if the gateway had only 13 buffers? How many buffers do we need to keep packet loss below one packet per million?
• Arrival rate l = 125 pps
• Service rate m = 1/.002 = 500 pps
• Gateway Utilization r = l/m = 0.25
• Probability of n packets in the gateway = (1-r)rn = 0.75(0.25)n)
Example 31.2 (cont’d)
• Mean Number of packets in the gateway = (r/(1-r) = 0.25/0.75 = 0.33)
• Mean time spent in the gateway = ((1/m)/(1-r)= (1/500)/(1-0.25) = 2.66 milliseconds
• Probability of buffer overflow
• To limit the probability of loss to less than 10-6: We need about ten buffers.
Example 31.2 (cont’d)
• The last two results about buffer overflow are approximate. Strictly speaking, the gateway should actually be modeled as a finite buffer M/M/1/B queue. However, since the utilization is low and the number of buffers is far above the mean queue length, the results obtained are a close approximation.
Summary
• Birth-death processes: Compute probability of having n jobs in the system
• M/M/1 Queue: Load-independent => Arrivals and service do not depend upon the number in the system ln= l, mn=m
• Traffic Intensity: r = l/ m
• Mean Number of Jobs in the system = r/(1-r)
• Mean Response Time = (1/m)/(1-r)