CS433 Modeling and Simulation Lecture 04 Statistical Models

1. Al-Imam Mohammad Ibn Saud University CS433Modeling and SimulationLecture 04 Statistical Models http://10.2.230.10:4040/akoubaa/cs433/ Dr. Anis Koubâa 27 Oct 2008

2. Goals for Today • Understand the difference between discrete and continuous random variables • Review of the most common statistical models • Understand how to determine the empirical distribution from a statistical sample.

3. Topics • Discrete Random Variable • Continuous Random Variable • Discrete Probability Distributions • Binomial Distribution • Bernoulli Distribution • Discrete Poisson Distribution • Continuous Probability Distribution • Uniform • Exponential • Normal • Weibull • Lognormal • Empirical Distributions

4. Discrete and Continuous Random Variables

5. Discrete Random Variables • X is a discrete random variable if the number of possible values of X (the sample space) is finite. • Example: Consider jobs arriving at a job shop. • Let X be the number of jobs arriving each week at a job shop. • Rx= possible values of X (range space of X) = {0,1,2,…} • p(xi) = probability the random variable is xi = p(X = xi) • The collection of pairs [xi, p(xi)], i = 1,2,…, is called the probability distribution of X, • p(xi) is called the probability mass function (PMF) of X. • Characteristics of the PMF: p(xi), i = 1,2, …must satisfy:

6. Continuous Random Variables • X is a continuous random variable if its range space Rx is an interval or a collection of intervals. • The probability that X lies in the interval [a,b] is given by: Wheref(x) is the probability density function (PDF). • Characteristics of the PDF: f(x) must satisfies: • Properties

7. Discrete versus Continuous Random Variables Continuous Random Variable Discrete Random Variable Finite Sample Space e.g. {0, 1, 2, 3} Infinite Sample Space e.g. [0,1], [2.1, 5.3] Probability Mass Function (PMF) Probability Density Function (PDF) Cumulative Distribution Function (CDF)

8. Five Minutes Break You are free to discuss with your classmates about the previous slides, or to refresh a bit, or to ask questions. Administrative issues Groups Formation Choose a “class coordinator”

9. Expectation • The expected value (the mean) of X is denoted by E(X) • If X is discrete • If X is continuous • A measure of the central tendency • The variance of X is denoted by V(X) or var(X) or s2 • Definition: • Also, • A measure of the spread or variation of the possible values of X around the mean • The standard deviation of X is denoted by s • Definition: square root of V(X) • Expressed in the same units as the mean

10. Example: Continuous Random Variables Example: modeling the lifetime of a device • Time is a continuous random variable • Random Time is typically modeled as exponential distribution • We assume that with average lifetime of a device is 2 years • Probability that the device’s life is between 2 and 3 years is:

11. Example: Continuous Random Variables • Cumulative Distribution Function: A device has the CDF: • The probability that the device lasts for less than 2 years: • The probability that it lasts between 2 and 3 years:

12. Example: Continuous Random Variables Expected Value and Variance • Example: The mean of life of the previous device is: • To compute variance of X, we first compute E(X2): • Hence, the variance and standard deviation of the device’s life are:

13. Discrete Probability Distributions Bernoulli Trials Binomial Distribution Geometric Distribution Poisson Distribution Poisson Process

14. Discrete Distributions • Discrete random variables are used to describe random phenomena in which only integer values can occur. • In this section, we will learn about: • Bernoulli trials and Bernoulli distribution • Binomial distribution • Geometric and negative binomial distribution • Poisson distribution

15. Modeling of Random Events with Two-States Bernoulli Trials Binomial Distribution

16. Bernoulli Trials • In the theory of probability and statistics, a Bernoulli trial is an experiment whose outcome is random and can be either of two possible outcomes, "success" and "failure". • In practice it refers to a single experiment, which can have one of two possible outcomes. These events can be phrased into “yes” or “no” questions: • Did the coin land heads? • Was the newborn child or a girl? • Were a person's eyes green?

17. Bernoulli Distribution • Consider an experiment consisting of n trials, each can be a success or a failure. • Let Xj = 1 if the jthexperiment is a success • and Xj = 0 if the jth experiment is a failure • The Bernoulli distribution (one trial): • Bernoulli process • It is the n Bernoulli trials where trials are independent:

18. Binomial Distribution • A binomial random variable is the number of successes in a series of n trials. • Example: the number of 'heads' occurring when a coin is tossed 50 times. • A discrete random variable X is said to follow a Binomial distribution with parameters n and p, written X ~ Bi(n,p)or X ~ B(n,p)if it has the probability distribution: • where • x = 0, 1, 2, ......., n • n = 1, 2, 3, ....... • p = success probability; 0 < p < 1 Where

19. Binomial Distribution • The trials must meet the following requirements: • the total number of trials is fixed in advance; • there are just two outcomes of each trial; success and failure; • the outcomes of all the trials are statistically independent; • all the trials have the same probability of success.

20. Binomial Distribution • The number of successes in n Bernoulli trials, X, has a binomial distribution. • The formula can be understood as follows: we want k successes (pk) and n − kfailures (1 − p)n − k. However, the k successes can occur anywhere among the n trials, and there are C(n, k) different ways of distributing k successes in a sequence of n trials. The number of outcomes having the required number of successes and failures Probability that there are x successes and (n-x) failures

21. End of Part 01 Administrative issues Groups Formation Choose a “class coordinator”

22. Modeling of Discrete Random Time Geometric Distribution

23. Geometric Distribution • Geometric Distribution representsthe number X of Bernoulli trials to achieve the FIRST SUCCESS. • It is used to represent random time until a first transition occurs PMF k

24. Negative Binomial Distribution

25. Negative Binomial Distribution • The negative binomial distribution is a discrete probability distribution that can be used to describe the distribution arising from an experiment consisting of a sequence of independent trials, subject to several constraints. • Firstly each trial results in success or failure, the probability of success for each trial, p, is constant across the experiment and finally the experiment continues until a fixed number of successes have been achieved. • Negative Binomial Distribution • The number of Bernoulli trials, X, until the kth success • If X is a negative binomial distribution with parameters p and r, then:

26. Modeling of Random Number of Arrivals/Events Poisson Distribution Poisson Process

27. Poisson Distribution • the Poisson distribution is a discrete probability distribution that expresses the probability of a number of events occurring in a fixed period of time if these events occur with a known average rate and independently of the time since the last event. • Poisson random variable represents the count of the number of events that occur in a certain time interval or spatial area. • Example: • The number of cars passing a fixed point in a 5 minute interval, • The number of calls received by a switchboard during a given period of time. • The number of message coming to a router in a given period of time

28. Discrete Poisson Distribution • A discrete random variable X is said to follow a Poisson distribution with parameter l, written X ~ Po(l), if it has probability distribution • The PMF represents the probability that there are k arrivals in a certain period of time. • where • X = 0, 1, 2, ..., n • l > 0 is called the arrival rate.

29. Poisson distribution describes many random processes quite well and is mathematically quite simple. The Poisson distribution with the parameter l is characterized by: Discrete Poisson Distribution PMF CDF

30. Discrete Poisson Distribution • The following requirements must be met in the Poisson Distribution: • the length of the observation period is fixed in advance; • the events occur at a constant average rate; • the number of events occurring in disjoint intervals are statistically independent.

31. Example: Poisson Distribution • The number of cars that enter the parking follows a Poisson distribution with a mean rate equal to l = 20 cars/hour • The probability of having exactly15 cars entering the parking in one hour: or • The probability of having more than 3 cars entering the parking in one hour: USE EXCEL/MATLAB FOR COMPUTATIONS

32. Example: Poisson Distribution Probability Mass Function Poisson (l = 20 cars/hour) Cumulative Distribution Function Poisson (l = 20 cars/hour)

33. Five Minutes Break You are free to discuss with your classmates about the previous slides, or to refresh a bit, or to ask questions. Administrative issues Groups Formation

34. Modeling of Random Number of Arrivals/Events Poisson Distribution Poisson Process

35. Poisson Process • Wikipedia:A Poisson process, named after the French mathematician Siméon-Denis Poisson (1781 – 1840), is the stochastic process in which events (e.g. arrivals) occur continuously and independently of one another. • Formal Definition: The Poisson Process is a counting function{N(t), t≥0} where N(t) is the number of events that have occurred up to time t , i.e. in the interval [0, t]. • Fact: The number of events between time a and time b is given as N(b) − N(a) and has a Poisson distribution. • The Poisson process is a continuous-time process: Time is continuous • Its discrete-time counterpart is the Bernoulli process • Bernoulli process is a discrete-time stochastic process consisting of a sequence of independent random variables taking values over two symbols.

36. Examples of using Poisson Process • The number of web page requests arriving at a server may be characterized by a Poisson process except for unusual circumstances such as coordinated denial of service attacks. • The number of telephone calls arriving at a switchboard, or at an automatic phone-switching system, may be characterized by a Poisson process. • The number of raindrops falling over a wide spatial area may be characterized by a spatial Poisson process. • The arrival of "customers" is commonly modelled as a Poisson process in the study of simple queueing systems. • The execution of trades on a stock exchange, as viewed on a tick by tick basis, is a Poisson process.

37. (Homogenous) Poisson Process • The homogeneous Poisson process is characterized by a CONSTANTrate parameter λ, also known as intensity, such that the number of events in time interval follows a Poisson distribution with associated parameter . • Formally, A counting process is a (homogenous) Poisson process with mean rate l if: • describes the number of events in time interval • The mean and the variance are equal

38. (Homogenous) Poisson Process • Properties of Poisson process • Arrivals occur one at a time (not simultaneous) • has stationary increments, which means The number of arrivals in time s to t is also Poisson-distributed with mean • has independent increments

39. Inter-Arrival Times of a Poisson Process CDF of Exponential distribution • Inter-arrival time: time between two consecutive arrivals • The inter-arrival times of a Poisson process are random. What is its distribution? • Consider the inter-arrival times of a Poisson process (A1, A2, …), where Ai is the elapsed time between arrival i and arrival i+1 • The first arrival occurs after time tMEANS that there are no arrivals in the interval [0,t], As a consequence: The Inter-arrival times of a Poisson process are exponentially distributed and independent with mean 1/l

40. N1(t) ~ Poi[lp] lp l N(t) ~ Poi(l) N2(t) ~ Poi[l(1-p)] l(1-p) l1 N1(t) ~ Poi[l1] l1 + l2 N(t) ~ Poi(l1 + l2) N2(t) ~ Poi[l2] l2 Splitting and Pooling • Splitting • A Poisson process can be split into two Poisson processes: The first with a probability p andthe second with probability 1-p. • whereand are both Poisson processes with rates and • Pooling • The summation of two Poisson processes is a Poisson process • , where is a Poisson processes with rates

41. Modeling of Random Number of Arrivals/Events Poisson Distribution Non Homogenous Poisson Process

42. Non Homogenous (Non-stationary) Poisson Process (NSPP) • The non homogeneous Poisson process is characterized by a VARIABLErate parameter λ(t), the arrival rate at time t. In general, the rate parameter may change over time. • The stationary increments, property is not satisfied • The expected number of events (e.g. arrival) between timesand time t is l1 l2 l3

43. Example: Non-stationary Poisson Process (NSPP) • The number of cars that cross the intersection of King Fahd Road and Al-Ourouba Road is distributed according to a non homogenous Poisson process with a mean l(t) defined as follows: • Let us consider the time 8 am as t=0. • Q1. Compute the average arrival number of cars at 11H30? • Q2. Determine the equation that gives the probability of having only 10000 car arrivals between 12 pm and 16 pm. • Q3. What is the distribution and the average (in seconds) of the inter-arrival time of two cars between 8 am and 9 am?

44. Example: Non-stationary Poisson Process (NSPP) • Q1. Compute the average arrival number of cars at 11H30? • Q2. Determine the equation that gives the probability of having only 10000 car arrivals between 12 pm and 16 pm. We know that the number of cars between 12 pm and 16 pm, i.e. follows a Poisson distribution. During 12 pm and 16pm, the average number of cars is Thus, • Q3. What is the distribution and the average (in seconds) of the inter-arrival time of two cars between 8 am and 9 am? (Homework)

45. Two Minutes Break You are free to discuss with your classmates about the previous slides, or to refresh a bit, or to ask questions. Administrative issues Groups Formation

46. Continuous Probability Distributions Uniform Distribution Exponential Distribution Normal (Gaussian) Distribution Weibull Distribution Lognormal Distribution

47. Continuous Distributions • Continuous random variables can be used to describe random phenomena in which the variable can take on any value in some interval. • In this section, the distributions studied are: • Uniform • Exponential • Normal • Weibull • Lognormal

48. Uniform Distribution

49. Continuous Uniform Distribution • The continuous uniform distribution is a family of probability distributions such that for each member of the family, all intervals of the same length on the distribution's support are equally probable • A random variable X is uniformly distributed on the interval [a,b], U(a,b), if its PDF and CDF are:

50. Uniform Distribution PDF • Properties • is proportional to the length of the interval • Special case: a standard uniform distribution U(0,1). • Very useful for random number generators in simulators CDF