1 / 37

Random Numbers

Random Numbers. All stochastic simulations need to “generate” IID U(0,1) “random numbers” Other random variates coming from other distribution can be generated using U(0,1) random numbers. Uniform Random Numbers. Uniform Random Numbers. Early Methods. Hamamaker’s Die

torrespatricia
Download Presentation

Random Numbers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Random Numbers • All stochastic simulations need to “generate” IID U(0,1) “random numbers” • Other random variates coming from other distribution can be generated using U(0,1) random numbers

  2. UniformRandomNumbers

  3. UniformRandomNumbers

  4. Early Methods Hamamaker’s Die Having no access to existing tables of random variables during world war II, the late well-known statistics professor H.C. Hamamaker at Philips Research Laboratories in Eindhoven, invented this die and the dice-throwing technique to produce random sampling numbers. The technique has been tested in various ways on series of up to 40,000 throws without showing evidence of bias. • Physical • Dice • Cards • Lotteries • Mechanical • Spinning disks • Electrical • Electronic Random Number Indicator Equipment (ERNIE, used by British Post Office to pick winners in a lottery (1959)) • RAND Corporation Tables (A million random digits (1955)) • Other • Pick digits randomly from phone directories • Decimals in expansion of π to 100,000 places

  5. AlgorithmicandSequential Computer Methods • Sequential means that the next “random” numberis determined by one or several of itspredecessors according to a fixed mathematical formula • The midsquare method: (von Neumann and Metropolis, 1945) • Start with Z0 = 4-digit positive integer • Z1 = middle 4 digits of Z02 (append 0s if necessary to left of Z02 to get exactly 8digits); U1 = Z1, with decimal point at left • Z2 = middle 4 digits of Z12; U2 = Z2, with decimal point at left • Z3 = middle 4 digits of Z22; U3 = Z3, with decimal point at left • So on ... • Problemsthe with midsquare method • Not really “random” since the entire sequence is determined by Z0 • If a Ziever reappears, the entire sequence will be recycled(This willoccur, since the only choices for 4-digit positive integers are 0000,0001, 0002, ..., 9999) • “Design” generators so Ui’s “appear” to be IID U(0,1) and cycle length is long

  6. The Midsquare Method

  7. Can We Generate “Truly” Random Numbers? • “True” randomness • Only possible with physical experiment having U(0,1)output • Counting gamma rays from space • Problems • Not reproducible • Impractical for computers • Practical approach: produce stream of numbers that appearto be IID U(0,1) • Use theoretical properties as far as possible • Perform empirical tests

  8. Criteriafor RandomNumber Generators • Appear to be distributed uniformly on [0, 1] and independent • Fast, low memory requirement • Should be able to reproduce a particular stream of random numbers • This makes debugging easier • One can use identical random numbers to simulate alternative system configurations for better comparison • Should have provision in the generator for a large number of separate (nonoverlapping) streamsof random numbers (usually such streams are just carefully chosen subsequences of the larger overall sequence) • Beware: There are many RNGs in use (and in software) that have extremely poor statistical properties

  9. Linear Congruential Generators • Specify four parameters (all nonnegative integers): Z0 = seed (or starting value) m = modulus (or divisor) a = multiplier c = increment • Then, Z1, Z2, Z3, ... are recursively generated by Zi= (aZi-1 + c) (mod m) • Ziis the remainder of dividing aZi-1 + c by m • Thus, 0 ≤ Zi ≤ m – 1, so the random number is defined as Ui = Zi/m • Note that 0 ≤ Ui <1 • c = 0 (Multiplicative LCG), c > 0 (Mixed LCG)

  10. Example of a “Toy” LCG • m = 16, a = 5, c = 3, Z0 = 7 • Formula • Zi = 5Zi-1 + 3 (mod 16) • Cycle length (or period) here is • 16 = m (the longest possible) • Questions: • Can the period be “predicted” in advance? • Can a full period be guaranteed?

  11. Objections to LCGs • Not really “random”. Indeed, an explicit formula for Ziis • Cycles when a previous Zireappears (there are only m choices, so cycle length is ≤ m) • Ui’s can only take the discrete values 0/m, 1/m, 2/m, ..., (m-1)/m , but they’re supposed to be continuous • For this reason, pick m ≥ 109 or more in practice

  12. Full-Period Theorem • (Hull and Dobell, 1966) The LCG Zi= (aZi-1 + c) (mod m) has full period (m) if and only ifall three ofthefollowing conditionshold: • Bothc and m are relatively prime (i.e., the only positive integer that dividesbothcand m is 1) • If q is any prime number that divides m, then q also divides a – 1 • If 4 divides m, then 4 also divides a – 1 • Choice of initial seed Z0is irrelevant • Checking these conditions for a “real” generator may be difficult • Condition 1 says that if c = 0, then full period is impossible (since m divides both mand c = 0); but m – 1 is possible • Theorem is about period only, it says nothing about uniformity of a subcycle, independence, or other statistical properties

  13. Desirable Properties of LCGs • Fast • Low storagerequirement • Reproducible (remember Z0) • Restart in middle (remember last Zi, use as Z0 next time) • Multiple streams (save separated seeds) • May have good statistical properties (depends on choice of parameters)

  14. Implementation/Portability Issues • LCGs (and other generators) generally deal with very large integers • Usually, m ≥ 2.1 billion ≈ 2.1 × 109 ≈ 231 • Take care in coding, especially in high level languages (FORTRAN, C) • To avoid need to store and operate on large integers: use sophisticated abstract-algebra tricks to “break up” the arithmetic on large integers, then reassemble

  15. Other Kinds of Generators More General Congruences Zi = g(Zi-1, Zi-2, ...) (mod m), Ui = Zi/m • LCG: g(Zi-1) = aZi-1 + c • Multiplicative recursive generator: g(Zi-1, Zi-2, ..., Zi-q) = a1Zi-1 + ... + aq Zi-q • Quadratic congruential generator: g(Zi-1) = a’Zi-12 + aZi-1 + c • Fibonacci (bad): g(Zi-1, Zi-2) = Zi-1 + Zi-2

  16. Composite Generators • Shuffling • Fill a vector of length 128 (say) from generator 1 • Use generator 2 to pick one of the 128 in the vector • Fill the hole with the next value from generator 1, use generator 2 to pick one ofthe 128 in the vector, etc. • Shuffling a bad generator improves it, but shuffling a good generatordoesn’t gain much • Differencing LCGs • Get Z1iand Z2ifrom LCGs with different moduli • Let Zi= (Z1i– Z2i) (mod m); Ui= Zi/ m • They have very long periods (like 1018) and very good statistical properties • Wichmann/Hill • Use three LCGs to get U1i, U2i, and U3isequences • Let Ui= fractional part of U1i+ U2i + U3i • They have long period and good statistical properties (equivalent to a LCG!)

  17. Testing RandomNumber Generators • Since RNGs are completely deterministic, we need to test them to see if they appear to be random and IID uniform on [0, 1] • General advice: be wary of “canned” RNGs in software that is not specifically simulation software, especially if they are not thoroughly documented; perhaps even test them before using • Two types of testing procedures • Empirical statistical tests • Chi-square • Kolmogorov-Smirnov • Serial test • Runs test • Correlation test • Theoretical procedures • Full period values • Lattice structure

  18. Empirical Statistical Tests • Use trial generator to generate some U’s and apply statistical tests • This gives information only on that part of a generator’s cycle examined (local) Chi-square, K-S tests for U(0,1) (all parameters known) Serial test: generalized chi-square test to higher dimensions • Two dimensions, subdivide unit square into subcells • Provides indirect test of independence

  19. Empirical Statistical Tests Correlation test: performcorrelationanalysisusingthecorrelationcoefficient of lagjto test H0: independence

  20. Checking the Excel RAND() generator Data Characteristic Value Source file <edited> Observation type Real valued Number of observations 100 Minimum observation 0.03580 Maximum observation 0.99638 Mean 0.50290 Median 0.50590 Variance 0.06893 Coefficient of variation 0.52207 Skewness 0.04632

  21. Checking the Excel RAND() generator

  22. Checking the Excel RAND() generator

  23. Checking the Excel RAND() generator

  24. Checking the Excel RAND() generator

  25. Checking the Excel RAND() generator

  26. Some General Guidelines • Beware of “canned” generators; especially in non-simulation software, and especially if it is poorly documented • Insist on documentation of the generator, check with “tried and true” ones • Don’t “seed” the generator with the square root of the clock or any other such scatterbrained scheme, we wantto get reproducibility of the random-number stream • Reasonably safe ideas • Use one of the generators from thebook • Look for highly portable generators • See if they provide default streams with controlled, known spacing • Take care to avoid overlapping the streams

More Related