1 / 30

Parallel Algorithms for Pseudorandom Number Generators

Parallel Algorithms for Pseudorandom Number Generators. CS 387 Se Jun Kim Todd Hartman. Application: Simulation. http://tugtugbug.deviantart.com/art/Epic-Minecraft-Scenery-1680x1050-348488303. Application: Sampling. http://en.wikipedia.org/wiki/File:Pi_30K.gif. Application: Cryptography.

shayna
Download Presentation

Parallel Algorithms for Pseudorandom Number Generators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Algorithms for Pseudorandom Number Generators CS 387 Se Jun Kim Todd Hartman

  2. Application: Simulation http://tugtugbug.deviantart.com/art/Epic-Minecraft-Scenery-1680x1050-348488303

  3. Application: Sampling http://en.wikipedia.org/wiki/File:Pi_30K.gif

  4. Application: Cryptography http://en.wikipedia.org/wiki/File:Tor-logo-2011-flat.svg

  5. Pseudorandom Numbers What makes a good PRN sequence?

  6. Randomness: Predictability Indistinguishable from Random Discernible Pattern http://www.random.org/analysis/

  7. Randomness: Uniform Distribution http://mathworld.wolfram.com/DiskPointPicking.html

  8. Cryptographically Secure PRNG • Stricter requirements • Next-bit Test • State Compromise Extensions

  9. Linear Congruence Generator • Simple and Fast • Limitations • Period • Distribution http://en.wikipedia.org/wiki/File:Lcg_3d.gif

  10. What is CUDA? • Compute Unified Device Architecture • Parallel Computing on GPUs

  11. Motivation Why PRNG on GPUs? • Use GPU for Monte Carlo integration • Ideal for GPGPU: compute a lot, output a little • Mean, median; uncertainty • Small computation on each thread • RNG on CPU: heavy computation

  12. Constraints • Linear RNGs? • Each thread outputs 1 block from long sequence • Long-range correlation  Easy to get burned!

  13. (Combined) Multiple Recursive Generator • Longer periods than LCG • Better uniformity

  14. L’Ecuyer MRG: MRG32k3a

  15. MRG32k3a: Parallelization • Reproducibility & Skip-Ahead • 3-term Recurrence

  16. MRG32k3a: Parallelization • Matrix multiplication optimization

  17. Parallel MRG32k3a: Time Complexity

  18. MRG32k3a: Performance

  19. Mersenne Twister What is it? • A PRNG developed by Matsumoto & Nishimura in 1997 • Produces uniform distribution in [0, 2k-1] Properties • Long Period : 219937-1 • Efficient use of Memory • Good Distribution Properties • High Performance

  20. Mersenne Twister • Based on the recurrence… • 11 parameters ..

  21. Mersenne Twister • Matrix A (w*w) // w = word size • For simple computation.. where

  22. Mersenne Twister • Step 0 • Create a mask for the upper bits and another one for the lower bits. • Step 1 • The x array is initialized with a seed • Step 2 • y is the upper bits of x[i] concatenated with the lower bits of x[i+1].

  23. Mersenne Twister • Step 3 • y * A. • A is chosen carefully so it can be easily calculated with right shift and xor. • Step 4 • Multiply with T (tempering matrix) for better equidistribution and good acurracy. • Steps 5 & 6 • Increment i by 1 and repeat the process

  24. MT19937 on GPUs Why on GPUs? • Uses bitwise arithmetic • Arbitrary amount of memory writes But… • PRNGs are iterative Hard to parallelize single twister state update • GPU needs many(>1000) threads to be fully utilized Simultaneous Mersenne twister processed in Parallel

  25. MT19937 on GPUs Another problem… • Correlated sequences can be generated • Even with “very different” initial state values  Each thread has its own parameters (like “seed)

  26. MT19937: Performance Comparison

  27. Sources • L. Bassham, “A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications “, National Institute of Standards and Technology, (2010). <http://csrc.nist.gov/groups/ST/toolkit/rng/documents/SP800-22rev1a.pdf> • Wikipedia,< http://en.wikipedia.org> • P. L'Ecuyer, “Combined Multiple Recursive Generators”, Operations Research, 44, 5 (1996), 816--822. <http://www.iro.umontreal.ca/~lecuyer/myftp/papers/combmrg.ps> • P. L'Ecuyer, ``Good Parameter Sets for Combined Multiple Recursive Random Number Generators'', Shorter version in Operations Research, 47, 1 (1999), 159--164. <http://www.iro.umontreal.ca/~lecuyer/myftp/papers/combmrg2.ps> • A. Freeth, "Pseudo-Random Number Generators for Massively Parallel Discrete-Event Simulation", University of Canterbury, (2011). http://www.cosc.canterbury.ac.nz/research/reports/TechReps/2012/tr_1201.pdf • T. Bradley et al, "Parallelisation Techniques for Random Number Generators", (2011). <http://www.nag.com/IndustryArticles/gpu_gems_article.pdf> • Matsumoto, Makoto, and Takuji Nishimura. "Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator." ACM Transactions on Modeling and Computer Simulation (TOMACS) 8.1 (1998): 3-30. • Podlozhnyuk, Victor. "Parallel Mersenne Twister." NVIDIA white paper(2007).

  28. Resources

  29. MRG32k3a: Performance

  30. MRG32k3a: Performance

More Related