Generating Random Data in Python (Guide)

The random Module

Probably the most widely known tool for generating random data in Python is its random module, which uses the Mersenne Twister PRNG algorithm as its core generator.

Earlier, you touched briefly on random.seed(), and now is a good time to see how it works. First, let’s build some random data without seeding. The random.random() function returns a random float in the interval [0.0, 1.0). The result will always be less than the right-hand endpoint (1.0). This is also known as a semi-open range:

>>> # Don't call `random.seed()` yet
>>> import random
>>> random.random()
0.35553263284394376
>>> random.random()
0.6101992345575074

If you run this code yourself, I’ll bet my life savings that the numbers returned on your machine will be different. The default when you don’t seed the generator is to use your current system time or a “randomness source” from your OS if one is available.

With random.seed(), you can make results reproducible, and the chain of calls after random.seed() will produce the same trail of data:

>>> random.seed(444)
>>> random.random()
0.3088946587429545
>>> random.random()
0.01323751590501987

>>> random.seed(444)  # Re-seed
>>> random.random()
0.3088946587429545
>>> random.random()
0.01323751590501987

Notice the repetition of “random” numbers. The sequence of random numbers becomes deterministic, or completely determined by the seed value, 444.

Let’s take a look at some more basic functionality of random. Above, you generated a random float. You can generate a random integer between two endpoints in Python with the random.randint() function. This spans the full [x, y] interval and may include both endpoints:

>>> random.randint(0, 10)
7
>>> random.randint(500, 50000)
18601

With random.randrange(), you can exclude the right-hand side of the interval, meaning the generated number always lies within [x, y) and will always be smaller than the right endpoint:

>>> random.randrange(1, 10)
5

If you need to generate random floats that lie within a specific [x, y] interval, you can use random.uniform(), which plucks from the continuous uniform distribution:

>>> random.uniform(20, 30)
27.42639687016509
>>> random.uniform(30, 40)
36.33865802745107

Each of these functions draws from a distribution with its own characteristic shape, which appears once you collect many samples into a histogram. Switch between them to compare the flat spread of random.random() with the bell of random.gauss() and the sharp decay of random.expovariate():

To pick a random element from a non-empty sequence (like a list or a tuple), you can use random.choice(). There is also random.choices() for choosing multiple elements from a sequence with replacement (duplicates are possible):

>>> items = ["one", "two", "three", "four", "five"]
>>> random.choice(items)
'four'

>>> random.choices(items, k=2)
['three', 'three']
>>> random.choices(items, k=3)
['three', 'five', 'four']

To mimic sampling without replacement, use random.sample():

>>> random.sample(items, 4)
['one', 'five', 'four', 'three']

You can randomize a sequence in-place using random.shuffle(). This will modify the sequence object and randomize the order of elements:

>>> random.shuffle(items)
>>> items
['four', 'three', 'two', 'one', 'five']

If you’d rather not mutate the original list, you’ll need to make a copy first and then shuffle the copy. You can create copies of Python lists with the copy module, or just x[:] or x.copy(), where x is the list.

Before moving on to generating random data with NumPy, let’s look at one more slightly involved application: generating a sequence of unique random strings of uniform length.

It can help to think about the design of the function first. You need to choose from a “pool” of characters such as letters, numbers, and/or punctuation, combine these into a single string, and then check that this string has not already been generated. A Python set works well for this type of membership testing:

import string

def unique_strings(k: int, ntokens: int,
               pool: str=string.ascii_letters) -> set:
    """Generate a set of unique string tokens.

    k: Length of each token
    ntokens: Number of tokens
    pool: Iterable of characters to choose from

    For a highly optimized version:
    https://stackoverflow.com/a/48421303/7954504
    """

    seen = set()

    # An optimization for tightly-bound loops:
    # Bind these methods outside of a loop
    join = "".join
    add = seen.add

    while len(seen) < ntokens:
        token = join(random.choices(pool, k=k))
        add(token)
    return seen

"".join() joins the letters from random.choices() into a single Python str of length k. This token is added to the set, which can’t contain duplicates, and the while loop executes until the set has the number of elements that you specify.

Let’s try this function out:

>>> unique_strings(k=4, ntokens=5)
{'AsMk', 'Cvmi', 'GIxv', 'HGsZ', 'eurU'}

>>> unique_strings(5, 4, string.printable)
{"'O*1!", '9Ien%', 'W=m7<', 'mUD|z'}

For a fine-tuned version of this function, this Stack Overflow answer uses generator functions, name binding, and some other advanced tricks to make a faster, cryptographically secure version of unique_strings() above.

PRNGs for Arrays: numpy.random

One thing you might have noticed is that a majority of the functions from random return a scalar value (a single int, float, or other object). If you wanted to generate a sequence of random numbers, one way to achieve that would be with a Python list comprehension:

>>> [random.random() for _ in range(5)]
[0.021655420657909374,
 0.4031628347066195,
 0.6609991871223335,
 0.5854998250783767,
 0.42886606317322706]

But there is another option that is specifically designed for this. You can think of NumPy’s own numpy.random package as being like the standard library’s random, but for NumPy arrays. (It also comes loaded with the ability to draw from a lot more statistical distributions.)

Take note that numpy.random uses its own PRNG that is separate from plain old random. You won’t produce deterministically random NumPy arrays with a call to Python’s own random.seed():

>>> import numpy as np
>>> np.random.seed(444)
>>> np.set_printoptions(precision=2)  # Output decimal fmt.

Without further ado, here are a few examples to whet your appetite:

>>> # Return samples from the standard normal distribution
>>> np.random.randn(5)
array([ 0.36,  0.38,  1.38,  1.18, -0.94])

>>> np.random.randn(3, 4)
array([[-1.14, -0.54, -0.55,  0.21],
       [ 0.21,  1.27, -0.81, -3.3 ],
       [-0.81, -0.36, -0.88,  0.15]])

>>> # `p` is the probability of choosing each element
>>> np.random.choice([0, 1], p=[0.6, 0.4], size=(5, 4))
array([[0, 0, 1, 0],
       [0, 1, 1, 1],
       [1, 1, 1, 0],
       [0, 0, 0, 1],
       [0, 1, 0, 1]])

In the syntax for randn(d0, d1, ..., dn), the parameters d0, d1, ..., dn are optional and indicate the shape of the final object. Here, np.random.randn(3, 4) creates a 2d array with 3 rows and 4 columns. The data will be i.i.d., meaning that each data point is drawn independent of the others.

Another common operation is to create a sequence of random Boolean values, True or False. One way to do this would be with np.random.choice([True, False]). However, it’s actually about 4x faster to choose from (0, 1) and then view-cast these integers to their corresponding Boolean values:

>>> # NumPy's `randint` is [inclusive, exclusive), unlike `random.randint()`
>>> np.random.randint(0, 2, size=25, dtype=np.uint8).view(bool)
array([ True, False,  True,  True, False,  True, False, False, False,
       False, False,  True,  True, False, False, False,  True, False,
        True, False,  True,  True,  True, False,  True])

What about generating correlated data? Let’s say you want to simulate two correlated time series. One way of going about this is with NumPy’s multivariate_normal() function, which takes a covariance matrix into account. In other words, to draw from a single normally distributed random variable, you need to specify its mean and variance (or standard deviation).

To sample from the multivariate normal distribution, you specify the means and covariance matrix, and you end up with multiple, correlated series of data that are each approximately normally distributed.

However, rather than covariance, correlation is a measure that is more familiar and intuitive to most. It’s the covariance normalized by the product of standard deviations, and so you can also define covariance in terms of correlation and standard deviation:

So, could you draw random samples from a multivariate normal distribution by specifying a correlation matrix and standard deviations? Yes, but you’ll need to get the above into matrix form first. Here, S is a vector of the standard deviations, P is their correlation matrix, and C is the resulting (square) covariance matrix:

This can be expressed in NumPy as follows:

def corr2cov(p: np.ndarray, s: np.ndarray) -> np.