Basic statistics¶

This notebook is an element of the free risk-engineering.org courseware. It can be distributed under the terms of the Creative Commons Attribution-ShareAlike licence.

Author: Eric Marsden eric.marsden@risk-engineering.org.

This notebook contains an introduction to use of Python and the NumPy library for basic statistical calculations. It uses Python3.

We start by importing the numpy library, which makes it possible to use functions and variables from the library, prefixed by numpy..

In :
import numpy


We can use Python as simple interactive calculator:

In :
2 + 3 + 4

Out:
9

Here we call the sqrt function from the numpy library.

In :
numpy.sqrt(2 + 2)

Out:
2.0

Some useful constants are predefined.

In :
numpy.pi

Out:
3.141592653589793
In :
numpy.sin(numpy.pi)

Out:
1.2246467991473532e-16

The notation e-16 above means $10^{-16}$; the number above is very very small (it’s a numerical approximation to the mathematical answer of zero).

We can generate a random number from a uniform distribution between 20 and 30. If you evaluate this several times (in most Jupyter interfaces, press Shift-Enter or press on the Run button in the toolbar above), it will generate a different random number each time.

In :
numpy.random.uniform(20, 30)

Out:
23.825535015986542
In :
numpy.random.uniform(20, 30)

Out:
21.461777665455113

We can generate an array of random numbers by passing a third argument to the numpy.random.uniform function, saying how many random numbers we want. We store the array in a variable named obs.

In :
obs = numpy.random.uniform(20, 30, 10)
obs

Out:
array([20.74435904, 27.52127233, 21.26729415, 23.91754033, 24.06014691,
29.71653659, 28.15372604, 28.99197159, 24.09842574, 25.57165679])

The builtin function len in Python tells us the length of an array or a list.

In :
len(obs)

Out:
10

We can do arithmetic on arrays, adding them together or subtracting a constant from each element.

In :
obs + obs

Out:
array([41.48871807, 55.04254467, 42.5345883 , 47.83508067, 48.12029382,
59.43307317, 56.30745209, 57.98394319, 48.19685148, 51.14331359])
In :
obs - 25

Out:
array([-4.25564096,  2.52127233, -3.73270585, -1.08245967, -0.93985309,
4.71653659,  3.15372604,  3.99197159, -0.90157426,  0.57165679])

The array has methods, a kind of function that acts on the array.

In :
obs.mean()

Out:
25.404292952324695
In :
obs.sum()

Out:
254.04292952324695
In :
obs.min()

Out:
20.744359035357306

There are similar functions in the numpy library that take an array as argument:

In :
numpy.mean(obs)

Out:
25.404292952324695
In :
numpy.sum(obs)

Out:
254.04292952324695
In :
numpy.min(obs)

Out:
20.744359035357306

Simple plotting¶

The matplotlib library allows you to generate many types of plots and statistical graphs in a convenient way. The online gallery shows the variety of plots available, and the documentation is also available online. We import the pyplot component of matplotlib and give it an alias plt.

In :
import matplotlib.pyplot as plt
plt.style.use('bmh')
%config InlineBackend.figure_formats=['svg']

In :
X = numpy.random.uniform(20, 30, 10)
Y = numpy.random.uniform(50, 100, 10)
plt.scatter(X, Y);

In :
x = numpy.linspace(-2, 10, 100)
plt.plot(x, numpy.sin(x));


We can add two vectors together, assuming that all their dimensions are identical. Our array $x$ has one dimension of size 100. We can add another random vector of size 100 to it, containing numbers drawn from a uniform probability distribution between -0.1 and 0.1 (these represent some random “noise” which is added to our sine curve).

In :
x = numpy.linspace(0, 10, 100)
obs = numpy.sin(x) + numpy.random.uniform(-0.1, 0.1, 100)
plt.plot(x, obs);


The central limit theorem states that the sum of a number of independent random variables tends toward a normal distribution even if the original variables themselves are not normally distributed. We illustrate this result by examining the distribution of the sums of 1000 realizations of a uniformly distributed random variable, plotting the distribution as a histogram.

In :
N = 10000
sim = numpy.zeros(N)
for i in range(N):
sim[i] = numpy.random.uniform(30, 40, 1000).mean()
plt.hist(sim, bins=20, alpha=0.5, density=True);

In [ ]: