Definition – Poisson distribution

Contents
1. Definitions
2. Derivation of the binomial distribution
3. Use of the binomial distribution
3. Mean.
4. Variance.

1. Definition

Let $X_i$ be a random variable with domain $\mathbb{N}$ . If

$p(X=x) = \frac{e^{-\lambda} \lambda^x}{x!}$

Where $e$ is Euler’s constant. We write that $X \sim Poisson(\lambda)$ .

2. Derivation of the Poisson distribution

The Poisson distribution is the limit of the binomial distribution as $n\rightarrow \infty$ . We will derive the pdf by taking this limit, and subsequently check that the conditions of a valid probability distribution are met after taking the limit. The reason we must do this is that when taking the limit (for example, $n\rightarrow\infty$ ), $n$ never equals infinity (clearly), and so we must check that the conditions are still valid after assuming $n = \infty$ (in very loose terms).

To derive the pdf of the Poisson distribution, we consider the limit,

$\lim_{n\rightarrow\infty} \binom{n}{x}p^x (1-p)^{n-x} = \lim_{n\rightarrow\infty} \frac{n!}{x!(n-x)!}p^x (1-p)^{n-x}\quad\text{(1)}$

Firstly, let $\lambda = np \implies n = \frac{\lambda}{p}$ , $p\neq 0$ . Thus, we can re-write $\text{(1)}$ as

$\lim_{n\rightarrow\infty} \frac{n!}{x!(n-x)!} (\frac{\lambda}{p})^x (1-\frac{\lambda}{p})^{n-x} \\\\ \Leftrightarrow \lim_{n\rightarrow\infty} \frac{\lambda^x}{x!}\frac{n!}{(n-x)! n^x} (1-\frac{\lambda}{n})^n (1-\frac{\lambda}{n})^{-x} \\\\ \Leftrightarrow (\lim_{n\rightarrow\infty} \frac{\lambda^x}{x!}) (\lim_{n\rightarrow\infty}\frac{n!}{(n-x)! n^x}) (\lim_{n\rightarrow\infty}(1-\frac{\lambda}{n})^n) \\ (\lim_{n\rightarrow\infty}(1-\frac{\lambda}{n})^{-x}) \quad\text{(2)}$

We now evaluate each of these limits, and return to their product to determine the pdf.

$\lim_{n\rightarrow\infty} \frac{\lambda^x}{x!} = \frac{\lambda^x}{x!}$ as none of the terms depend on $n$ .
For $\lim_{n\rightarrow\infty}\frac{n!}{(n-x)! n^x}$ , note that

$\lim_{n\rightarrow\infty} \frac{n!}{(n-x)!n^x} = \lim_{n\rightarrow\infty} \frac{n}{n}\times\lim_{n\rightarrow\infty}\frac{n-1}{n}\times...\lim_{n\rightarrow\infty}\times\frac{n-k+1}{n}$

But for each $lim_{n\rightarrow\infty} \frac{n-i}{n}$ , we have by L’Hospital’s rule,

$lim_{n\rightarrow\infty} \frac{n-i}{n} = lim_{n\rightarrow\infty} \frac{1}{1} = 1$

Thus,

$\lim_{n\rightarrow\infty} \frac{n!}{(n-x)!n^x} = \lim_{n\rightarrow\infty} \frac{n}{n}\times\lim_{n\rightarrow\infty}\frac{n-1}{n}\times...\lim_{n\rightarrow\infty}\times\frac{n-k+1}{n} = 1^{k-1} = 1$

$\lim_{n\rightarrow\infty} (1-\frac{\lambda}{n})^n = e^{-\lambda}$ based on the definition of Euler’s constant and a parameterisation $\frac{1}{n} = \frac{-\lambda}{t}$ .
$\lim_{n\rightarrow\infty} (1-\frac{\lambda}{n})^{-x} = 1$

And returning back to the product of limits in $\text{(2)}$ , we arrive at

$\lim_{n\rightarrow\infty} \binom{n}{x}p^x (1-p)^{n-x} = \frac{\lambda^x}{x!}\times 1\times e^{-\lambda} \times 1 = \frac{\lambda^x e^{-\lambda}}{x!}$

3. Use of the Poisson distribution

We use the Poisson distribution when we want to determine the probability of an event $P$ occurring $x\in\mathbb{N}$ times, where each occurrence is independent from all the others, and $P$ occurs at a rate of $\lambda$ occurrences per unit of time.

4. Mean & Variance

$E(X) = Var(X) = \lambda$

Proof

To prove the mean and variance of the Poisson distribution we will use its moment generating function. The moment generating function for the Poisson distribution is,

$\begin{array}{ccc} M(t) = E(e^{tx}) &=& \sum_{x=0}^\infty e^{tx}p(x) \\\\ &=& \sum_{x=0}^\infty e^{tx} \cdot \frac{e^{-\lambda} \lambda^x}{x!} \\\\ &=&e^{\lambda (e^t -1)} \end{array}$

Differentiating once gives

$M'(t) = \lambda e^t e^{\lambda (e^t -1)}$

And twice,

$M''(t) = \lambda e^t e^{\lambda (e^t -1)}+\lambda^2 e^{2t} e^{\lambda (e^t -1)}$

And at $t=0$ ,

$M'(0) = E(X) = \lambda$

$M''(0) = E(X^2) = \lambda + \lambda^2$

So, we have that $Var(X) = E(X^2) - E(X)^2 = (\lambda + \lambda^2) - \lambda^2 = \lambda$ , which completes the proof. $\square$