The Poisson approximation to the binomial distribution
The following approximation is a classical one.
We wish to show that for small p and sufficiently large n
P
(
X
=
k
)
=
C
(
n
,
k
)
p
k
(
1

p
)
n

k
≈
e

n
p
n
p
k
!
P
(
X
=
k
)
=
C
(
n
,
k
)
p
k
(
1

p
)
n

k
≈
e

n
p
n
p
k
!
(1)Suppose p=μ/np=μ/n with n large and μ/n<1μ/n<1. Then,
P
(
X
=
k
)
=
C
(
n
,
k
)
(
μ
/
n
)
k
(
1

μ
/
n
)
n

k
=
n
(
n

1
)
⋯
(
n

k
+
1
)
n
k
1

μ
n

k
1

μ
n
n
μ
k
k
!
P
(
X
=
k
)
=
C
(
n
,
k
)
(
μ
/
n
)
k
(
1

μ
/
n
)
n

k
=
n
(
n

1
)
⋯
(
n

k
+
1
)
n
k
1

μ
n

k
1

μ
n
n
μ
k
k
!
(2)The first factor in the last expression is the ratio of polynomials in n of the same degree k,
which must approach one as n becomes large. The second factor approaches one as n becomes large.
According to a well known property of the exponential
1

μ
n
n
→
e

μ
as
n
→
∞
1

μ
n
n
→
e

μ
as
n
→
∞
(3)The result is that for large n, P(X=k)≈eμμkk!P(X=k)≈eμμkk!,
where μ=npμ=np.
The Poisson and gamma distributions
Suppose Y∼Y∼ Poisson (λt)(λt).
Now X∼X∼ gamma (α,λ)(α,λ) iff
P
(
X
≤
t
)
=
λ
α
Γ
(
α
)
∫
0
t
x
α

1
e

λ
x
d
x
=
1
Γ
(
α
)
∫
0
t
(
λ
x
)
α

1
e

λ
x
d
(
λ
x
)
P
(
X
≤
t
)
=
λ
α
Γ
(
α
)
∫
0
t
x
α

1
e

λ
x
d
x
=
1
Γ
(
α
)
∫
0
t
(
λ
x
)
α

1
e

λ
x
d
(
λ
x
)
(4)
=
1
Γ
(
α
)
∫
0
λ
t
u
α

1
e

u
d
u
=
1
Γ
(
α
)
∫
0
λ
t
u
α

1
e

u
d
u
(5)A well known definite integral, obtained by integration by parts, is
∫
a
∞
t
n

1
e

t
d
t
=
Γ
(
n
)
e

a
∑
k
=
0
n

1
a
k
k
!
with
Γ
(
n
)
=
(
n

1
)
!
∫
a
∞
t
n

1
e

t
d
t
=
Γ
(
n
)
e

a
∑
k
=
0
n

1
a
k
k
!
with
Γ
(
n
)
=
(
n

1
)
!
(6)Noting that 1=eaea=ea∑k=0∞akk!1=eaea=ea∑k=0∞akk! we find
after some simple algebra that
1
Γ
(
n
)
∫
0
a
t
n

1
e

t
d
t
=
e

a
∑
k
=
n
∞
a
k
k
!
1
Γ
(
n
)
∫
0
a
t
n

1
e

t
d
t
=
e

a
∑
k
=
n
∞
a
k
k
!
(7)For a=λta=λt and α=nα=n, we have the following equality iff
X∼X∼ gamma (α,λ)(α,λ).
P
(
X
≤
t
)
=
1
Γ
(
n
)
∫
0
λ
t
u
n

1
d

u
d
u
=
e

λ
t
∑
k
=
n
∞
(
λ
t
)
k
k
!
P
(
X
≤
t
)
=
1
Γ
(
n
)
∫
0
λ
t
u
n

1
d

u
d
u
=
e

λ
t
∑
k
=
n
∞
(
λ
t
)
k
k
!
(8)Now
P
(
Y
≥
n
)
=
e

λ
t
∑
k
=
n
∞
(
λ
t
)
k
k
!
iff
Y
∼
Poisson
(
λ
t
)
P
(
Y
≥
n
)
=
e

λ
t
∑
k
=
n
∞
(
λ
t
)
k
k
!
iff
Y
∼
Poisson
(
λ
t
)
(9)The gaussian (normal) approximation
The central limit theorem, referred to in the discussion of the gaussian or normal distribution
above, suggests that the binomial and Poisson distributions should be approximated by the gaussian.
The number of successes in n trials has the binomial (n,p)(n,p) distribution. This random
variable may be expressed
X
=
∑
i
=
1
n
I
E
i
where
the
I
E
i
constitute
an
independent
class
X
=
∑
i
=
1
n
I
E
i
where
the
I
E
i
constitute
an
independent
class
(10)Since the mean value of X is npnp and the variance is npqnpq, the
distribution should be approximately N(np,npq)N(np,npq).
Use of the generating function shows that the sum of independent Poisson random
variables is Poisson. Now if X∼X∼ Poisson (μ)(μ), then X may be considered the sum of
n independent random variables, each Poisson (μ/n)(μ/n). Since the mean value and the variance
are both μ, it is reasonable to suppose that suppose that X is approximately N(μ,μ)N(μ,μ).
It is generally best to compare distribution functions. Since the binomial and Poisson
distributions are integervalued, it turns out that the best gaussian approximaton is obtained
by making a “continuity correction.” To get an approximation to a density for an integervalued
random variable, the probability at t=kt=k is represented by a rectangle of height
p_{k} and unit width, with k as the midpoint. Figure 1 shows a plot of the “density”
and the corresponding gaussian density for n=300n=300, p=0.1p=0.1. It is apparent that
the gaussian density is offset by approximately 1/2. To approximate the probability X≤kX≤k,
take the area under the curve from k+1/2k+1/2; this is called the continuity correction.
Use of mprocedures to compare
We have two mprocedures to make the comparisons. First, we consider approximation of the
Poisson (μ)(μ) distribution. The mprocedure poissapp calls for a value of μ,
selects a suitable range about k=μk=μ and plots the distribution function for the
Poisson distribution (stairs) and the normal (gaussian) distribution (dash dot) for
N(μ,μ)N(μ,μ). In addition, the continuity correction is applied to the gaussian distribution
at integer values (circles). Figure 3 shows plots for μ=10μ=10. It is clear that the
continuity correction provides a much better approximation. The plots in Figure 4 are for
μ=100μ=100. Here the continuity correction provides the better approximation, but not
by as much as for the smaller μ.
The mprocedure bincomp compares the binomial, gaussian, and Poisson distributions. It
calls for values of n and p, selects suitable k values, and plots the distribution function
for the binomial, a continuous approximation to the distribution function for the Poisson,
and continuity adjusted values of the gaussian distribution function at the integer values.
Figure 4 shows plots for n=1000n=1000, p=0.03p=0.03. The good agreement of all three distribution
functions is evident. Figure 5 shows plots for n=50n=50, p=0.6p=0.6. There is still good
agreement of the binomial and adjusted gaussian. However, the Poisson distribution does not
track very well. The difficulty, as we see in the unit Variance, is the difference in
variances—npqnpq for the binomial as compared with npnp for the Poisson.