Skip to content Skip to navigation Skip to collection information

OpenStax-CNX

You are here: Home » Content » Applied Probability » Distribution Approximations

Navigation

Table of Contents

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.
  • Rice Digital Scholarship

    This collection is included in aLens by: Digital Scholarship at Rice University

    Click the "Rice Digital Scholarship" link to see all content affiliated with them.

Also in these lenses

  • UniqU content

    This collection is included inLens: UniqU's lens
    By: UniqU, LLC

    Click the "UniqU content" link to see all content selected in this lens.

Recently Viewed

This feature requires Javascript to be enabled.
 

Distribution Approximations

Module by: Paul E Pfeiffer. E-mail the author

Summary: Various approximations for distributions are studied, especially those involving the Binomial, Poisson, gamma, and Gaussian (normal) distributions. m-procedures are used to make comparisons. A simple approximation to a continuous random variable is obtained by subdividing an interval which includes the range (the set of possible values) into small enough subintervals that the density is approximately constant over each subinterval. A point in each subinterval is selected and is assigned the probability mass in its subinterval. The combination of the selected points and the corresponding probabilities describes the distribution of an approximating simple random variable. Calculations based on this distribution approximate corresponding calculations on the continuous distribution.

Binomial, Poisson, gamma, and Gaussian distributions

The Poisson approximation to the binomial distribution

The following approximation is a classical one. We wish to show that for small p and sufficiently large n

P ( X = k ) = C ( n , k ) p k ( 1 - p ) n - k e - n p n p k ! P ( X = k ) = C ( n , k ) p k ( 1 - p ) n - k e - n p n p k !
(1)

Suppose p=μ/np=μ/n with n large and μ/n<1μ/n<1. Then,

P ( X = k ) = C ( n , k ) ( μ / n ) k ( 1 - μ / n ) n - k = n ( n - 1 ) ( n - k + 1 ) n k 1 - μ n - k 1 - μ n n μ k k ! P ( X = k ) = C ( n , k ) ( μ / n ) k ( 1 - μ / n ) n - k = n ( n - 1 ) ( n - k + 1 ) n k 1 - μ n - k 1 - μ n n μ k k !
(2)

The first factor in the last expression is the ratio of polynomials in n of the same degree k, which must approach one as n becomes large. The second factor approaches one as n becomes large. According to a well known property of the exponential

1 - μ n n e - μ as n 1 - μ n n e - μ as n
(3)

The result is that for large n, P(X=k)e-μμkk!P(X=k)e-μμkk!, where μ=npμ=np.

The Poisson and gamma distributions

Suppose YY Poisson (λt)(λt). Now XX gamma (α,λ)(α,λ) iff

P ( X t ) = λ α Γ ( α ) 0 t x α - 1 e - λ x d x = 1 Γ ( α ) 0 t ( λ x ) α - 1 e - λ x d ( λ x ) P ( X t ) = λ α Γ ( α ) 0 t x α - 1 e - λ x d x = 1 Γ ( α ) 0 t ( λ x ) α - 1 e - λ x d ( λ x )
(4)
= 1 Γ ( α ) 0 λ t u α - 1 e - u d u = 1 Γ ( α ) 0 λ t u α - 1 e - u d u
(5)

A well known definite integral, obtained by integration by parts, is

a t n - 1 e - t d t = Γ ( n ) e - a k = 0 n - 1 a k k ! with Γ ( n ) = ( n - 1 ) ! a t n - 1 e - t d t = Γ ( n ) e - a k = 0 n - 1 a k k ! with Γ ( n ) = ( n - 1 ) !
(6)

Noting that 1=e-aea=e-ak=0akk!1=e-aea=e-ak=0akk! we find after some simple algebra that

1 Γ ( n ) 0 a t n - 1 e - t d t = e - a k = n a k k ! 1 Γ ( n ) 0 a t n - 1 e - t d t = e - a k = n a k k !
(7)

For a=λta=λt and α=nα=n, we have the following equality iff XX gamma (α,λ)(α,λ).

P ( X t ) = 1 Γ ( n ) 0 λ t u n - 1 d - u d u = e - λ t k = n ( λ t ) k k ! P ( X t ) = 1 Γ ( n ) 0 λ t u n - 1 d - u d u = e - λ t k = n ( λ t ) k k !
(8)

Now

P ( Y n ) = e - λ t k = n ( λ t ) k k ! iff Y Poisson ( λ t ) P ( Y n ) = e - λ t k = n ( λ t ) k k ! iff Y Poisson ( λ t )
(9)

The gaussian (normal) approximation

The central limit theorem, referred to in the discussion of the gaussian or normal distribution above, suggests that the binomial and Poisson distributions should be approximated by the gaussian. The number of successes in n trials has the binomial (n,p)(n,p) distribution. This random variable may be expressed

X = i = 1 n I E i where the I E i constitute an independent class X = i = 1 n I E i where the I E i constitute an independent class
(10)

Since the mean value of X is npnp and the variance is npqnpq, the distribution should be approximately N(np,npq)N(np,npq).

Figure 1: Gaussian approximation to the binomial.
A graph of the Gaussian approximation to the binomial: n=300, p=0.1. The x-axis represents the values of k ranging from 10-50, while the y-axis shows range of density from 0.01-0.08. The distribution plotted rises and falls at an equal rate with its peak at (30,0.075). The distribution occurs over a series of vertical bars with their heights roughly approximate to the corresponding position of the distribution. 'The actual distribution looks like a bell curve'.

Use of the generating function shows that the sum of independent Poisson random variables is Poisson. Now if XX Poisson (μ)(μ), then X may be considered the sum of n independent random variables, each Poisson (μ/n)(μ/n). Since the mean value and the variance are both μ, it is reasonable to suppose that suppose that X is approximately N(μ,μ)N(μ,μ).

It is generally best to compare distribution functions. Since the binomial and Poisson distributions are integer-valued, it turns out that the best gaussian approximaton is obtained by making a “continuity correction.” To get an approximation to a density for an integer-valued random variable, the probability at t=kt=k is represented by a rectangle of height pk and unit width, with k as the midpoint. Figure 1 shows a plot of the “density” and the corresponding gaussian density for n=300n=300, p=0.1p=0.1. It is apparent that the gaussian density is offset by approximately 1/2. To approximate the probability XkXk, take the area under the curve from k+1/2k+1/2; this is called the continuity correction.

Use of m-procedures to compare

We have two m-procedures to make the comparisons. First, we consider approximation of the

Figure 2: Gaussian approximation to the Poisson distribution function μ=10μ=10.
A graph of a Gaussian Approximation to Poisson Distribution.The x-axis displays the values for t ranging from 2-16 while the y-axis represents the values of distribution functions ranging from 0-1. There are two plotted distributions. The Poisson approximation is represented stepwise with green circles present near the external right angles indicating the position of the adjusted Gaussian approximation. The Gaussian approximation is represented with a dashed blue line and corresponds roughly to the green circles.
Figure 3: Gaussian approximation to the Poisson distribution function μ=100μ=100.
A graph of a Gaussian Approximation to Poisson distribution for  mu=100. The x-axis displays the values for t ranging from 80-120 while the y-axis represents the values of distribution functions ranging from 0-1. There are two plotted distributions. The Poisson approximation is represented stepwise with green circles present near the external right angles indicating the position of the adjusted Gaussian approximation. The Gaussian approximation is represented with a dashed blue line and corresponds roughly to the green circles.

Poisson (μ)(μ) distribution. The m-procedure poissapp calls for a value of μ, selects a suitable range about k=μk=μ and plots the distribution function for the Poisson distribution (stairs) and the normal (gaussian) distribution (dash dot) for N(μ,μ)N(μ,μ). In addition, the continuity correction is applied to the gaussian distribution at integer values (circles). Figure 3 shows plots for μ=10μ=10. It is clear that the continuity correction provides a much better approximation. The plots in Figure 4 are for μ=100μ=100. Here the continuity correction provides the better approximation, but not by as much as for the smaller μ.

Figure 4: Poisson and Gaussian approximation to the binomial: n = 1000, p = 0.03.
A graph of a Gaussian Approximation to Poisson distribution for  mu=100. The x-axis displays the values for t ranging from 80-120 while the y-axis represents the values of distribution functions ranging from 0-1. There are two plotted distributions. The Poisson approximation is represented stepwise with green circles present near the external right angles indicating the position of the adjusted Gaussian approximation. The Gaussian approximation is represented with a dashed blue line and corresponds roughly to the green circles.
Figure 5: Poisson and Gaussian approximation to the binomial: n = 50, p = 0.6.
A graph of an Approximation of Binomial by Poisson and Gaussian. The x-axis displays the values for t ranging from 15-40 while the y-axis represents the values of distribution functions ranging from 0-1. There are two plotted distributions. The Binomial approximation is represented stepwise with green circles present near the external right angles indicating the position of the adjusted Gaussian approximation. The Poisson approximation is represented with a dashed blue line and corresponds roughly to the green circles, except at the top right of the graph where the Poisson distribution falls below the Binomial.

The m-procedure bincomp compares the binomial, gaussian, and Poisson distributions. It calls for values of n and p, selects suitable k values, and plots the distribution function for the binomial, a continuous approximation to the distribution function for the Poisson, and continuity adjusted values of the gaussian distribution function at the integer values. Figure 4 shows plots for n=1000n=1000, p=0.03p=0.03. The good agreement of all three distribution functions is evident. Figure 5 shows plots for n=50n=50, p=0.6p=0.6. There is still good agreement of the binomial and adjusted gaussian. However, the Poisson distribution does not track very well. The difficulty, as we see in the unit Variance, is the difference in variances—npqnpq for the binomial as compared with npnp for the Poisson.

Approximation of a real random variable by simple random variables

Simple random variables play a significant role, both in theory and applications. In the unit Random Variables, we show how a simple random variable is determined by the set of points on the real line representing the possible values and the corresponding set of probabilities that each of these values is taken on. This describes the distribution of the random variable and makes possible calculations of event probabilities and parameters for the distribution.

A continuous random variable is characterized by a set of possible values spread continuously over an interval or collection of intervals. In this case, the probability is also spread smoothly. The distribution is described by a probability density function, whose value at any point indicates "the probability per unit length" near the point. A simple approximation is obtained by subdividing an interval which includes the range (the set of possible values) into small enough subintervals that the density is approximately constant over each subinterval. A point in each subinterval is selected and is assigned the probability mass in its subinterval. The combination of the selected points and the corresponding probabilities describes the distribution of an approximating simple random variable. Calculations based on this distribution approximate corresponding calculations on the continuous distribution.

Before examining a general approximation procedure which has significant consequences for later treatments, we consider some illustrative examples.

Example 1: Simple approximation to Poisson

A random variable with the Poisson distribution is unbounded. However, for a given parameter value μ, the probability for knkn, n sufficiently large, is negligible. Experiment indicates n=μ+6μn=μ+6μ (i.e., six standard deviations beyond the mean) is a reasonable value for 5μ2005μ200.

>> mu = [5 10 20 30 40 50 70 100 150 200];
>> K = zeros(1,length(mu));
>> p = zeros(1,length(mu));
>> for i = 1:length(mu)
     K(i) = floor(mu(i)+ 6*sqrt(mu(i)));
     p(i) = cpoisson(mu(i),K(i));
end
>> disp([mu;K;p*1e6]')
    5.0000   18.0000    5.4163  % Residual probabilities are 0.000001
   10.0000   28.0000    2.2535  % times the numbers in the last column.
   20.0000   46.0000    0.4540  % K is the value of k needed to achieve
   30.0000   62.0000    0.2140  % the residual shown.
   40.0000   77.0000    0.1354  
   50.0000   92.0000    0.0668
   70.0000  120.0000    0.0359
  100.0000  160.0000    0.0205
  150.0000  223.0000    0.0159
  200.0000  284.0000    0.0133

An m-procedure for discrete approximation

If X is bounded, absolutely continuous with density functon fX, the m-procedure tappr sets up the distribution for an approximating simple random variable. An interval containing the range of X is divided into a specified number of equal subdivisions. The probability mass for each subinterval is assigned to the midpoint. If dxdx is the length of the subintervals, then the integral of the density function over the subinterval is approximated by fX(ti)dxfX(ti)dx. where ti is the midpoint. In effect, the graph of the density over the subinterval is approximated by a rectangle of length dxdx and height fX(ti)fX(ti). Once the approximating simple distribution is established, calculations are carried out as for simple random variables.

Example 2: A numerical example

Suppose fX(t)=3t2,0t1fX(t)=3t2,0t1. Determine P(0.2X0.9)P(0.2X0.9).

SOLUTION

In this case, an analytical solution is easy. FX(t)=t3FX(t)=t3 on the interval [0,1][0,1], so

P=0.93-0.23=0.7210P=0.93-0.23=0.7210. We use tappr as follows:

>> tappr
Enter matrix [a b] of x-range endpoints  [0 1]
Enter number of x approximation points  200
Enter density as a function of t  3*t.^2
Use row matrices X and PX as in the simple case
>> M = (X >= 0.2)&(X <= 0.9);
>> p = M*PX'
p  =  0.7210

Because of the regularity of the density and the number of approximation points, the result agrees quite well with the theoretical value.

The next example is a more complex one. In particular, the distribution is not bounded. However, it is easy to determine a bound beyond which the probability is negligible.

Figure 6: Distribution function for Example 3.
A graph of a Distribution figure. The x-axis displays the values for t ranging from 0-8 while the y-axis represents the values of u=F(t) ranging from 0-1. The plotted distribution rises gradually at first but at (4,0.3) the slope changes to be nearly vertical, and at around (5,0.95) the slope begins to plateau.

Example 3: Radial tire mileage

The life (in miles) of a certain brand of radial tires may be represented by a random variable X with density

f X ( t ) = t 2 / a 3 for 0 t < a ( b / a ) e - k ( t - a ) for a t f X ( t ) = t 2 / a 3 for 0 t < a ( b / a ) e - k ( t - a ) for a t
(11)

where a=40,000,b=20/3a=40,000,b=20/3, and k=1/4000k=1/4000. Determine P(X45,000)P(X45,000).

>> a = 40000;
>> b = 20/3;
>> k = 1/4000;
>> % Test shows cutoff point of 80000 should be satisfactory
>> tappr
Enter matrix [a b] of x-range endpoints  [0 80000]
Enter number of x approximation points  80000/20
Enter density as a function of t  (t.^2/a^3).*(t < 40000) + ...
(b/a)*exp(k*(a-t)).*(t >= 40000)
Use row matrices X and PX as in the simple case
>> P = (X >= 45000)*PX'
P   =  0.1910             % Theoretical value = (2/3)exp(-5/4) = 0.191003
>> cdbn
Enter row matrix of VALUES  X
Enter row matrix of PROBABILITIES  PX  % See Figure 7 for plot

In this case, we use a rather large number of approximation points. As a consequence, the results are quite accurate. In the single-variable case, designating a large number of approximating points usually causes no computer memory problem.

The general approximation procedure

We show now that any bounded real random variable may be approximated as closely as desired by a simple random variable (i.e., one having a finite set of possible values). For the unbounded case, the approximation is close except in a portion of the range having arbitrarily small total probability.

We limit our discussion to the bounded case, in which the range of X is limited to a bounded interval I=[a,b]I=[a,b]. Suppose I is partitioned into n subintervals by points ti, 1in-11in-1, with a=t0a=t0 and b=tnb=tn. Let Mi=[ti-1,ti)Mi=[ti-1,ti) be the ith subinterval, 1in-11in-1 and Mn=[tn-1,tn]Mn=[tn-1,tn] (see Figure 7). Now random variable X may map into any point in the interval, and hence into any point in each subinterval Mi. Let Ei=X-1(Mi)Ei=X-1(Mi) be the set of points mapped into Mi by X. Then the Ei form a partition of the basic space Ω. For the given subdivision, we form a simple random variable Xs as follows. In each subinterval, pick a point si,ti-1si<tisi,ti-1si<ti. Consider the simple random variable Xs=i=1nsiIEiXs=i=1nsiIEi.

Figure 7: Partition of the interval I including the range of X
Figure 7 (fig7_6_2.png)
Figure 8: Refinement of the partition by additional subdividion points.
Figure 8 (fig7_6_3.png)

This random variable is in canonical form. If ωEiωEi, then X(ω)MiX(ω)Mi and Xs(ω)=siXs(ω)=si. Now the absolute value of the difference satisfies

| X ( ω ) - X s ( ω ) | < t i - t i - 1 the length of subinterval M i | X ( ω ) - X s ( ω ) | < t i - t i - 1 the length of subinterval M i
(12)

Since this is true for each ω and the corresponding subinterval, we have the important fact

| X ( ω ) - X s ( ω ) | < maximum length of the M i | X ( ω ) - X s ( ω ) | < maximum length of the M i
(13)

By making the subintervals small enough by increasing the number of subdivision points, we can make the difference as small as we please.

While the choice of the si is arbitrary in each Mi, the selection of si=ti-1si=ti-1 (the left-hand endpoint) leads to the property Xs(ω)X(ω)ωXs(ω)X(ω)ω. In this case, if we add subdivision points to decrease the size of some or all of the Mi, the new simple approximation Ys satisfies

X s ( ω ) Y s ( ω ) X ( ω ) ω X s ( ω ) Y s ( ω ) X ( ω ) ω
(14)

To see this, consider ti*Miti*Mi (see Figure 8). Mi is partitioned into Mi'Mi''Mi'Mi'' and Ei is partitioned into Ei'Ei''Ei'Ei''. X maps Ei'Ei' into Mi'Mi' and Ei''Ei'' into Mi''Mi''. Ys maps Ei'Ei' into ti and maps Ei''Ei'' into ti*>titi*>ti. Xs maps both Ei'Ei' and Ei''Ei'' into ti. Thus, the asserted inequality must hold for each ω By taking a sequence of partitions in which each succeeding partition refines the previous (i.e. adds subdivision points) in such a way that the maximum length of subinterval goes to zero, we may form a nondecreasing sequence of simple random variables Xn which increase to X for each ω.

The latter result may be extended to random variables unbounded above. Simply let N th set of subdivision points extend from a to N, making the last subinterval [N,)[N,). Subintervals from a to N are made increasingly shorter. The result is a nondecreasing sequence {XN:1N}{XN:1N} of simple random variables, with XN(ω)X(ω)XN(ω)X(ω) as NN, for each ωΩωΩ.

For probability calculations, we simply select an interval I large enough that the probability outside I is negligible and use a simple approximation over I.

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks