The Central Limit Theorem 1.2 2003/05/15 2003/08/08 14:18:39.664 GMT-5 Don Johnson dhj@rice.edu Don Johnson dhj@rice.edu Eileen Krause erkrause@rice.edu Kyle Clarkson kclarks@rice.edu Elizabeth Gregory lizzardg@rice.edu Kevin Duh kevinduh@rice.edu Mariyah Poonawala mariyah@rice.edu Jeffrey Silverman jsilv@rice.edu Let X l denote a sequence of independent, identically distributed, random variables. Assuming they have zero means and finite variances (equaling σ 2 ), the Central Limit Theorem states that the sum l 1 L X l L converges in distribution to a Gaussian random variable. 1 L l 1 L X l → L → ∞ 0 σ 2 Because of its generality, this theorem is often used to simplify calculations involving finite sums of non-Gaussian random variables. However, attention is seldom paid to the convergence rate of the Central Limit Theorem. Kolmogorov, the famous twentieth century mathematician, is reputed to have said, "The Central Limit Theorem is a dangerous tool in the hands of amateurs." Let's see what he meant. Taking σ 2 1 , the key result is that the magnitude of the difference between P x , defined to be the probability that the sum given above exceeds x, and Q x , the probability that a unit-variance Gaussian random variable exceeds x, is bounded by a quantity inversely related to the square root of L (Cramer: Theorem 24). P x Q x c X 3 σ 3 1 L The constant of proportionality c is a number known to be about 0.8 (Hall: p6). The ratio of absolute third moment of X l to the cube of its standard deviation, known as the skew and denoted by γ X , depends only on the distribution of X l and is independent of scale. This bound on the absolute error has been shown to be tight (Cramer: pp. 79ff). Using our lower bound for Q · (see ), we find that the relative error in the Central Limit Theorem approximation to the distribution of finite sums is bounded for x 0 as P x Q x Q x c γ X 2 L x 2 2 2 x 1 1 x 2 x x 1 Suppose we require that the relative error not exceed some specific value ε. The normalized (by the standard deviation) boundary x at which the approximation is evaluated must not violate L ε 2 2 c 2 γ X 2 x 2 4 x 1 1 x 2 x 2 x 1 As shown in , the right side of this equation is a monotonically increasing function.

The quantity which governs the limits of validity for numerically applying the Central Limit Theorem on finite numbers of data is shown over a portion of its range. To judge these limits, we must compute the quantity L ε 2 2 c 2 γ X , where ε denotes the desired percentage error in the Central Limit Theorem approximation and L the number of observations. Selecting this value on the vertical axis and determining the value of x yielding it, we find the normalized ( x 1 implies unit variance) upper limit on an L-term sum to which the Central Limit Theorem is guaranteed to apply. Note how rapidly the curve increases, suggesting that large amounts of data are needed for accurate approximation.

If ε 0.1 and taking c γ X arbitrarily to be unity (a reasonable value), the upper limit of the preceding equation becomes 1.6-3 L . Examining , we find that for L 10000 , x must not exceed 1.17. Because we have normalized to unit variance, this example suggests that the Gaussian approximates the distribution of a ten-thousand term sum only over a range corresponding to a 76% area about the mean. Consequently, the Central Limit Theorem, as a finite-sample distributional approximation, is only guaranteed to hold near the mode of the Gaussian, with huge numbers of observations needed to specify the tail behavior. Realizing this fact will keep us from being ignorant amateurs. H. Cramér Random Variables and Probability Distributions Cambridge University Press 1970 Third Edition P. Hall Rates of Convergence in the Central Limit Theorem Research Notes in Mathematics Pitman Advanced Publishing Program 1982 62 6