Characteristic Functions 2.2 2003/03/13 2003/04/11 Nick Kingsbury ngk10@cam.ac.uk Liqun Wang liqun@rice.edu Nick Kingsbury ngk10@cam.ac.uk Characteristic Functions Fourier Transform This module introduces characteristic functions. You have already encountered the Moment Generating Function of a pdf in the Part IB probability course. This function was closely related to the Laplace Transform of the pdf. Now we introduce the Characteristic Function for a random variable, which is closely related to the Fourier Transform of the pdf. In the same way that Fourier Transforms allow easy manipulation of signals when they are convolved with linear system impulse responses, Characteristic Functions allow easy manipulation of convolved pdfs when they represent sums of random processes. The Characteristic Function of a pdf is defined as: ΦX u u x x u x fX x ℱ u where ℱ u is the Fourier Transform of the pdf. Note that whenever fX is a valid pdf, Φ 0 x fX x 1 Properties of Fourier Transforms apply with u substituted for ω. In particular: Convolution - (sums of independent rv's) Y i 1 N Xi fY f X 1 f X 2 … f X N ΦY u i 1 N Φ X i u Inversion fX x 1 2 u u x ΦX u Moments un ΦX u x x n u x fX x X n x x n fX x 1 n u 0 u n ΦX u Scaling If Y a X , fY y fX x a from this equation in our previous discussion of functions of random variables, then ΦY u y u y fY y x u a x fX x ΦX a u

Characteristic Function of a Gaussian pdf The Gaussian or normal distribution is very important, largely because of the Central Limit Theorem which we shall prove below. Because of this (and as part of the proof of this theorem) we shall show here that a Gaussian pdf has a Gaussian characteristic function too. A Gaussian distribution with mean μ and variance σ 2 has pdf: f x 1 2 σ 2 x μ 2 2 σ 2 Its characteristic function is obtained as follows, using a trick known as completing the square of the exponent: ΦX u u x x u x fX x 1 2 σ 2 x x 2 2 μ x μ 2 2 σ 2 u x 2 σ 2 1 2 σ 2 x x μ u σ 2 2 2 σ 2 2 u σ 2 μ u 2 σ 4 2 σ 2 u μ u 2 σ 2 2 since the integral in brackets is similar to a Gaussian pdf and integrates to unity. Thus the characteristic function of a Gaussian pdf is also Gaussian in magnitude, u 2 σ 2 2 , with standard deviation 1 σ , and with a linear phase rotation term, u μ , whose rate of rotation equals the mean μ of the pdf. This coincides with standard results from Fourier analysis of Gaussian waveforms and their spectra (e.g. Fourier transform of a Gaussian waveform with time shift).

Summation of two or more Gaussian random variables If two variables, X 1 and X 2 , with Gaussian pdfs are summed to produce X, their characteristic functions will be multiplied together (equivalent to convolving their pdfs) to give ΦX u Φ X1 u Φ X2 u u μ1 μ2 u 2 σ1 2 σ2 2 2 This is the characteristic function of a Gaussian pdf with mean ( μ1 μ2 ) and variance ( σ1 2 σ2 2 ). Further Gaussian variables can be added and the pdf will remain Gaussian with further terms added to the above expressions for the combined mean and variance.

Central Limit Theorem The central limit theorem states broadly that if a large number N of independent random variables of arbitrary pdf, but with equal variance σ 2 and zero mean, are summed together and scaled by 1 N to keep the total energy independent of N, then the pdf of the resulting variable will tend to a zero-mean Gaussian with variance σ 2 as N tends to infinity. This result is obvious from the previous result if the input pdfs are also Gaussian, but it is the fact that it applies for arbitrary input pdfs that is remarkable, and is the reason for the importance of the Gaussian (or normal) pdf. Noise generated in nature is nearly always the result of summing many tiny random processes (e.g. noise from electron energy transitions in a resistor or transistor, or from distant worldwide thunder storms at a radio antenna) and hence tends to a Gaussian pdf. Although for simplicity, we shall prove the result only for the case when all the summed processes have the same variance and pdfs, the central limit result is more general than this and applies in many cases even when the variance and pdfs are not all the same.

Proof: Let X i ( i 1 to N) be the N independent random processes, each will zero mean and variance σ 2 , which are combined to give X 1 N i 1 N Xi Then, if the characteristic function of each input process before scaling is Φ u and we use to include the scaling by 1 N , the characteristic function of X is ΦX u i 1 N Φ X i u N Φ u N N Taking logs: ΦX u N Φ u N Using Taylor's theorem to expand Φ u N in terms of its derivatives at u 0 (and hence its moments) gives Φ u N Φ 0 u N Φ 0 1 2 u N 2 2 Φ 0 1 6 u N 3 3 Φ 0 1 24 u N 4 4 Φ 0 … From the Moments property of characteristic functions with zero mean: valid pdf Φ 0 Xi 0 1 zero mean Φ 0 Xi 0 variance 2 Φ 0 2 Xi 2 σ 2 scaled skewness 3 Φ 0 3 Xi 3 γ σ 3 scaled kurtosis 4 Φ 0 4 Xi 4 κ 3 σ 4 These are all constants, independent of N, and dependent only on the shape of the pdfs f Xi . Substituting these moments into and and using the series expansion, 1 x x + (terms of order x 2 or smaller), gives ΦX u N Φ u N N 1 u 2 2 N σ 2 ** N u 2 σ 2 2 N ** u 2 σ 2 2 ## where ** represents the terms of order N 3 2 or smaller and ## represents the terms of order N 1 2 or smaller. As N , ΦX u u 2 σ 2 2 Therefore, as N ΦX u u 2 σ 2 2 Note that, if the input pdfs are symmetric, the skewness will be zero and the error terms will decay as N -1 rather than N 1 2 ; and so convergence to a Gaussian characteristic function will be more rapid. Hence we may now infer from , and that the pdf of X as N will be given by fX x 1 2 σ 2 x 2 2 σ 2 Thus we have proved the required central limit result.

shows an example of convergence when the input pdfs are uniform, and N is gradually increased from 1 to 50. By N 12 , convergence is good, and this is how some 'Gaussian' random generator functions operate - by summing typically 12 uncorrelated random numbers with uniform pdfs. For some less smooth or more skewed pdfs, convergence can be slower, as shown for a highly skewed triangular pdf in ; and pdfs of discrete processes are particularly problematic in this respect, as illustrated in .

Convergence toward a Gaussian pdf (Central Limit Theorem) for 3 different input pdfs for N 1 to 50. Note that the uniform pdf (a) with smallest higher-order moments converges fastest. Curves are shown for N 1 2 3 4 6 8 10 12 15 20 30 50 .