Three basic transforms

We define each of three transforms, determine some key properties, and use them to study various probability distributions associated with random variables. In the section on integral transforms, we show their relationship to well known integral transforms. These have been studied extensively and used in many other applications, which makes it possible to utilize the considerable literature on these transforms.

Definition. The moment generating functionM_X for random variable X (i.e., for its distribution) is the function

The characteristic functionφ_X for random variable X is

The generating functiongX(s)gX(s) for a nonnegative, integer-valued random variable X is

The generating function E[sX]E[sX] has meaning for more general random variables, but its usefulness is greatest for nonnegative, integer-valued variables, and we limit our consideration to that case.

The defining expressions display similarities which show useful relationships. We note two which are particularly useful.

Because of the latter relationship, we ordinarily use the moment generating function instead of the characteristic function to avoid writing the complex unit i. When desirable, we convert easily by the change of variable.

The integral transform character of these entities implies that there is essentially a one-to-one relationship between the transform and the distribution.

Moments

The name and some of the importance of the moment generating function arise from the fact that the derivatives of M_X evaluateed at s=0s=0 are the moments about the origin. Specifically

Since expectation is an integral and because of the regularity of the integrand, we may differentiate inside the integral with respect to the parameter.

Upon setting s=0s=0, we have MX'(0)=E[X]MX'(0)=E[X]. Repeated differentiation gives the general result. The corresponding result for the characteristic function is φ(k)(0)=ikE[Xk]φ(k)(0)=ikE[Xk].

The generating function does not lend itself readily to computing moments, except that

For higher order moments, we may convert the generating function to the moment generating function by replacing s with e^s, then work with M_X and its derivatives.

Operational properties

We refer to the following as operational properties.

Note that we have not shown that being uncorrelated implies the product rule.

We utilize these properties in determining the moment generating and generating functions for several of our common distributions.

Some discrete distributions

Some absolutely continuous distributions

Moment generating function and simple random variables

Suppose X=∑i=1ntiIAiX=∑i=1ntiIAi in canonical form. That is, A_i is the event {X=ti}{X=ti} for each of the distinct values in the range of X, with pi=P(Ai)=P(X=ti)pi=P(Ai)=P(X=ti). Then the moment generating function for X is

The moment generating function M_X is thus related directly and simply to the distribution for random variable X.

Consider the problem of determining the sum of an independent pair {X,Y}{X,Y} of simple random variables. The moment generating function for the sum is the product of the moment generating functions. Now if Y=∑j=1mujIBjY=∑j=1mujIBj, with P(Y=uj)=πjP(Y=uj)=πj, we have

The various values are sums ti+ujti+uj of pairs (ti,uj)(ti,uj) of values. Each of these sums has probability piπjpiπj for the values corresponding to ti,ujti,uj. Since more than one pair sum may have the same value, we need to sort the values, consolidate like values and add the probabilties for like values to achieve the distribution for the sum. We have an m-function mgsum for achieving this directly. It produces the pair-products for the probabilities and the pair-sums for the values, then performs a csort operation. Although not directly dependent upon the moment generating function analysis, it produces the same result as that produced by multiplying moment generating functions.

X = [1 3 5 7];
Y = 2:4;
PX = 0.1*[2 4 3 1];
PY = 0.1*[3 5 2];
[Z,PZ] = mgsum(X,Y,PX,PY);
disp([Z;PZ]')
    3.0000    0.0600
    4.0000    0.1000
    5.0000    0.1600
    6.0000    0.2000
    7.0000    0.1700
    8.0000    0.1500
    9.0000    0.0900
   10.0000    0.0500
   11.0000    0.0200

This could, of course, have been achieved by using icalc and csort, which has the advantage that other functions of X and Y may be handled. Also, since the random variables are nonnegative, integer-valued, the MATLAB convolution function may be used (see Example 7). By repeated use of the function mgsum, we may obtain the distribution for the sum of more than two simple random variables. The m-functions mgsum3 and mgsum4 utilize this strategy.

The techniques for simple random variables may be used with the simple approximations to absolutely continuous random variables.

Example 5: Difference of uniform distribution

The moment generating functions for the uniform and the symmetric triangular show that the latter appears naturally as the difference of two uniformly distributed random variables. We consider X and Y iid, uniform on [0,1].

tappr
Enter matrix [a b] of x-range endpoints  [0 1]
Enter number of x approximation points  200
Enter density as a function of t  t<=1
Use row matrices X and PX as in the simple case
[Z,PZ] = mgsum(X,-X,PX,PX);
plot(Z,PZ/d)               % Divide by d to recover f(t)
%  plotting details   ---  see Figure 1

Figure 1: Density for the difference of an independent pair, uniform (0,1).

The generating function

The form of the generating function for a nonnegative, integer-valued random variable exhibits a number of important properties.