Skip to content Skip to navigation

OpenStax-CNX

You are here: Home » Content » Sub-Gaussian random variables

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Sub-Gaussian random variables

Module by: Mark A. Davenport. E-mail the author

Summary: In this module we introduce the sub-Gaussian and strictly sub-Gaussian distributions. We provide some simple examples and illustrate some of the key properties of sub-Gaussian random variables.

A number of distributions, notably Gaussian and Bernoulli, are known to satisfy certain concentration of measure inequalities. We will analyze this phenomenon from a more general perspective by considering the class of sub-Gaussian distributions [1].

Definition 1:

A random variable XX is called sub-Gaussian if there exists a constant c>0c>0 such that

E exp ( X t ) exp ( c 2 t 2 / 2 ) E exp ( X t ) exp ( c 2 t 2 / 2 )
(1)

holds for all tRtR. We use the notation X Sub (c2)X Sub (c2) to denote that XX satisfies Equation 1.

The function Eexp(Xt)Eexp(Xt) is the moment-generating function of XX, while the upper bound in Equation 1 is the moment-generating function of a Gaussian random variable. Thus, a sub-Gaussian distribution is one whose moment-generating function is bounded by that of a Gaussian. There are a tremendous number of sub-Gaussian distributions, but there are two particularly important examples:

Example 1

If XN(0,σ2)XN(0,σ2), i.e., XX is a zero-mean Gaussian random variable with variance σ2σ2, then X Sub (σ2)X Sub (σ2). Indeed, as mentioned above, the moment-generating function of a Gaussian is given by Eexp(Xt)=exp(σ2t2/2)Eexp(Xt)=exp(σ2t2/2), and thus Equation 1 is trivially satisfied.

Example 2

If XX is a zero-mean, bounded random variable, i.e., one for which there exists a constant BB such that |X|B|X|B with probability 1, then X Sub (B2)X Sub (B2).

A common way to characterize sub-Gaussian random variables is through analyzing their moments. We consider only the mean and variance in the following elementary lemma, proven in [1].

Lemma 1: (Buldygin-Kozachenko [1])

If X Sub (c2)X Sub (c2) then,

E ( X ) = 0 E ( X ) = 0
(2)

and

E ( X 2 ) c 2 . E ( X 2 ) c 2 .
(3)

Lemma 1 shows that if X Sub (c2)X Sub (c2) then E(X2)c2E(X2)c2. In some settings it will be useful to consider a more restrictive class of random variables for which this inequality becomes an equality.

Definition 2:

A random variable XX is called strictly sub-Gaussian if X Sub (σ2)X Sub (σ2) where σ2=E(X2)σ2=E(X2), i.e., the inequality

E exp ( X t ) exp ( σ 2 t 2 / 2 ) E exp ( X t ) exp ( σ 2 t 2 / 2 )
(4)

holds for all tRtR. To denote that XX is strictly sub-Gaussian with variance σ2σ2, we will use the notation X SSub (σ2)X SSub (σ2).

Example 3

If XN(0,σ2)XN(0,σ2), then X SSub (σ2)X SSub (σ2).

Example 4

If XU(-1,1)XU(-1,1), i.e., XX is uniformly distributed on the interval [-1,1][-1,1], then X SSub (1/3)X SSub (1/3).

Example 5

Now consider the random variable with distribution such that

P ( X = 1 ) = P ( X = - 1 ) = 1 - s 2 , P ( X = 0 ) = s , s [ 0 , 1 ) . P ( X = 1 ) = P ( X = - 1 ) = 1 - s 2 , P ( X = 0 ) = s , s [ 0 , 1 ) .
(5)

For any s[0,2/3]s[0,2/3], X SSub (1-s)X SSub (1-s). For s(2/3,1)s(2/3,1), XX is not strictly sub-Gaussian.

We now provide an equivalent characterization for sub-Gaussian and strictly sub-Gaussian random variables, proven in [1], that illustrates their concentration of measure behavior.

Theorem 1: (Buldygin-Kozachenko [1])

A random variable X Sub (c2)X Sub (c2) if and only if there exists a t00t00 and a constant a0a0 such that

P ( | X | t ) 2 exp - t 2 2 a 2 P ( | X | t ) 2 exp - t 2 2 a 2
(6)

for all tt0tt0. Moreover, if X SSub (σ2)X SSub (σ2), then Equation 6 holds for all t>0t>0 with a=σa=σ.

Finally, sub-Gaussian distributions also satisfy one of the fundamental properties of a Gaussian distribution: the sum of two sub-Gaussian random variables is itself a sub-Gaussian random variable. This result is established in more generality in the following lemma.

Lemma 2

Suppose that X=[X1,X2,...,XN]X=[X1,X2,...,XN], where each XiXi is independent and identically distributed (i.i.d.) with Xi Sub (c2)Xi Sub (c2). Then for any αRNαRN, X,α Sub c2α22X,α Sub c2α22. Similarly, if each Xi SSub (σ2)Xi SSub (σ2), then for any αRNαRN, X,α SSub σ2α22X,α SSub σ2α22.

Proof

Since the XiXi are i.i.d., the joint distribution factors and simplifies as:

E exp t i = 1 N α i X i = E i = 1 N exp t α i X i = i = 1 N E exp t α i X i i = 1 N exp c 2 ( α i t ) 2 / 2 = exp i = 1 N α i 2 c 2 t 2 / 2 . E exp t i = 1 N α i X i = E i = 1 N exp t α i X i = i = 1 N E exp t α i X i i = 1 N exp c 2 ( α i t ) 2 / 2 = exp i = 1 N α i 2 c 2 t 2 / 2 .
(7)

If the XiXi are strictly sub-Gaussian, then the result follows by setting c2=σ2c2=σ2 and observing that EX,α2=σ2α22EX,α2=σ2α22.

References

  1. Buldygin, V. and Kozachenko, Y. (2000). Metric Characterization of Random Variables and Random Processes. Providence, RI: American Mathematical Society.

Content actions

Download module as:

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks