Skip to content Skip to navigation Skip to collection information

OpenStax_CNX

You are here: Home » Content » Purdue Digital Signal Processing Labs (ECE 438) » Lab 7b - Discrete-Time Random Processes (part 2)

Navigation

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.
  • NSF Partnership display tagshide tags

    This collection is included inLens: NSF Partnership in Signal Processing
    By: Sidney Burrus

    Click the "NSF Partnership" link to see all content affiliated with them.

    Click the tag icon tag icon to display tags associated with this content.

  • Featured Content display tagshide tags

    This collection is included inLens: Connexions Featured Content
    By: Connexions

    Click the "Featured Content" link to see all content affiliated with them.

    Click the tag icon tag icon to display tags associated with this content.

Also in these lenses

  • UniqU content

    This collection is included inLens: UniqU's lens
    By: UniqU, LLC

    Click the "UniqU content" link to see all content selected in this lens.

  • Lens for Engineering

    This module and collection are included inLens: Lens for Engineering
    By: Sidney Burrus

    Click the "Lens for Engineering" link to see all content selected in this lens.

Recently Viewed

This feature requires Javascript to be enabled.

Tags

(What is a tag?)

These tags come from the endorsement, affiliation, and other lenses that include this content.
 

Lab 7b - Discrete-Time Random Processes (part 2)

Module by: Charles A. Bouman. E-mail the author

Questions or comments concerning this laboratory should be directed to Prof. Charles A. Bouman, School of Electrical and Computer Engineering, Purdue University, West Lafayette IN 47907; (765) 494-0340; bouman@ecn.purdue.edu

Bivariate Distributions

In this section, we will study the concept of a bivariate distribution. We will see that bivariate distributions characterize how two random variables are related to each other. We will also see that correlation and covariance are two simple measures of the dependencies between random variables, which can be very useful for analyzing both random variables and random processes.

Background on Bivariate Distributions

Sometimes we need to account for not just one random variable, but several. In this section, we will examine the case of two random variables–the so called bivariate case–but the theory is easily generalized to accommodate more than two.

The random variables XX and YY have cumulative distribution functions (CDFs) FX(x)FX(x) and FY(y)FY(y), also known as marginal CDFs. Since there may be an interaction between XX and YY, the marginal statistics may not fully describe their behavior. Therefore we define a bivariate, or joint CDF as

F X , Y ( x , y ) = P ( X x , Y y ) . F X , Y ( x , y ) = P ( X x , Y y ) .
(1)

If the joint CDF is sufficiently “smooth”, we can define a joint probability density function,

f X , Y ( x , y ) = 2 x y F X , Y ( x , y ) . f X , Y ( x , y ) = 2 x y F X , Y ( x , y ) .
(2)

Conversely, the joint probability density function may be used to calculate the joint CDF:

F X , Y ( x , y ) = - y - x f X , Y ( s , t ) d s d t . F X , Y ( x , y ) = - y - x f X , Y ( s , t ) d s d t .
(3)

The random variables XX and YY are said to be independent if and only if their joint CDF (or PDF) is a separable function, which means

f X , Y ( x , y ) = f X ( x ) f Y ( y ) f X , Y ( x , y ) = f X ( x ) f Y ( y )
(4)

Informally, independence between random variables means that one random variable does not tell you anything about the other. As a consequence of the definition, if XX and YY are independent, then the product of their expectations is the expectation of their product.

E [ X Y ] = E [ X ] E [ Y ] E [ X Y ] = E [ X ] E [ Y ]
(5)

While the joint distribution contains all the information about XX and YY, it can be very complex and is often difficult to calculate. In many applications, a simple measure of the dependencies of XX and YY can be very useful. Three such measures are the correlation, covariance, and the correlation coefficient.

  • Correlation
    E[XY]=--xyfX,Y(x,y)dxdyE[XY]=--xyfX,Y(x,y)dxdy
    (6)
  • Covariance
    E[(X-μX)(Y-μY)]=--(x-μX)(y-μY)fX,Y(x,y)dxdyE[(X-μX)(Y-μY)]=--(x-μX)(y-μY)fX,Y(x,y)dxdy
    (7)
  • Correlation coefficient
    ρXY=E[(X-μX)(Y-μY)]σXσY=E[XY]-μXμYσXσYρXY=E[(X-μX)(Y-μY)]σXσY=E[XY]-μXμYσXσY
    (8)

If the correlation coefficient is 0, then XX and YY are said to be uncorrelated. Notice that independence implies uncorrelatedness, however the converse is not true.

Samples of Two Random Variables

In the following experiment, we will examine the relationship between the scatter plots for pairs of random samples (Xi,Zi)(Xi,Zi) and their correlation coefficient. We will see that the correlation coefficient determines the shape of the scatter plot.

Let XX and YY be independent Gaussian random variables, each with mean 0 and variance 1. We will consider the correlation between XX and ZZ, where ZZ is equal to the following:

  1. Z = Y Z = Y
    (9)
  2. Z = ( X + Y ) / 2 Z = ( X + Y ) / 2
    (10)
  3. Z = ( 4 * X + Y ) / 5 Z = ( 4 * X + Y ) / 5
    (11)
  4. Z = ( 99 * X + Y ) / 100 Z = ( 99 * X + Y ) / 100
    (12)

Notice that since ZZ is a linear combination of two Gaussian random variables, ZZ will also be Gaussian.

Use Matlab to generate 1000 i.i.d. samples of XX, denoted as X1X1, X2X2, ..., X1000X1000. Next, generate 1000 i.i.d. samples of YY, denoted as Y1Y1, Y2Y2, ..., Y1000Y1000. For each of the four choices of ZZ, perform the following tasks:

  1. Use Equation 8 to analytically calculate the correlation coefficient ρXZρXZ between XX and ZZ. Show all of your work. Remember that independence between XX and YY implies that E[XY]=E[X]E[Y]E[XY]=E[X]E[Y]. Also remember that XX and YY are zero-mean and unit variance.
  2. Create samples of ZZ using your generated samples of XX and YY.
  3. Generate a scatter plot of the ordered pair of samples (Xi,Zi)(Xi,Zi). Do this by plotting points (X1,Z1)(X1,Z1), (X2,Z2)(X2,Z2), ..., (X1000,Z1000)(X1000,Z1000). To plot points without connecting them with lines, use the '.' format, as in plot(X,Z,'.'). Use the command subplot(2,2,n) (n=1,2,3,4) to plot the four cases for ZZ in the same figure. Be sure to label each plot using the title command.
  4. Empirically compute an estimate of the correlation coefficient using your samples XiXi and ZiZi and the following formula.
    ρ^XZ=i=1N(Xi-μ^X)(Zi-μ^Z)i=1N(Xi-μ^X)2i=1N(Zi-μ^Z)2ρ^XZ=i=1N(Xi-μ^X)(Zi-μ^Z)i=1N(Xi-μ^X)2i=1N(Zi-μ^Z)2
    (13)

INLAB REPORT

  1. Hand in your derivations of the correlation coefficient ρXZρXZ along with your numerical estimates of the correlation coefficient ρ^XZρ^XZ.
  2. Why are ρXZρXZ and ρ^XZρ^XZ not exactly equal?
  3. Hand in your scatter plots of (Xi,Zi)(Xi,Zi) for the four cases. Note the theoretical correlation coefficient ρXZρXZ on each plot.
  4. Explain how the scatter plots are related to ρXZρXZ.

Autocorrelation for Filtered Random Processes

In this section, we will generate discrete-time random processes and then analyze their behavior using the correlation measure introduced in the previous section.

Background

A discrete-time random process XnXn is simply a sequence of random variables. So for each nn, XnXn is a random variable.

The autocorrelation is an important function for characterizing the behavior of random processes. If XX is a wide-sense stationary (WSS) random process, the autocorrelation is defined by

r X X ( m ) = E [ X n X n + m ] m = , - 1 , 0 , 1 , . r X X ( m ) = E [ X n X n + m ] m = , - 1 , 0 , 1 , .
(14)

Note that for a WSS random process, the autocorrelation does not vary with nn. Also, since E[XnXn+m]=E[Xn+mXn]E[XnXn+m]=E[Xn+mXn], the autocorrelation is an even function of the “lag” value mm.

Intuitively, the autocorrelation determines how strong a relation there is between samples separated by a lag value of mm. For example, if XX is a sequence of independent identically distributed (i.i.d.) random variables each with zero mean and variance σX2σX2, then the autocorrelation is given by

r X X ( m ) = E [ X n X n + m ] = E [ X n ] E [ X n + m ] if m 0 E [ X n 2 ] if m = 0 = σ X 2 δ ( m ) . r X X ( m ) = E [ X n X n + m ] = E [ X n ] E [ X n + m ] if m 0 E [ X n 2 ] if m = 0 = σ X 2 δ ( m ) .
(15)

We use the term white or white noise to describe this type of random process. More precisely, a random process is called white if its values XnXn and Xn+mXn+m are uncorrelated for every m0m0.

Figure 1: A LTI system diagram
Figure 1 (sys.png)

If we run a white random process XnXn through an LTI filter as in Figure 1, the output random variables YnYn may become correlated. In fact, it can be shown that the output autocorrelation rYY(m)rYY(m) is related to the input autocorrelation rXX(m)rXX(m) through the filter's impulse response h(m)h(m).

r Y Y ( m ) = h ( m ) * h ( - m ) * r X X ( m ) r Y Y ( m ) = h ( m ) * h ( - m ) * r X X ( m )
(16)

Experiment

Consider a white Gaussian random process XnXn with mean 0 and variance 1 as input to the following filter.

y ( n ) = x ( n ) - x ( n - 1 ) + x ( n - 2 ) y ( n ) = x ( n ) - x ( n - 1 ) + x ( n - 2 )
(17)

Calculate the theoretical autocorrelation of YnYn using Equation 15 and Equation 16. Show all of your work.

Generate 1000 independent samples of a Gaussian random variable XX with mean 0 and variance 1. Filter the samples using Equation 17. We will denote the filtered signal YiYi, i=1,2,,1000i=1,2,,1000.

Draw 4 scatter plots using the form subplot(2,2,n), (n=1,2,3,4)(n=1,2,3,4). The first scatter plot should consist of points, (Yi,Yi+1)(Yi,Yi+1), (i=1,2,,900)(i=1,2,,900). Notice that this correlates samples that are separated by a lag of “1”. The other 3 scatter plots should consist of the points (Yi,Yi+2)(Yi,Yi+2), (Yi,Yi+3)(Yi,Yi+3), (Yi,Yi+4)(Yi,Yi+4), (i=1,2,,900)(i=1,2,,900), respectively. What can you deduce about the random process from these scatter plots?

For real applications, the theoretical autocorrelation may be unknown. Therefore, rYY(m)rYY(m) may be estimated by the sample autocorrelation, rYY'(m)rYY'(m) defined by

r Y Y ' ( m ) = 1 N - | m | n = 0 N - | m | - 1 Y ( n ) Y ( n + | m | ) - ( N - 1 ) m N - 1 r Y Y ' ( m ) = 1 N - | m | n = 0 N - | m | - 1 Y ( n ) Y ( n + | m | ) - ( N - 1 ) m N - 1
(18)

where NN is the number of samples of YY.

Use Matlab to calculate the sample autocorrelation of YnYn using Equation 18. Plot both the theoretical autocorrelation rYY(m)rYY(m), and the sample autocorrelation rYY'(m)rYY'(m) versus mm for -20m20-20m20. Use subplot to place them in the same figure. Does Equation 18 produce a reasonable approximation of the true autocorrelation?

INLAB REPORT

For the filter in Equation 17,

  1. Show your derivation of the theoretical output autocorrelation, rYY(m)rYY(m).
  2. Hand in the four scatter plots. Label each plot with the corresponding theoretical correlation, from rYY(m)rYY(m). What can you conclude about the output random process from these plots?
  3. Hand in your plots of rYY(m)rYY(m) and rYY'(m)rYY'(m) versus mm. Does Equation 18 produce a reasonable approximation of the true autocorrelation? For what value of mm does rYY(m)rYY(m) reach its maximum? For what value of mm does rYY'(m)rYY'(m) reach its maximum?
  4. Hand in your Matlab code.

Correlation of Two Random Processes

Background

The cross-correlation is a function used to describe the correlation between two separate random processes. If XX and YY are jointly WSS random processes, the cross-correlation is defined by

c X Y ( m ) = E [ X n Y n + m ] m = , - 1 , 0 , 1 , . c X Y ( m ) = E [ X n Y n + m ] m = , - 1 , 0 , 1 , .
(19)

Similar to the definition of the sample autocorrelation introduced in the previous section, we can define the sample cross-correlation for a pair of data sets. The sample cross-correlation between two finite random sequences XnXn and YnYn is defined as

c X Y ' ( m ) = 1 N - m n = 0 N - m - 1 X ( n ) Y ( n + m ) 0 m N - 1 c X Y ' ( m ) = 1 N - m n = 0 N - m - 1 X ( n ) Y ( n + m ) 0 m N - 1
(20)
c X Y ' ( m ) = 1 N - | m | n = | m | N - 1 X ( n ) Y ( n + m ) 1 - N m < 0 c X Y ' ( m ) = 1 N - | m | n = | m | N - 1 X ( n ) Y ( n + m ) 1 - N m < 0
(21)

where NN is the number of samples in each sequence. Notice that the cross-correlation is not an even function of mm. Hence a two-sided definition is required.

Cross-correlation of signals is often used in applications of sonar and radar, for example to estimate the distance to a target. In a basic radar set-up, a zero-mean signal X(n)X(n) is transmitted, which then reflects off a target after traveling for D/2D/2 seconds. The reflected signal is received, amplified, and then digitized to form Y(n)Y(n). If we summarize the attenuation and amplification of the received signal by the constant αα, then

Y ( n ) = α X ( n - D ) + W ( n ) Y ( n ) = α X ( n - D ) + W ( n )
(22)

where W(n)W(n) is additive noise from the environment and receiver electronics.

In order to compute the distance to the target, we must estimate the delay DD. We can do this using the cross-correlation. The cross-correlation cXYcXY can be calculated by substituting Equation 22 into Equation 19.

c X Y ( m ) = E [ X ( n ) Y ( n + m ) ] = E [ X ( n ) ( α X ( n - D + m ) + W ( n + m ) ) ] = α E [ X ( n ) X ( n - D + m ) ] + E [ X ( n ) ] E [ W ( n + m ) ] = α E [ X ( n ) X ( n - D + m ) ] c X Y ( m ) = E [ X ( n ) Y ( n + m ) ] = E [ X ( n ) ( α X ( n - D + m ) + W ( n + m ) ) ] = α E [ X ( n ) X ( n - D + m ) ] + E [ X ( n ) ] E [ W ( n + m ) ] = α E [ X ( n ) X ( n - D + m ) ]
(23)

Here we have used the assumptions that X(n)X(n) and W(n+m)W(n+m) are uncorrelated and zero-mean. By applying the definition of autocorrelation, we see that

c X Y ( m ) = α r X X ( m - D ) c X Y ( m ) = α r X X ( m - D )
(24)

Because rXX(m-D)rXX(m-D) reaches its maximum when m=Dm=D, we can find the delay DD by searching for a peak in the cross correlation cXY(m)cXY(m). Usually the transmitted signal X(n)X(n) is designed so that rXX(m)rXX(m) has a large peak at m=0m=0.

Experiment

Download the file radar.mat for the following section.

Using Equation 20 and Equation 21, write a Matlab function C=CorR(X,Y,m) to compute the sample cross-correlation between two discrete-time random processes, XX and YY, for a single lag value mm.

To test your function, generate two length 1000 sequences of zero-mean Gaussian random variables, denoted as XnXn and ZnZn. Then compute the new sequence Yn=Xn+ZnYn=Xn+Zn. Use CorR to calculate the sample cross-correlation between XX and YY for lags -10m10-10m10. Plot your cross-correlation function.

INLAB REPORT

  1. Submit your plot for the cross-correlation between XX and YY. Label the mm-axis with the corresponding lag values.
  2. Which value of mm produces the largest cross-correlation? Why?
  3. Is the cross-correlation function an even function of mm? Why or why not?
  4. Hand in the code for your CorR function.

Next we will do an experiment to illustrate how cross-correlation can be used to measure time delay in radar applications. Down load the MAT file radar.mat and load it using the command load radar. The vectors trans and received contain two signals corresponding to the transmitted and received signals for a radar system. First compute the autocorrelation of the signal trans for the lags -100m100-100m100. (Hint: Use your CorR function.)

Next, compute the sample cross-correlation between the signal trans and received for the range of lag values -100m100-100m100, using your CorRCorR function. Determine the delay DD.

INLAB REPORT

  1. Plot the transmitted signal and the received signal on a single figure using subplot. Can you estimate the delay DD by a visual inspection of the received signal?
  2. Plot the sample autocorrelation of the transmitted signal, rXX'(m)rXX'(m) vs. mm for -100m100-100m100.
  3. Plot the sample cross-correlation of the transmitted signal and the received signal, cXY'(m)cXY'(m) vs. mm for -100m100-100m100.
  4. Determine the delay DD from the sample correlation. How did you determine this?

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks