Connexions

You are here: Home » Content » Statistical Signal Processing » Wiener Filtering and the DFT

Recently Viewed

This feature requires Javascript to be enabled.

Inside Collection (Course):

Course by: Clayton Scott. E-mail the author

Wiener Filtering and the DFT

Module by: Clayton Scott, Robert Nowak. E-mail the authors

Connecting the Vector Space and Classical Wiener Filters

Suppose we observe x=y+w x y w which are all N×1 N 1 vectors and where w𝒩0σ2I w 0 σ 2 I . Given xx we wish to estimate yy. Think of yy as a signal in additive white noise ww. xx is a noisy observation of the signal.

Taking a Bayesian approach, put a prior on the signal yy: y𝒩0 R yy y 0 R yy which is independent of noise ww. The minimum MSE (MMSE) estimator is y ^= R yx R xx -1x y R yx R xx x Under the modeling assumptions above

R yx =Ey(y+w)T=EyyT+EywT=EyyT= R yy R yx y y w y y y w y y R yy
(1)
since EywT=0 y w 0 and since yy and ww are zero-mean and independent.
R xx =ExxT=E(y+w)(y+w)T=EyyT+EywT+EwyT+EwwT= R yy + R ww R xx x x y w y w y y y w w y w w R yy R ww
(2)
since EwwT= R ww w w R ww . Hence y ^= R yy R yy + R ww -1x= H opt x y R yy R yy R ww x H opt x Where H opt H opt is the Wiener filter. Recall the frequency domain case H opt f= S yy f S yy f+ S ww f H opt f S yy f S yy f S ww f Now let's look at an actual problem scenario. Suppose that we know a priori that the signal yy is smooth or lowpass. We can incorporate this prior knowledge by carefully choosing the prior covariance R yy R yy .

Recall the DFT 𝒴 k =1N n =0N Y k e(i2πknN)  ,   k= 0 , , N - 1    k k 0 , , N - 1 𝒴 k 1 N n 0 N Y k 2 k n N or in vector notation 𝒴 k =y, u k   ,   k= 0 , , N - 1    k k 0 , , N - 1 𝒴 k y u k where u k =( 1ei2πkNei2π2kNei2π(N1)kN )HN u k 1 2 k N 2 2 k N 2 N 1 k N N (H dehotes Hermitian transpose)

Note:

u k , u k = u k H u k =1 u k u k u k u k 1 , u k , u l = u k H u l =0  ,   kl    k l k l u k u l u k u l 0 , i.e., u k   ,   k= 0 , , N - 1    k k 0 , , N - 1 u k is an orthonormal basis.
The vector u k u k spans the subspace corresponding to a frequency band centered at frequency f k =2πkN f k 2 k N ("digital" frequency on 0 1 0 1 ). If we know that yy is lowpass, then Ey, u k 2=E 𝒴 k 2 y u k 2 𝒴 k 2 should be relatively small (compared to Ey, u 0 2 y u 0 2 ) for high frequencies.

Let σ k 2=Ey, u k 2 σ k 2 y u k 2 A lowpass model implies σ 0 2> σ 1 2>> σ N 2 2 σ 0 2 σ 1 2 σ N 2 2 , assuming NN even, and conjugate symmetry implies σ N - j 2= σ j 2  ,   j= 1 , , N 2    j j 1 , , N 2 σ N - j 2 σ j 2 Furthermore, let's model the DFT coefficients as zero-mean and independent E 𝒴 k =0 𝒴 k 0 E 𝒴 k 𝒴 l ¯={ σ k 2  if  l=k0  if  lk 𝒴 k 𝒴 l σ k 2 l k 0 l k This completely specifies our prior y𝒩0 R yy y 0 R yy R yy =UDU¯T R yy U D U where D=( σ 0 200 0 σ 1 20 00 σ N - 1 2 ) D σ 0 2 0 0 0 σ 1 2 0 0 0 σ N - 1 2 and U=( u 0 u 1 u N - 1 ) U u 0 u 1 u N - 1

Note:

𝒴=UHy 𝒴 U y is the DFT and y=U𝒴 y U 𝒴 is the inverse DFT.
With this prior on yy the Wiener filter is y ^=UDUHUDUH+σ2I-1x y U D U U D U σ 2 I x Since UU is a unitary matrix UUH=I U U I and therefore
y ^=UDUHU(D+σ2I)UH-1x=UDUHUD+σ2I-1UHx=UDD+σ2I-1UHx y U D U U D σ 2 I U x U D U U D σ 2 I U x U D D σ 2 I U x
(3)
1 Now take the DFT of both sides 𝒴 ^=UH y ^=DD+σ2I-1𝒳 𝒴 U y D D σ 2 I 𝒳 where 𝒳=UHx 𝒳 U x and is the DFT of xx. Both DD and D+σ2I D σ 2 I are diagonal so 𝒴^k=dk,kdk,k+σ2 𝒳 k = σ k 2 σ k 2+σ2 𝒳 k 𝒴 k d k k d k k σ 2 𝒳 k σ k 2 σ k 2 σ 2 𝒳 k Hence the Wiener filter is a frequency (DFT) domain filter 𝒴^k= H k 𝒳 k 𝒴 k H k 𝒳 k where 𝒳 k 𝒳 k is the k th k th DFT coefficient of x x and the filter response at digital frequency 2πkN 2 k N is H k = σ k 2 σ k 2+σ2 H k σ k 2 σ k 2 σ 2 Assuming σ 0 2> σ 1 2>> σ N 2 2 σ 0 2 σ 1 2 σ N 2 2 and σ N - j 2= σ j 2  ,   j= 1 , , N 2    j j 1 , , N 2 σ N - j 2 σ j 2 . The filter's response is a digital lowpass filter!

Summary of Wiener Filter

Problem: Observe x=y+w x y w

Recover/estimate signal yy.

Classical Wiener Filter (continuous-time): Hω= S yy ω S yy ω+ S ww ω H ω S yy ω S yy ω S ww ω where yt y t and wt w t are stationary processes.

Vector Space Wiener Filter: H= R yy R yy + R ww -1 H R yy R yy R ww

Wiener Filter and DFT: ( R ww =σ2I R ww σ 2 I ). If R yy =UDUH R yy U D U , where U U is DFT, then H H is a discrete-time filter whose DFT is given by

H k = n =0N1 h n ei2πkN=dk,kdk,k+σ2 H k n 0 N 1 h n 2 k N d k k d k k σ 2
(4)
Here, dk,k d k k plays the same role as S yy ω S yy ω .

Footnotes

1. If A A, B B, C C are all invertible, compatible matrices, then ABC-1=C-1B-1A-1 A B C C B A . U-1=UH U U , UH-1=U U U .

Content actions

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks