Connecting the Vector Space and Classical Wiener Filters

Suppose we observe x=y+w x y w which are all N×1 N 1 vectors and where w∼0σ2I w 0 σ 2 I . Given xx we wish to estimate yy. Think of yy as a signal in additive white noise ww. xx is a noisy observation of the signal.

Taking a Bayesian approach, put a prior on the signal yy: y∼0 R yy y 0 R yy which is independent of noise ww. The minimum MSE (MMSE) estimator is y ̂= R yx R xx -1x y R yx R xx x Under the modeling assumptions above

since EywT=0 y w 0 and since yy and ww are zero-mean and independent. since EwwT= R ww w w R ww . Hence y ̂= R yy R yy + R ww -1x= H opt x y R yy R yy R ww x H opt x Where H opt H opt is the Wiener filter. Recall the frequency domain case H opt f= S yy f S yy f+ S ww f H opt f S yy f S yy f S ww f Now let's look at an actual problem scenario. Suppose that we know a priori that the signal yy is smooth or lowpass. We can incorporate this prior knowledge by carefully choosing the prior covariance R yy R yy .

Recall the DFT ∀k,k= 0 , … , N - 1 : 𝒴 k =1N∑n=0N Y k ⅇ-ⅈ2πknN k k 0 , … , N - 1 𝒴 k 1 N n 0 N Y k 2 k n N or in vector notation ∀k,k= 0 , … , N - 1 : 𝒴 k =<y,uk> k k 0 , … , N - 1 𝒴 k y u k where uk=1ⅇⅈ2πkNⅇⅈ2π2kN…ⅇⅈ2πN-1kNHN u k 1 2 k N 2 2 k N … 2 N 1 k N N (H dehotes Hermitian transpose)

The vector uk u k spans the subspace corresponding to a frequency band centered at frequency f k =2πkN f k 2 k N ("digital" frequency on 01 0 1 ). If we know that yy is lowpass, then E∥<y,uk>∥2=E∥ 𝒴 k ∥2 y u k 2 𝒴 k 2 should be relatively small (compared to E∥<y,u0>∥2 y u 0 2 ) for high frequencies.

Let σ k 2=E∥<y,uk>∥2 σ k 2 y u k 2 A lowpass model implies σ 0 2> σ 1 2>…> σ N 2 2 σ 0 2 σ 1 2 … σ N 2 2 , assuming NN even, and conjugate symmetry implies ∀j,j= 1 , … , N 2 : σ N - j 2= σ j 2 j j 1 , … , N 2 σ N - j 2 σ j 2 Furthermore, let's model the DFT coefficients as zero-mean and independent E 𝒴 k =0 𝒴 k 0 E 𝒴 k 𝒴 l ¯= σ k 2ifl=k0ifl≠k 𝒴 k 𝒴 l σ k 2 l k 0 l k This completely specifies our prior y∼0 R yy y 0 R yy R yy =UDU¯T R yy U D U where D= σ 0 20…00 σ 1 2…0⋮⋮⋱⋮00… σ N - 1 2 D σ 0 2 0 … 0 0 σ 1 2 … 0 ⋮ ⋮ ⋱ ⋮ 0 0 … σ N - 1 2 and U=u0u1…u N - 1 U u 0 u 1 … u N - 1

With this prior on yy the Wiener filter is y ̂=UDUHUDUH+σ2I-1x y U D U U D U σ 2 I x Since UU is a unitary matrix UUH=I U U I and therefore 1 Now take the DFT of both sides 𝒴 ̂=UH y ̂=DD+σ2I-1𝒳 𝒴 U y D D σ 2 I 𝒳 where 𝒳=UHx 𝒳 U x and is the DFT of xx. Both DD and D+σ2I D σ 2 I are diagonal so 𝒴̂k=dkkdkk+σ2 𝒳 k = σ k 2 σ k 2+σ2 𝒳 k 𝒴 k d k k d k k σ 2 𝒳 k σ k 2 σ k 2 σ 2 𝒳 k Hence the Wiener filter is a frequency (DFT) domain filter 𝒴̂k= H k 𝒳 k 𝒴 k H k 𝒳 k where 𝒳 k 𝒳 k is the k th k th DFT coefficient of x x and the filter response at digital frequency 2πkN 2 k N is H k = σ k 2 σ k 2+σ2 H k σ k 2 σ k 2 σ 2 Assuming σ 0 2> σ 1 2>…> σ N 2 2 σ 0 2 σ 1 2 … σ N 2 2 and ∀j,j= 1 , … , N 2 : σ N - j 2= σ j 2 j j 1 , … , N 2 σ N - j 2 σ j 2 . The filter's response is a digital lowpass filter!

Summary of Wiener Filter

Problem: Observe x=y+w x y w

Recover/estimate signal yy.

Classical Wiener Filter (continuous-time): Hω= S yy ω S yy ω+ S ww ω H ω S yy ω S yy ω S ww ω where yt y t and wt w t are stationary processes.

Vector Space Wiener Filter: H= R yy R yy + R ww -1 H R yy R yy R ww

Wiener Filter and DFT: ( R ww =σ2I R ww σ 2 I ). If R yy =UDUH R yy U D U , where U U is DFT, then H H is a discrete-time filter whose DFT is given by

Here, dkk d k k plays the same role as S yy ω S yy ω .