Suppose we observe x=y+w x y w which are all N×1 N 1 vectors and where w∼𝒩0σ2⁢I w 0 σ 2 I . Given xx we wish to estimate yy. Think of yy as a signal in additive white noise ww. xx is a noisy observation of the signal.

Taking a Bayesian approach, put a prior on the signal yy: y∼𝒩0 R yy y 0 R yy which is independent of noise ww. The minimum MSE (MMSE) estimator is y ^= R yx ⁢ R xx -1⁢x y R yx R xx x Under the modeling assumptions above

since Ey⁢wT=0 y w 0 and since yy and ww are zero-mean and independent. since Ew⁢wT= R ww w w R ww . Hence y ^= R yy ⁢ R yy + R ww -1⁢x= H opt ⁢x y R yy R yy R ww x H opt x Where H opt H opt is the Wiener filter. Recall the frequency domain case H opt ⁢f= S yy ⁢f S yy ⁢f+ S ww ⁢f H opt f S yy f S yy f S ww f Now let's look at an actual problem scenario. Suppose that we know a priori that the signal yy is smooth or lowpass. We can incorporate this prior knowledge by carefully choosing the prior covariance R yy R yy .

Recall the DFT ∀ k ,k= 0 , … , N - 1 : 𝒴 k =1N⁢∑ n =0N Y k ⁢e−(i⁢2⁢π⁢k⁢nN) k k 0 , … , N - 1 𝒴 k 1 N n 0 N Y k 2 k n N or in vector notation ∀ k ,k= 0 , … , N - 1 : 𝒴 k =〈y, u k 〉 k k 0 , … , N - 1 𝒴 k y u k where u k =( 1ei⁢2⁢π⁢kNei⁢2⁢π⁢2⁢kN…ei⁢2⁢π⁢(N−1)⁢kN )HN u k 1 2 k N 2 2 k N … 2 N 1 k N N (H dehotes Hermitian transpose)

The vector u k u k spans the subspace corresponding to a frequency band centered at frequency f k =2⁢π⁢kN f k 2 k N ("digital" frequency on 0 1 0 1 ). If we know that yy is lowpass, then E∥〈y, u k 〉∥2=E∥ 𝒴 k ∥2 y u k 2 𝒴 k 2 should be relatively small (compared to E∥〈y, u 0 〉∥2 y u 0 2 ) for high frequencies.

Let σ k 2=E∥〈y, u k 〉∥2 σ k 2 y u k 2 A lowpass model implies σ 0 2> σ 1 2>…> σ N 2 2 σ 0 2 σ 1 2 … σ N 2 2 , assuming NN even, and conjugate symmetry implies ∀ j ,j= 1 , … , N 2 : σ N - j 2= σ j 2 j j 1 , … , N 2 σ N - j 2 σ j 2 Furthermore, let's model the DFT coefficients as zero-mean and independent E 𝒴 k =0 𝒴 k 0 E 𝒴 k ⁢ 𝒴 l ¯={ σ k 2  if  l=k0  if  l≠k 𝒴 k 𝒴 l σ k 2 l k 0 l k This completely specifies our prior y∼𝒩0 R yy y 0 R yy R yy =U⁢D⁢U¯T R yy U D U where D=( σ 0 20…0 0 σ 1 2…0 ⋮⋮⋱⋮ 00… σ N - 1 2 ) D σ 0 2 0 … 0 0 σ 1 2 … 0 ⋮ ⋮ ⋱ ⋮ 0 0 … σ N - 1 2 and U=( u 0 u 1 … u N - 1 ) U u 0 u 1 … u N - 1

With this prior on yy the Wiener filter is y ^=U⁢D⁢UH⁢U⁢D⁢UH+σ2⁢I-1⁢x y U D U U D U σ 2 I x Since UU is a unitary matrix U⁢UH=I U U I and therefore 1 Now take the DFT of both sides 𝒴 ^=UH⁢ y ^=D⁢D+σ2⁢I-1⁢𝒳 𝒴 U y D D σ 2 I 𝒳 where 𝒳=UH⁢x 𝒳 U x and is the DFT of xx. Both DD and D+σ2⁢I D σ 2 I are diagonal so 𝒴^k=dk,kdk,k+σ2⁢ 𝒳 k = σ k 2 σ k 2+σ2⁢ 𝒳 k 𝒴 k d k k d k k σ 2 𝒳 k σ k 2 σ k 2 σ 2 𝒳 k Hence the Wiener filter is a frequency (DFT) domain filter 𝒴^k= H k ⁢ 𝒳 k 𝒴 k H k 𝒳 k where 𝒳 k 𝒳 k is the k th k th DFT coefficient of x x and the filter response at digital frequency 2⁢π⁢kN 2 k N is H k = σ k 2 σ k 2+σ2 H k σ k 2 σ k 2 σ 2 Assuming σ 0 2> σ 1 2>…> σ N 2 2 σ 0 2 σ 1 2 … σ N 2 2 and ∀ j ,j= 1 , … , N 2 : σ N - j 2= σ j 2 j j 1 , … , N 2 σ N - j 2 σ j 2 . The filter's response is a digital lowpass filter!

Problem: Observe x=y+w x y w

Recover/estimate signal yy.

Classical Wiener Filter (continuous-time): H⁢ω= S yy ⁢ω S yy ⁢ω+ S ww ⁢ω H ω S yy ω S yy ω S ww ω where y⁢t y t and w⁢t w t are stationary processes.

Vector Space Wiener Filter: H= R yy ⁢ R yy + R ww -1 H R yy R yy R ww

Wiener Filter and DFT: ( R ww =σ2⁢I R ww σ 2 I ). If R yy =U⁢D⁢UH R yy U D U , where U U is DFT, then H H is a discrete-time filter whose DFT is given by

Here, dk,k d k k plays the same role as S yy ⁢ω S yy ω .