Digital Signal Processing Digital ≡ sampled, discrete-time, quantized Signal ≡ waveform, sequnce of measurements or observations Processing ≡ analyze, modify, filter, synthesize

Examples of Digital Signals sampled speech waveform "pixelized" image Dow-Jones Index

DSP Applications Filtering (noise reduction) Pattern recognition (speech, faces, fingerprints) Compression

A Major Difficulty In many (perhaps most) DSP applications we don't have complete or perfect knowledge of the signals we wish to process. We are faced with many unknowns and uncertainties.

Examples noisy measurements unknown signal parameters noisy system or environmental conditions natural variability in the signals encountered

Functional Magnetic Resonance Imaging Challenges are measurement noise and intrinsic uncertainties in signal behavior. How can we design signal processing algorithms in the face of such uncertainty? Can we model the uncertainty and incorporate this model into the design process? Statistical signal processing is the study of these questions.

Modeling Uncertainty The most widely accepted and commonly used approach to modeling uncertainty is Probability Theory (although other alternatives exist such as Fuzzy Logic). Probability Theory models uncertainty by specifying the chance of observing certain signals. Alternatively, one can view probability as specifying the degree to which we believe a signal reflects the true state of nature.

Examples of Probabilistic Models errors in a measurement (due to an imprecise measuring device) modeled as realizations of a Gaussian random variable. uncertainty in the phase of a sinusoidal signal modeled as a uniform random variable on 0 2 . uncertainty in the number of photons stiking a CCD per unit time modeled as a Poisson random variable.

Statistical Inference A statistic is a function of observed data. Suppose we observe N scalar values x 1 , … , x N . The following are statistics: x 1 N n 1 N x n (sample mean) x 1 , … , x N (the data itself) x 1 … x N (order statistic) ( x 1 2 x 2 x 3 , x 1 x 3 ) A statistic cannot depend on unknown parameters. Probability is used to model uncertainty. Statistics are used to draw conclusions about probability models.

Probability models our uncertainty about signals we may observe. Statistics reasons from the measured signal to the population of possible signals.

Statistical Signal Processing Step 1 Postulate a probability model (or models) that reasonably capture the uncertainties at hand Step 2 Collect data Step 3 Formulate statistics that allow us to interpret or understand our probability model(s)

In this class The two major kinds of problems that we will study are detection and estimation. Most SSP problems fall under one of these two headings.

Detection Theory Given two (or more) probability models, which on best explains the signal?

Examples Decode wireless comm signal into string of 0's and 1's Pattern recognition voice recognition face recognition handwritten character recognition Anomaly detection radar, sonar irregular, heartbeat gamma-ray burst in deep space

Estimation Theory If our probability model has free parameters, what are the best parameter settings to describe the signal we've observed?

Examples Noise reduction Determine parameters of a sinusoid (phase, amplitude, frequency) Adaptive filtering track trajectories of space-craft automatic control systems channel equalization Determine location of a submarine (sonar) Seismology: estimate depth below ground of an oil deposit

Detection Example Suppose we observe N tosses of an unfair coin. We would like to decide which side the coin favors, heads or tails. Step 1 Assume each toss is a realization of a Bernoulli random variable. Heads p 1 Tails Must decide p 1 4 vs. p 3 4 . Step 2 Collect data x 1 , … , x N x i 1 Heads x i 0 Tails Step 3 Formulate a useful statistic k n 1 N x n If k N 2 , guess p 1 4 . If k N 2 , guess p 3 4 . Estimation Example Suppose we take N measurements of a DC voltage A with a noisy voltmeter. We would like to estimate A. Step 1 Assume a Gaussian noise model x n A w n where w n 0 1 . Step 2 Gather data x 1 , … , x N Step 3 Compute the sample mean, A 1 N n 1 N x n and use this as an estimate. In these examples ( and ), we solved detection and estimation problems using intuition and heuristics (in Step 3). This course will focus on developing principled and mathematically rigorous approaches to detection and estimation, using the theoretical framework of probability and statistics.

Summary DSP ≡ processing signals with computer algorithms. SSP ≡ statistical DSP ≡ processing in the presence of uncertainties and unknowns.