Introduction to statistical signal processing

Digital signal processing

• Digitalsampled, discrete-time, quantized
• Signalwaveform, sequnce of measurements or observations
• Processinganalyze, modify, filter, synthesize

Examples of digital signals

• sampled speech waveform
• "pixelized" image
• Dow-Jones Index

Dsp applications

• Filtering (noise reduction)
• Pattern recognition (speech, faces, fingerprints)
• Compression

A major difficulty

In many (perhaps most) DSP applications we don't have complete or perfect knowledge of the signals we wishto process. We are faced with many unknowns and uncertainties .

Examples

• noisy measurements
• unknown signal parameters
• noisy system or environmental conditions
• natural variability in the signals encountered

How can we design signal processing algorithms in the face of such uncertainty?

Can we model the uncertainty and incorporate this model into the design process?

Statistical signal processing is the study of these questions.

Modeling uncertainty

The most widely accepted and commonly used approach to modeling uncertainty is Probability Theory (although other alternatives exist such as Fuzzy Logic).

Probability Theory models uncertainty by specifying the chance of observing certain signals.

Alternatively, one can view probability as specifying the degree to which we believe a signal reflects the true state of nature .

Examples of probabilistic models

• errors in a measurement (due to an imprecise measuring device) modeled as realizations of a Gaussian randomvariable.
• uncertainty in the phase of a sinusoidal signal modeled as a uniform random variable on $\left[0 , 2\pi \right)$ .
• uncertainty in the number of photons stiking a CCD per unit time modeled as a Poisson random variable.

Statistical inference

A statistic is a function of observed data.

Probability is used to model uncertainty.

Statistics are used to draw conclusions about probability models.

Probability models our uncertainty about signals we may observe.

Statistics reasons from the measured signal to the population of possible signals.

Statistical signal processing

• Step 1

  Postulate a probability model (or models) that reasonably capture the uncertainties at hand

• Step 2

  Collect data

• Step 3

  Formulate statistics that allow us to interpret or understand our probability model(s)

In this class

The two major kinds of problems that we will study are detection and estimation . Most SSP problems fall under one of these two headings.

Detection theory

Given two (or more) probability models, which on best explains the signal?

Examples

• Decode wireless comm signal into string of 0's and 1's
• Pattern recognition
  • voice recognition
  • face recognition
  • handwritten character recognition
• Anomaly detection
  • radar, sonar
  • irregular, heartbeat
  • gamma-ray burst in deep space

Estimation theory

If our probability model has free parameters, what are the best parameter settings to describe the signalwe've observed?

Examples

• Noise reduction
• Determine parameters of a sinusoid (phase, amplitude, frequency)
• Adaptive filtering
  • track trajectories of space-craft
  • automatic control systems
  • channel equalization
• Determine location of a submarine (sonar)
• Seismology: estimate depth below ground of an oil deposit

In these examples ( and ), we solved detection and estimation problems using intuition and heuristics (in Step 3).

This course will focus on developing principled and mathematically rigorous approaches to detection and estimation,using the theoretical framework of probability and statistics.

Summary

• DSPprocessing signals with computer algorithms.
• SSPstatistical DSPprocessing in the presence of uncertainties and unknowns.