Consider the following two-model evaluation problem (van Trees; prob.2.2.1). ℳ 0 :   r=n ℳ 0 :   r n ℳ 1 :   r=s+n ℳ 1 :   r s n where ss and nn are statistically independent, positively valued, random variables having the densities pss=aⅇ-as p s s a a s and pnn=bⅇ-bn p n n b b n

The two models describe different equi-variance statistical models for the observations (van Trees; Prob. 2.2.11). ℳ 0 :   prr=12ⅇ-2|r| ℳ 0 :   p r r 1 2 2 r ℳ 1 :   prr=12πⅇ-12r2 ℳ 1 :   p r r 1 2 1 2 r 2

A hypothesis testing criterion radically different from those discussed in this section and this section is minimum equivocation. In this information theoretic approach, the two-model testing problem is modeled as a digital channel, shown in this figure. The channel's inputs, generically represented by the x x, are the models and the channel's ouputs, denoted by y y, are the decisions.

The quality of such information theoretic channels is quantified by the mutual information ℐx;y x y defined to be the difference between the entropy of the inputs and the equivocation (Cover and Thomas; sections 2.3, 2.4). ℐx;y=Hx−H x | y x y H x H x | y Hx=-∑iP x i logP x i H x i i P x i P x i H x | y =-∑ijP x i y j logP x i y j P y j H x | y i j i j P x i y j P x i y j P y j Here, P x i P x i denotes the a priori probabilities, P y j P y j the output probabilities, and P x i y j P x i y j the joint probability of input x i x i resulting in output y j y j . For example, P x 0 y 0 =P x 0 1− P F P x 0 y 0 P x 0 1 P F and P y 0 =P x 0 1− P F +P x 1 1− P D P y 0 P x 0 1 P F P x 1 1 P D . For a fixed set of a priori probabilities, show that the decision rule that maximizes the mutual information is the likelihood ratio test. What is the threshold when this criterion is employed?

Non-Gaussian statistical models sometimes yield surprising results in comparison to Gaussian ones. Consider the following hypothesis testing problem where the observations have a Laplacian probability distribution. ℳ 0 :   prr=12ⅇ-|r+m| ℳ 0 :   p r r 1 2 r m ℳ 1 :   prr=12ⅇ-|r−m| ℳ 1 :   p r r 1 2 r m

Developing a Neyman-Pearson decision rule for more than two models has not been detailed. Assume K K distinct models are required to account for the observations. We seek to maximize the probability of correctly announcing ℳ i ℳ i under the constraint that the probability of announcing ℳ i ℳ i when model ℳ 0 ℳ 0 was indeed true does not exceed a specified value.

Pattern recognition relies heavily on ideas derived from the principles of statistical model testing. Measurements are made of a test object and these are compared with those of "standard" objects to determine which the test object most closely resembles. Assume that the measurement vector r r is jointly Gaussian with mean m i m i ( i∈1…K i 1 … K ) and covariance matrix σ2I σ 2 I (i.e., statistically independent components). Thus there are K K possible objects, each having an "ideal" measurement vector mi m i and probability P i P i of being present.

Define y y to be y=∑k=0L x k y k 0 L x k where the x k x k are statistically independent random variables, each having a Gaussian density 0σ2 0 σ 2 . The number L L of variables in the sum is a random variable with a Poisson distribution. PrL=l=λll!ⅇ-λ L l λ l l λ where l∈01… l 0 1 …

Based upon the observation of y y, we want to decide whether L≤1 L 1 or L>1 L 1 . Write an expression for the minimum P e P e .

One observation of the random variable r r is obtained. This random variable is either uniformly distributed between -1 and +1 or expressed as the sum of statistically independent random variables, each of which is also uniformly distributed between -1 and +1.

The observed random variable r r has a Gaussian density on each of five models. pr| ℳ i r=12πσⅇ-r− m i 22σ2 p r ℳ i r 1 2 σ r m i 2 2 σ 2 for i∈12…5 i 1 2 … 5 , where m 1 =-2m m 1 -2 m , m 2 =-m m 2 m , m 3 =0 m 3 0 , m 4 =m m 4 m , m 5 =2m m 5 2 m . The models are equally likely and the criterion of the test is to minimize P e P e .

The goal is to choose which of the following four models is true upong the reception of the three-dimensional vector r r (van Trees; Prob. 2.6.6). ℳ 0 :   r=m0+n ℳ 0 :   r m 0 n ℳ 1 :   r=m1+n ℳ 1 :   r m 1 n ℳ 2 :   r=m2+n ℳ 2 :   r m 2 n ℳ 3 :   r=m3+n ℳ 3 :   r m 3 n where m0=a0b ,   m1=0ab ,   m2=-a0b ,   m3=0-ab m 0 a 0 b ,   m 1 0 a b ,   m 2 a 0 b ,   m 3 0 a b The noise vector n n is a Gaussian random vector having statistically independent, identically distributed components, each of which has zero mean and variance σ2 σ 2 . We have L L independent observations of the received vector r r.

To gain some appreciation of some of the issues in implementing a detector, this problem asks you to program (preferably in Matlab) a simple detector and numerically compare its performance with theoretical predictions. Let the observations consist of a signal contained in additive Gaussian white noise. ℳ 0 :   rl=nl    l∈0…L−1 ℳ 0 :   r l n l    l 0 … L 1 ℳ 1 :   r=Asin2πlL+nl    l∈0…L−1 ℳ 1 :   r A 2 l L n l    l 0 … L 1 The variance of each noise value equals σ2 σ 2 .

Calculate the Kullback-Leibler distance between the following pairs of densities. Use these results to find the Fisher information for the mean parameter m m.

The Kullback-Leibler and Chernoff distances can be related to the Fisher information matrix. Let p r ; θ r ; θ p r ; θ r ; θ be a probability density that depends on the parameter vector θ θ We want to consider the distance between probability densities that differ by a perturbation δθ δ θ in their parameter vectors.

Insights into certain detection problems can be gained by examining the Kullback-Leibler distance and the properties of Fisher information. We begin by first showing that the Gaussian distribution has the smallest Fisher information for the mean parameter for all differentiable distributions having the same variance.

Find the Chernoff distance between the following distributions.

Let's explore how well Stein's Lemma predicts optimal detector performance probabilities. Consider the two-model detection problem wherein L L statistically independent, identically distributed Gaussian random variables are observed. Under ℳ 0 ℳ 0 , the mean is zero and the variance one; under ℳ 1 ℳ 1 , the mean is one and the variance one.

Figure 1: A binary symmetric digital communications channel.

We observe a Gaussian random variable r r. This random variable has zero mean under model ℳ 0 ℳ 0 and mean m m under ℳ 1 ℳ 1 . The variance of r r in either instance is σ2 σ 2 . The models are equally likely.

The optimum reception of binary information can be viewed as a model testing problem. Here, equally-likely binary data (a "zero" or a "one") is transmitted through a binary symmetric channel. The indicated parameters denote the probabilities of receiving a binary digit given that a particular digit was sent. Assume that ε=0.1 ε 0.1 .

You have accepted the (dangerous) job of determining whether the radioactivity levels at the Chernobyl reactor are elevated or not. Because you want to stay as short a time as possible to render you professional opinion, you decide to use a sequential-like decision rule. Radioactivity is governed by Poisson probability laws, which means that the probability that n n counts are observed in T T seconds equals Prn=λTnⅇ-λTn! n λ T n λ T n where λ λ is the radiation intensity. Safe radioactivity levels occur when λ= λ 0 λ λ 0 and unsafe ones at λ= λ 1 λ λ 1 , λ 1 > λ 0 λ 1 λ 0 .

Sequential tests can be used to advantage in situations where analytic difficulties obscure the problem. Consider the case where the observations either contain no signal ( ℳ 0 ℳ 0 ) or a signal whose components are randomly set to zero. ℳ 0 :   r l = n l ℳ 0 :   r l n l ℳ 1 :   r l = a l + s l ℳ 1 :   r l a l s l where a l =1Probp0Prob1−p a l 1 p 0 1 p The probability that a signal value remains intact is a known quantity p p and "drop-outs" are statistically independent of each other and of the noise. This kind of model is often used to describe intermittent behavior in electronic equipment.

In some cases it might be wise to not make a decision when the data do not justify it. Thus, in addition to declaring that one of two models occurred, we might declare "no decision" when the data are indecisive. Assume you observe L L statistically independent observations r l r l , each of which is Gaussian and has a variance of two. Under one model the mean is zero, and under the other the mean is one. The models are equally likely to occur.

You decide to flip coins with Sleazy Sam. If heads is the result of a coin flip, you win one dollar; if tails, Sam wins a dollar. However, Sam's reputation has preceded him. You suspect that the probability of tails, p p, may not be 1/2 12. You want to determine whether a biased coin is being used or not after observing the results of three coin tosses.

When a patient is screened for the presence of a disease in an organ, a section of tissue is viewed under a microscope and a count of abnormal cells made. Even under healthy conditions, a small number of abnormal cells will be present. Presumably a much larger number will be present if the organ is diseased. Assume that the number L L of abnormal cells in a section is geometrically distributed. PrL=l=1−ααl L l 1 α α l for l∈01… l 0 1 … . The parameter α α of a diseased organ will be larger than that of a healthy one. The probability of a randomly selected organ being diseased is p p.

How can the standard sequential test be extended to unknown parameter situations? Formulate the theory, determine the formulas for the thresholds. How would you approach finding the average number of observations?

A common situation in statistical signal processing problems is that the variance of the observations is unknown (there is no reason that noise should be nice to us!). Consider the two Gaussian model testing problem where the models differ in their means and have a common, but unknown variance. ℳ 0 :   r∼0σ2I ℳ 0 :   r 0 σ 2 I ℳ 1 :   r∼mσ2I ℳ 1 :   r m σ 2 I σ2=? σ 2 ?

Consider the following composite hypothesis testing problem (van Trees; Prob. 2.5.2). ℳ 0 :   prr=12π σ 0 ⅇ-r22 σ 0 2 ℳ 0 :   p r r 1 2 σ 0 r 2 2 σ 0 2 ℳ 1 :   prr=12π σ 1 ⅇ-r22 σ 1 2 ℳ 1 :   p r r 1 2 σ 1 r 2 2 σ 1 2 where σ 0 σ 0 is known but σ 1 σ 1 is known only to be greater than σ 0 σ 0 . Assume that we require that P F =10-2 P F 10 -2 .

Data are often processed "in the field," with the results from several systems sent to a central place for final analysis. Consider a detection system wherein each of N N field radar systems detects the presence or absence of an airplane. The detection results are collected together so that a final judgment about the airplane's presence can be made. Assume each field system has false-alarm and detection probabilities P F P F and P D P D respectively.

Mathematically, a preconception is a model for the "world" that you believe applies over a broad class of circumstances. Clearly, you should be vigilant and continually judge your assumption's correctness.

Let X l X l denote a sequence of random variables that you believe to be independent and identically distributed with a Gaussian distribution having zero mean and variance σ2 σ 2 . Elements of this sequence arrive one after the other, and you decide to use the sample average M n M n as a test statistic. M l =1l∑i=1l X i M l 1 l i 1 l X i

Delegating Responsibility

Modern management styles tend to want decisions to be made locally (by people at the scene) rather than by "the boss." While this approach might be considered more democratic, we should understand how to make decisions under such organizational constraints and what the performance might be.

Let three "local" systems separately make observations. Each local system's observations are identically distributed and statistically independent of the others, and based on the observations, each system decides which of two models applies best. The judgments are relayed to the central manager who must make the final decision. Assume the local observations consist either of white Gaussian noise or of a signal having energy E E to which the same white Gaussian noise has been added. The signal energy is the same at each local system. Each local decision system must meet a performance standard on the probability it declares the presence of a signal when none is present.