In the most general setup, the observation is a collection x 1 , … , x N x 1 , … , x N of random vectors. A common assumption, which facilitates analysis, is that the data are independent and identically distributed (IID). The random vectors may be continuous, discrete, or in some cases mixed. It is generally assumed that all of the data is available at once, although for some applications, such as Sequential Hypothesis Testing, the data is a never ending stream.

When there are two competing hypotheses, we refer to a binary hypothesis test. When the number of hypotheses is M≥2 M 2 , we refer to an M-ary hypothesis test. Clearly, binary is a special case of MM-ary, but binary tests are accorded a special status for certain reasons. These include their simplicity, their prevalence in applications, and theoretical results that do not carry over to the MM-ary case.

In many binary hypothesis tests, one hypothesis represents the absence of a ceratin feature. In such cases, the hypothesis is usually labelled H 0 H 0 and called the null hypothesis. The other hypothesis is labelled H 1 H 1 and called the alternative hypothesis.

Consider the general hypothesis testing problem where we have NN dd-dimensional observations x 1 , … , x N x 1 , … , x N and MM hypotheses. If the data are real-valued, for example, then a hypothesis test is a mapping φ : RdN →1…M → φ : d N 1 … M For every possible realization of the input, the test outputs a hypothesis. The test φφ partitions the input space into a disjoint collection R 1 , … , R M R 1 , … , R M , where R k = ( x 1 , … , x N ) | φ⁢ x 1 … x N =k R k ( x 1 , … , x N ) | φ x 1 … x N k The sets R k R k are called decision regions. The boundary between two decision regions is a decision boundary. Figure 2 depicts these concepts when d=2 d 2 , N=1 N 1 , and M=3 M 3 .

Figure 2

If the distribution of the data under a certain hypothesis is fully known, we call it a simple hypothesis. All of the hypotheses in the examples above are simple. In many cases, however, we only know the distribution up to certain unknown parameters. For example, in a Gaussian noise model we may not know the variance of the noise. In this case, a hypothesis is said to be composite.

Often a test involving a composite hypothesis has the form H 0 : θ= θ 0 H 0 : θ θ 0 H 1 : θ≠ θ 0 H 1 : θ θ 0 where θ 0 θ 0 is fixed. Such problems are called two-sided because the composite alternative "lies on both sides of H 0 H 0 ." When θθ is a scalar, the test H 0 : θ≤ θ 0 H 0 : θ θ 0 H 1 : θ> θ 0 H 1 : θ θ 0 is called one-sided. Here, both hypotheses are composite.