2.2 Hypothesis testing

Suppose you measure a collection of scalars $x_1,,x_N$ . You believe the data is distributed in one of two ways. Your first model, call it $H_0$ , postulates the data to be governed by the density $f_0(x)$ (some fixed density). Your second model, $H_1$ , postulates a different density $f_1(x)$ . These models, termed hypotheses , are denoted as follows: $$H_0:(x_n, f_0(x)),n=1N$$ $$H_1:(x_n, f_1(x)),n=1N$$ A hypothesis test is a rule that, given a measurement $x$ , makes a decision as to which hypothesis best "explains" the data.

Generalizations and nomenclature

The concepts introduced above can be extended inseveral ways. In what follows we provide more rigorous definitions, describe different kinds of hypothesis testing, andintroduce terminology.

Data

In the most general setup, the observation is a collection $x_1,,x_N$ of random vectors. A common assumption, which facilitates analysis, is that the data are independent and identicallydistributed (IID). The random vectors may be continuous, discrete, or in some cases mixed. It is generally assumedthat all of the data is available at once, although for some applications, such as Sequential Hypothesis Testing , the data is a never ending stream.

Binary versus m-ary tests

When there are two competing hypotheses, we refer to a binary hypothesis test. When the number of hypotheses is $M\ge 2$ , we refer to an M-ary hypothesis test. Clearly, binary is a special case of $M$ -ary, but binary tests are accorded a special status for certain reasons. These includetheir simplicity, their prevalence in applications, and theoretical results that do not carry over to the $M$ -ary case.

In many binary hypothesis tests, one hypothesis represents the absence of a ceratinfeature. In such cases, the hypothesis is usually labelled $H_0$ and called the null hypothesis. The other hypothesis is labelled $H_1$ and called the alternative hypothesis.