2.2 Hypothesis testing  (Page 2/2)

Tests and decision regions

Consider the general hypothesis testing problem where we have $N$ $d$ -dimensional observations $x_1,,x_N$ and $M$ hypotheses. If the data are real-valued, for example, then a hypothesistest is a mapping $$(:\mathbb{R}^d^N, \{1, , M\})$$ For every possible realization of the input, the test outputs a hypothesis. The test $$ partitions the input space into a disjoint collection $R_1,,R_M$ , where $$R_k=\{(x_1,,x_N)|(x_1, , x_N)=k\}$$ The sets $R_k$ are called decision regions . The boundary between two decision regions is a decision boundary . depicts these concepts when $d=2$ , $N=1$ , and $M=3$ .

Simple versus composite hypotheses

If the distribution of the data under a certain hypothesis is fully known, we call it a simple hypothesis. All of the hypotheses in the examples above are simple. In many cases, however, we onlyknow the distribution up to certain unknown parameters. For example, in a Gaussian noise model we may not know thevariance of the noise. In this case, a hypothesis is said to be composite .

Often a test involving a composite hypothesis has the form $$H_0:={}_0$$ $$H_1:\neq {}_0$$ where ${}_0$ is fixed. Such problems are called two-sided because the composite alternative "lies on both sides of $H_0$ ." When $$ is a scalar, the test $$H_0:\le {}_0$$ $$H_1:> {}_0$$ is called one-sided . Here, both hypotheses are composite.

Errors and probabilities

In binary hypothesis testing, assuming at least one of the two models does indeed correspond to reality, thereare four possible scenarios:

• Case 1

  $H_0$ is true, and we declare $H_0$ to be true

• Case 2

  $H_0$ is true, but we declare $H_1$ to be true

• Case 3

  $H_1$ is true, and we declare $H_1$ to be true

• Case 4

  $H_1$ is true, but we declare $H_0$ to be true

Consider the general binary hypothesis testing problem $$H_0:(x, f_{}(x)),\in {}_0$$ $$H_1:(x, f_{}(x)),\in {}_1$$ If $H_0$ is simple, that is, ${}_0=\{{}_0\}$ , then the size (denoted $$ ), also called the false-alarm probability ( $P_F$ ), is defined to be $$=P_F=({}_0, \text{declare}{H}_1)$$ When ${}_0$ is composite, we define $$=P_F={\sup }_{{}_0}((, \text{declare}{H}_1))$$ For $\in {}_1$ , the power (denoted $$ ), or detection probability ( $P_D$ ), is defined to be $$=P_D=(, \text{declare}{H}_1)$$ The probability of a type II error, also called the miss probability , is $$P_M=1-P_D$$ If $H_1$ is composite, then $=()$ is viewed as a function of $$ .

Criteria in hypothesis testing

The design of a hypothesis test/detector often involves constructing the solution to an optimizationproblem. The optimality criteria used fall into two classes: Bayesian and frequent.

Representing the former approach is the Bayes Risk Criterion . Representing the latter is the Neyman-Pearson Criterion . These two approaches are developed at length in separate modules.

Statistics versus engineering lingo

The following table, adapted from Kay, p.65 , summarizes the different terminology for hypothesis testing from statistics and signal processing: