2.3 The neyman-pearson criterion

In hypothesis testing , as in all other areas of statistical inference, there are two major schools of thought on designing good tests:Bayesian and frequentist (or classical). Consider the simple binary hypothesis testing problem $${}_0:(x, f_0(x))$$ $${}_1:(x, f_1(x))$$ In the Bayesian setup, the prior probability ${}_i=({}_i)$ of each hypothesis occurring is assumed known. This approach to hypothesis testing is represented by the minimum Bayes risk criterion and the minimum probability of error criterion .

In some applications, however, it may not be reasonable to assign an a priori probability to a hypothesis. For example, what is the a priori probability of a supernova occurring in any particular region of the sky? What is the prior probability of being attacked by aballistic missile? In such cases we need a decision rule that does not depend on making assumptions about the a priori probability of each hypothesis. Here the Neyman-Pearson criterion offers an alternative to the Bayesianframework.

The Neyman-Pearson criterion is stated in terms of certain probabilities associated with a particular hypothesis test. The relevant quantities are summarized in . Depending on the setting, different terminology is used.

Here $P_i(\text{declare}{}_j)$ dentoes the probability that we declare hypothesis ${}_j$ to be in effect when ${}_i$ is actually in effect. The probabilities $P_0(\text{declare}{}_0)$ and $P_1(\text{declare}{}_0)$ (sometimes called the miss probability), are equal to $1-P_F$ and $1-P_D$ , respectively. Thus, $P_F$ and $P_D$ represent the two degrees of freedom in a binary hypothesis test. Note that $P_F$ and $P_D$ do not involve a priori probabilities of the hypotheses.

These two probabilities are related to each other through the decision regions . If $R_1$ is the decision region for ${}_1$ , we have $$P_F=\int f_0(x)\,d x$$ $$P_D=\int f_1(x)\,d x$$ The densities $f_i(x)$ are nonnegative, so as $R_1$ shrinks, both probabilities tend to zero. As $R_1$ expands, both tend to one. The ideal case, where $P_D=1$ and $P_F=0$ , cannot occur unless the distributions do not overlap ( i.e. , $\int f_0(x)f_1(x)\,d x=0$ ). Therefore, in order to increase $P_D$ , we must also increase $P_F$ . This represents the fundamental tradeoff in hypothesis testing and detection theory.

The neyman-pearson lemma: a first look

The Neyman-Pearson criterion says that we should construct our decision rule to have maximum probability ofdetection while not allowing the probability of false alarm to exceed a certain value $$ . In other words, the optimal detector according to theNeyman-Pearson criterion is the solution to the following constrainted optimization problem: