Neyman-pearson criterion
The maximization is over all decision rules (equivalently, over
all decision regions
,
).
Using different terminology, the Neyman-Pearson criterionselects the
most powerful test of size (not exceeding)
.
Fortunately, the above optimization problem has
an explicit solution. This is given by the celebrated
Neyman-Pearson lemma , which we now state. To ease
the exposition, our initial statement of this result onlyapplies to continuous random variables, and places a technical
condition on the densities. A more general statement is givenlater in the module.
Neyman-pearson lemma: initial statement
Consider the test
where
is a density. Define
, and assume that
satisfies the condition that for each
,
takes on the value
with probability
zero under hypothesis
. The solution to the optimization problem
in
is given by
where
is such that
If
, then
. The optimal test is unique up to a set of
probability zero under
and
.
The optimal decision rule is called the
likelihood ratio
test .
is the
likelihood ratio , and
is a
threshold . Observe that neither the likelihood
ratio nor the threshold depends on the
a
priori probabilities
. they depend only on the conditional densities
and the size constraint
. The threshold can often
be solved for as a function of
, as the next example
shows.
Continuing with
,
suppose we wish to design a Neyman-Pearson decision rule withsize constraint
. We have
By taking the natural logarithm of both sides of the LRT and
rarranging terms, the decision rule is not changed, and weobtain
Thus, the optimal rule is in fact a thresholding rule like we
considered in
. The false-alarm
probability was seen to be
Thus, we may express the value of
required by the
Neyman-Pearson lemma in terms of
:
Sufficient statistics and monotonic transformations
For hypothesis testing involving multiple or
vector-valued data, direct evaluation of the size(
) and power
(
)
of a Neyman-Pearson decision rule would require integrationover multi-dimensional, and potentially complicated decision
regions. In many cases, however, this can be avoided bysimplifying the LRT to a test of the form
where the test statistic
is a
sufficient
statistic for the data. Such a simplified form is
arrived at by modifying both sides of the LRT withmontonically increasing transformations, and by algebraic
simplifications. Since the modifications do not change thedecision rule, we may calculate
and
in terms of the sufficient statistic. For
example, the false-alarm probability may be written
where
denotes the density of
under
. Since
is typically of lower dimension than
, evaluation of
and
can be greatly simplified. The key is being able to reduce theLRT to a threshold test involving a sufficient statistic
for which we know the distribution .
Common variances, uncommon means
Let's design a Neyman-Pearson decision rule
of size
for the
problem
where
,
are known,
,
are
-dimensional
vectors, and
is the
identity
matrix. The likelihood ratio is
To simplify the test further we may apply the natural
logarithm and rearrange terms to obtain
We have used the assumption
. If
, then division by
is not a
monotonically increasing operation, and the inequalitieswould be reversed.
In the context of signal processing, the
foregoing problem may be viewed as the problem of detecting aconstant (DC) signal in
additive white
Gaussian noise :
where
is a known, fixed
amplitude, and
. Here
corresponds
to the mean
in the
example.
