Detection theory basics

Detection theory concerns making decisions from data. Decisions are based on presumptive models that may have produced the data.Making a decision involves inspecting the data and determining which model was most likely to have produced them. In this way, we are detecting which model was correct.Decision problems pervade signal processing. In digital communications, for example, determining if the current bit received in the presence of channel disturbances was a zero or a one is a detection problem.

More concretely, we denote by ${ℳ}_{\mathrm{i}}$ the $i^{\text{th}}$ model that could have generated the data $R$ . A "model" is captured by the conditional probability distribution of the data, which is denoted by the vector $R$ . For example, model $i$ is described by $p(R, {ℳ}_i, r)$ . Given all the models that can describe the data, we need to choose which model best matched what was observed.The word "best" is key here: what is the optimality criterion, and does the detection processing and the decision rule depend heavily on the criterion used?Surprisingly, the answer to the second question is "No." All of detection theory revolves around the likelihood ratio test , which as we shall see, emerges as the optimal detector under a wide variety of optimality criteria.

The likelihood ratio test

In a binary detection problem in which we have two models, four possible decision outcomes can result. Model ${ℳ}_0$ did in fact represent the best model for the data and the decision rule said it was (a correct decision) or saidit wasn't (an erroneous decision). The other two outcomes arise when model ${ℳ}_1$ was in fact true with either a correct or incorrect decision made. The decision process operates by segmentingthe range of observation values into two disjoint decision regions $Z_0$ and $Z_1$ . All values of $R$ fall into either $Z_0$ or $Z_1$ . If a given $R$ lies in $Z_0$ , we will announce our decision

"model ${ℳ}_0$ was true"

The Bayes' decision criterion seeks to minimize a cost function associated with making a decision. Let $C_{ij}$ be the cost of mistaking model $j$ for model $i$ ( $i\neq j$ ) and $C_{ii}$ the presumably smaller cost of correctly choosing model $i$ : $C_{ij}> C_{ii}$ , $i\neq j$ . Let ${\pi }_i$ be the a priori probability of model $i$ . The so-called Bayes' cost $\langle C\rangle$ is the average cost of making a decision.

The data processing operations are captured entirely by the likelihood ratio $\Lambda (r)$ . However, the calculations required by the likelihood ratio can be simplified in many cases. Note that only the value of thelikelihood ratio relative to the threshold matters.Consequently, we can perform any positively monotonic transformation simultaneously on the likelihood ratio and the threshold without affecting the result of thecomparison. For example, we can multiply by a positive constant, add any constant or apply a monotonically increasing functionto reduce the complexity of the expressions. We single out one such function, the logarithm, because it often simplifies likelihoodratios that commonly occur in signal processing applications. Known as the log-likelihood , we explicitly express the likelihood ratio test with it as

The likelihood ratio is comprised of the quantities $p(R, {ℳ}_i, r)$ , which are known as likelihood functions and play an important role in estimation theory. It is the likelihood function that portrays the probabilistic modeldescribing data generation. The likelihood function completely characterizes the kind of "world" assumed by eachmodel. For each model, we must specify the likelihood function so that we can solve the hypothesis testing problem.

A complication, which arises in some cases, is that the sufficient statistic may not be monotonic. If it is monotonic, thedecision regions $Z_0$ and $Z_1$ are simply connected: all portions of a region canbe reached without crossing into the other region. If not, the regions are not simply connected and decision regionislands are created. Disconnected regions usually complicate calculations of decision performance. Monotonic ornot, the decision rule proceeds as described: the sufficient statistic is computed for each observation vector and comparedto a threshold.