0.6 Nonparametric regression with wavelets  (Page 2/5)

Nonlinear smoothing with wavelets

Hard-, soft-thresholding and wavelet estimator

Given the regression model [link] , we can decompose the empirical detail coefficient ${\widehat{d}}_{jk}$ in [link] as

If the function $m\left(x\right)$ allows for a sparse wavelet representation, only a few number of detail coefficients $d_{jk}$ contribute to the signal and are non-negligible. However, every empirical coefficient ${\widehat{d}}_{jk}$ has a non-zero contribution coming from the noise part ${\rho }_{jk}$ .

In words, $d_{jk}$ constitutes a first order approximation (using the trapezoidal rule) of the integral $d_{jk}^{\circ }$ . For the scaling coefficients $s_{jk}^{\circ }$ , it can be proved [link] that the order of accuracy of the trapezoidal rule is equal to $N-1$ , where $N$ is the order of the MRA associated to the scaling function.

Suppose the noise level is not too high, so that the signal can be distinguished from the noise. Then, from the sparsity property of the wavelet, only the largest detail coefficients should be included in the wavelet estimator.Hence, when estimating an unknown function, it makes sense to include only those coefficients that are larger than some specified threshold value $t$ :

This `keep-or-kill' operation is called hard thresholding , see [link] (a).

Now, since each empirical coefficient consists of both a signal part and a noise part, it may be desirable to shrink even the coefficients that are larger than the threshold:

Since the function ${\eta }_S$ is continuous in its first argument, this procedure is called soft thresholding . More complex thresholding schemes have been proposed in the literature [link] , [link] , [link] . They often appear as a compromise between soft and hard thresholding, see [link] (b) for an example.

For a given threshold value $t$ and a thresholding scheme ${\eta }_{(.)}$ , the nonlinear wavelet estimator is given by

where $j_0$ denotes the primary resolution level . It indicates the level above which the detail coefficients are being manipulated.

Let now ${\widehat{d}}_j=\{{\widehat{d}}_{jk},k=0,...,2^j-1\}$ denote the vector of empirical detail coefficients at level $j$ and similarly define ${\widehat{s}}_j$ . In practice a nonlinear wavelet estimator is obtained in three steps.

1. Apply the analyzing (forward) wavelet transform on the observations ${\left\{Y_i\right\}}_{i=1}^n$ , yielding ${\widehat{s}}_{j_0}$ and ${\widehat{d}}_j,$ for $j=j_0,...,J-1$ .
2. Manipulate the detail coefficients above the level $j_0$ , e.g. by soft-thresholding them.
3. Invert the wavelet transform and produce an estimation of $m$ at the design points: ${\left\{\widehat{m}\left(x_i\right)\right\}}_{i=1}^n$ .

If necessary, a continuous estimator $\widehat{m}$ can then be constructed by an appropriate interpolation of the estimated $\widehat{m}\left(x_i\right)$ values [link] .