0.6 Nonparametric regression with wavelets

In this section, we consider only real-valued wavelet functions that form an orthogonal basis, hence $\varphi \equiv \tilde{\varphi }$ and $\psi \equiv \tilde{\psi }$ . We saw in Orthogonal Bases from Multiresolution analysis and wavelets how a given function belonging to $L_2\left(\mathbb{R}\right)$ could be represented as a wavelet series. Here, we explain how to use a wavelet basis to construct a nonparametric estimator for the regression function $m$ in the model

where $x_i=\frac{i}{n}$ are equispaced design points and the errors are i.i.d. Gaussian, ${ϵ}_i\sim N(0,{\sigma }_{ϵ}^2)$ .

A wavelet estimator can be linear or nonlinear . The linear wavelet estimator proceeds by projecting the data onto a coarse level space. This estimator is of a kernel-type, see "Linear smoothing with wavelets" . Another possibility for estimating $m$ is to detect which detail coefficients convey the important information about the function $m$ and to put equal to zero all the other coefficients. This yields a nonlinear wavelet estimator as described in "Nonlinear smoothing with wavelets" .

Linear smoothing with wavelets

Suppose we are given data ${(x_i,Y_i)}_{i=1}^n$ coming from the model [link] and an orthogonal wavelet basis generated by $\{\varphi ,\psi \}$ . The linear wavelet estimator proceeds by choosing a cutting level $j_1$ and represents an estimation of the projection of $m$ onto the space $V_{j_1}$ :

with $j_0$ the coarsest level in the decomposition, and where the so-called empirical coefficients are computed as

The cutting level $j_1$ plays the role of a smoothing parameter: a small value of $j_1$ means that many detail coefficients are left out, and this may lead to oversmoothing. On the other hand, if $j_1$ is too large, too many coefficients will be kept, and some artificial bumps will probably remain in the estimation of $m\left(x\right)$ .

To see that the estimator [link] is of a kernel-type, consider first the projection of $m$ onto $V_{j_1}$ :

where the (convolution) kernel $K_{j_1}(x,y)$ is given by

Härdle et al. [link] studied the approximation properties of this projection operator. In order to estimate [link] , Antoniadis et al. [link] proposed to take:

Approximating the last integral by $\frac{1}{n}{\varphi }_{j_1,k}\left(x_i\right)$ , we find back the estimator $\widehat{m}\left(x\right)$ in [link] .

By orthogonality of the wavelet transform and Parseval's equality, the $L_2-$ risk (or integrated mean square error IMSE) of a linear wavelet estimator is equal to the $l_2-$ risk of its wavelet coefficients:

where

are called `theoretical' coefficients in the regression context. The term $S_1+S_2$ in [link] constitutes the stochastic bias whereas $S_3$ is the deterministic bias. The optimal cutting level is such that these two bias are of the same order. If $m$ is $\beta -$ Hölder continuous, it is easy to see that the optimal cutting level is $j_1\left(n\right)=O\left(n^{1/(1+2\beta )}\right)$ . The resulting optimal IMSE is of order $n^{-\frac{2\beta }{2\beta +1}}$ . In practice, cross-validation methods are often used to determine the optimal level $j_1$ [link] , [link] .