2.2 Second-order convergence analysis of the lms algorithm and

Adaptive filters Page 1 / 1

Convergence of the mean (first-order analysis) is insufficient to guarantee desirable behavior of the LMS algorithm; the variancecould still be infinite. It is important to show that the variance of the filter coefficients is finite, and to determine how closethe average squared error is to the minimum possible error using an exact Wiener filter.

_{k} 2 d_{k} W^{k} X^{k} 2 d_{k} 2 2 d_{k} X^{k} W^{k} W^{k} X^{k} X^{k} W^{k} r_{dd} 0 2 W^{k} P W^{k} R W^{k}

The minimum error is obtained using the Wiener filter

W_{opt} R P

_{\min} 2 2 r_{dd} 0 2 P R P P R R R P r_{dd} 0 P R P

To analyze the average error in LMS, write in terms of

V^{'} Q W W_{opt}

, where

Q Q R

_{k} 2 r_{dd} 0 2 W^{k} P W^{k} R W^{k} W^{k} R W_{opt} W_{opt} R W^{k} W_{opt} R W_{opt} W^{k} R W_{opt} W_{opt} R W^{k} W_{opt} R W_{opt} r_{dd} 0 V^{k} R V^{k} P R P_{\min} 2 V^{k} R V^{k}_{\min} 2 V^{k} Q Q R Q Q V^{k}_{\min} 2 V^{'k} V^{'k}

_{k} 2_{\min} 2 j 0 N 1_{j} v_{j}^{'k} 2

So we need to know

v_{j}^{'k} 2

, which are the diagonal elements of the covariance matrix of

V^{'k}

, or

V^{'k} V^{'k}

From the LMS update equation $W^{k + 1} W^{k} 2_{k} X^{k}$ we get $V^{'k + 1} W^{'k} 2_{k} Q X^{k}$

^{k + 1} V^{' k + 1} V^{' k + 1} V^{'k} V^{'k} 2_{k} Q X^{k} V^{'k} 2_{k} V^{'k} X^{k} Q 42_{k} 2 Q X^{k} X^{k} Q^{k} 2_{k} Q X^{k} V^{'k} 2_{k} V^{'k} X^{k} Q 42_{k} 2 Q X^{k} X^{k} Q

Note that

_{k} d_{k} W^{k} X^{k} d_{k} W_{opt} V^{'k} Q X^{k}

_{k} Q X^{k} V^{'k} d_{k} Q X^{k} V^{'k} W_{opt} X^{k} Q X^{k} V^{'k} V^{'k} Q X^{k} V^{'k} 0 0 Q X^{k} X^{k} Q V^{'k} V^{'k} Q X^{k} X^{k} Q V^{'k} V^{'k}^{k}

Note that the Patently False independence Assumption was invoked here.

To analyze $_{k} 2 Q X^{k} X^{k} Q$ , we make yet another obviously false assumptioon that $_{k} 2$ and $X^{k}$ are statistically independent. This is obviously false, since $_{k} d_{k} W^{k} X^{k}$ . Otherwise, we get 4th-order terms in $X$ in the product. These can be dealt with, at the expense of a more complicated analysis, if aparticular type of distribution (such as Gaussian) is assumed. See, for example Gardner . A questionable justification for this assumption is that as $W^{k} W_{opt}$ , $W^{k}$ becomes uncorrelated with $X^{k}$ (if we invoke the original independence assumption), which tends to randomize the error signal relative to $X^{k}$ . With this assumption, $_{k} 2 Q X^{k} X^{k} Q_{k} 2 Q X^{k} X^{k} Q_{k} 2$ Now $_{k} 2_{\min} 2 V^{'k} V^{'k}$ so

_{k} 2_{\min} 2 j_{j} V_{j}^{'k} 2_{\min} 2 j_{j}_{jj}^{k}

Thus, becomes

^{k + 1} I 4^{k} 42 j_{j}_{jj}^{k} 42_{\min} 2

Now if this system is stable and converges, it converges to

^{}^{+ 1}

4^{} 42 j_{j}_{jj}_{\min} 2

^{} j_{j}_{jj}_{\min} 2 I

So it is a diagonal matrix with all elements on the diagonal equal:

Then $_{ii}^{}_{ii}^{} j_{j}_{\min} 2$ $_{ii}^{} 1 j_{j}_{\min} 2$ $_{ii}^{}_{\min} 2 1 j_{j}$ Thus the error in the LMS adaptive filter after convergence is

_{} 2_{\min} 2 V^{'} V^{'}_{\min} 2_{\min} 2 j_{j} 1 j_{j}_{\min} 2 1 1 j_{j}_{\min} 2 1 1 tr R_{\min} 2 1 1 r_{xx} 0 N

_{} 2_{\min} 2 1 1 N_{x} 2

1 N_{x} 2

is called the misadjustment factor . Oftern, one chooses

to select a desired misadjustment factor, such as an error 10% higher than theWiener filter error.

2nd-order convergence (stability)

To determine the range for for which converges, we must determine the for which the matrix difference equation converges. $^{k + 1} I 4^{k} 42 j_{j}_{jj}^{k} 42_{\min} 2$ The off-diagonal elements each evolve independently according to $_{ij}^{k + 1} 1 4_{i}_{ij}^{k}$ These terms will decay to zero if $i 4_{i} 2$ , or $1 2_{\max}$

The diagonal terms evolve according to $_{ii}^{k + 1} 1 4_{i}_{ii}^{k} 42_{i} j_{j}_{jj}^{k} 42_{\min} 2_{i}$ For the homoegeneous equation $_{ii}^{k + 1} 1 4_{i}_{ii}^{k} 42_{i} j_{j}_{jj}^{k}$ for $1 4_{i}$ positive,

_{ii}^{k + 1} 1 4_{i}_{iimax}^{k} 42_{i} j_{j}_{jjmax}^{k} 1 4_{i} 42_{i} j_{j}_{jjmax}^{k}

_{ii}^{k + 1}

will be strictly less than

_{jjmax}^{k}

for

1 4_{i} 42_{i} j_{j} 1

42_{i} j_{j} 4_{i}

1 j_{j} 1 tr R 1 N r_{xx} 0 1 N_{x} 2

This is a more rigorous bound than the first-order bounds. Ofter engineers choose

a few times smaller than this, since more rigorous analyses yield a slightly smallerbound.

3 N_{x} 2

is derived in some analyses assuming Gaussian

x_{k}

d_{k}

<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

100% Free Mobile Applications
Receive real-time job alerts and never miss the right job again

Source: OpenStax, Adaptive filters. OpenStax CNX. May 12, 2005 Download for free at http://cnx.org/content/col10280/1.1

Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Adaptive filters' conversation and receive update notifications?

Ask

	Classical Music By Marion Cabalfin Start Quiz
	15 Dr. Amberg Pharm quiz By Brooke Delaney Start Exam
	Introduction to sociology 2e By OpenStax Read Online Course
	Microeconomics Test 1 By IES Portal Start Test
	Music 113 By Eric Crawford Start Quiz
	19 AP Key Terms 19 Cardiovascular System Heart By OpenStax Start Key Terms
	5 Sociology 05 Socialization MCQ By OpenStax Start Quiz
	2 AP 02 Chemical Level of Organization MCQ By OpenStax Start Quiz
	1 Pharmacology Nervous System MCQ By Rohini Ajay Start Quiz
©flickr:	Anatomy Physiology By Jemekia Weeden Start Quiz