Robust least mean mixed norm adaptive filtering for α-stable random processes

1997 IEEE International Symposium on Circuits and Systems, June 9-12, 1!197, Hoizg Kong

Robust

Least Mean Mixed

Norm

Adaptive

Filtering

for

a-Stable

Random

Processes

Gul A

y d z n l ,

0 . i ~ ~

Tanrzkulu2,

A.Enis

Cetznl

'Bilkent University,

Dcpartment

of

Electrical Engineering,

06533,

Bilkent, Ankara, Turkey

Imperial College

of

Science, Technology

and

Medicine,

Department

of

Electrical and Electronic Engineering

Exhibition

Road

London SW7

2BT,

United Kingdom

AB s T RAC T

Uased on the concept of Fractional Lower Order

Stat,ist,ics (FLOS), we present the Robust Least Mean

Mixed Norin ( R L M M N ) adaptive algorithm for ap-

plical,ioiis i r r impidsive environments inodeled by N-

ble distribut,ioiis. A sufficient condition for finite

variance of t,he iipdxte term i s obtained for the un-

tlrrlyirig u-stable process. Simula,tion results are pro- viclcd regarding t>he identification of the parameters

of air AR system.

In ina.iiy signal processing applica.tions the noise

is iiioclciid as a Gaussian stochastic process. This as- sumpt,iori ha,.; been broadly accepted due t o the Cen-

t r d Limit Theorein but a large class of physical ob-

tioiis such as low frequency atmospheric noise,

~ i ~ i ~ ~ r - i i i a ~ l ~ ~ noise and underwater acoustic noise ex-

hillit. uon-GaussitLii behaviour [1]-[4]. A class of statis- t,icaJ nrodels applicable in these cases is the a-stable

(0

<

Q

5

2) distributions

[5].

Having heavier tails

t l i i ~ r i the Gaussian distributions, these distributions

1 spikes or occasional bursts in their re-

a

I

iz n t i oils.

ci-stahle distributions do riot have closed form

proha.Ijilit~y density functioiis except the cases N = 1

( C a i i c h y tlistxibiition) and a = 2 (Gaussian distribu-

l . i o i i ) . Howc:vccr t,hcy have closed forrri characteristic

fiinct.ions givrn hy :

~ > ( t ) = fix[p{ist - ritl"[I

+

i p s g n ( t ) w ( t , a)]} _{( I )}

wlicre ((1

<

n

5

2 ) is the characteristic exponent,

s is t,he location parameter,

j3

(--1

5

p 5

1) is the

iiiclex of skewiiess, 7

>

0

is the dispersion parameter,

sgn(. ) c l e n d PS tlic sigiium function ~ and

T h c distribution is called Symmetric a-Stable

( S Q S ) ,

if ,J' = 0 T h e parameter a controls the tails of the

distribution. For 0

<

a

<

2 the distributions have

algebraic tail which are significantly heavier than the

exponential tail of the Gaussian distribution. T h e

smaller the value of the a , the heavier the ta,ils of

the distributions. This property makes the cvstable

distributions a n appealing model for impulsive noise

environments. It is well known t h a t if the noise is

impulsive, a1:gorithrns developed under the Gaussian-

ity assumption may produce unacceptable results, [5],

[el

'

T h e Gaussian signals can be treated in a Hilbert

space framework which allows the use of La norm

in various optimiza.tion criteria. Whereas the linear

vector space genera.ted by a-st,able distributions is a

Banach space when 1

5

a

<

2. In the Banach spa.ce

of a-stable processes only L , norm exists for p

<

C Y ,

hence the us'e of La norm is unacceptable in this case,

[51, [71 .

Due t o the heavy tails, stable distributions do not have finite second or higher order moments, except

the limiting case of a = 2. Let X be a n a-stable

random variable with 0

<

a

<

2 , then

E[IXl'] = 00, if p

2

a . ( 3 )

However, for 0

5

p

<

a the Fractional Lower. Order

Monicnt (FLOM) is finite, i.e.,

E[lXlp]

<

00, if

0

5

p

<

C Y . ( 3 )

.E[lXl']

<

00, for all p

2

0. (5)

If N = 2, then

T h e fractional p t h order moment [5] of an S N S

random variable wii h zero location parameter, a = 0 ,

is given by :

E[IXI"'] = C ( p , a)yp'" for 0

<

I,

<

c v , (6) where

111 Lqua.tioii { 7 ) , l( ,) denotes the gainina function. The constant, C ; ( p > (I,) depends only on Q and p.

In t>liis paper, based on the concept of FLOS, we

iiit,rotliicci a new family of adaptive filtering algorithms t,lrat i ~ t ~ i l i z e a linear mixture of moinent8s similar t.o

t,hr I,MMY algorit,hm [S,9]. It is assumed that the a- st,able noise lias finite mean corresponding 1

5

a

<

2 .

11 Tirr LMAIN XLGORII'HV I \ I M P U L S I L E

b:N V I R 0 H b1 E I\ T S

' T h i . 1,kIMN algorithm [SI is proposed for the cases \ v h ( > i i tlre noise distribution is a mixture of short and

Ioiig-tail tlist,rihutions. This algorithm coinbines the

Mt'an Square (LMS)

[lo]

and Least Mean Fourth

) [ I l l algorithms. T h e cost, function and t,he

uliclat,c, c'qiiat,ion of the L M M N algorithm are respec- I i w l y given 1 3 ~ (8) 1 1 'L

4

.Jk = -XE[cl]

+

-(1 - A)E[ei] i111(1 W k + i = Wk

+

/ l e k ( x

+

(1 - X ) & X k . _{( 9 )}

The t m n p wvc-ights of t8he adaptive filter at time b are Wk = [ 1 i ) ( 1 , 1 . . . . ~ n ~ - l , k ] ~ . T h e input vector is X k =

[ . c ~ . . . . x ~ - n . ~ + I ] ' . The error signalis e k = d k - W k T X k

where i l k i s t,he desired signal. T h e step-size p which coiit#rols ( , ] l e spced of convergence and 0

<:

X

5

1 is

t,I it' mixing 1) ara.inet er. , ,

111 .App(-\iidix-I, the update equation of the L M h I N

algorit,liiii is st,iidied for a-stable distributions. It is

slio~vn k,lr;rt the variance of the gradient vector is 1111-

l)owidid3 i . e . ~

E[/IWk+i - WlcII:] = W .

(10)

poiids t o unstable behaviour that agrees

pwcisely with our experimental evaluations. 11.1 , > ' / U ) u l a t i o n ,$t,udies

Coiisider a n AR,(Df) a-stable process, defined as f'ollonh,

i = l

wlieue 111; is a symetric cy-stable ( S n S ) distribution.

If { b } is a n absolutely summable sequence, then x k

;\Is0 liiis R syminetric S a S distribution with the same

paraiiietrrs as Pl,k [5, 121.

i h i h K ( ' L ) process with parameters 61 = 0.99 and

li? = -0. L is ideiitified by using the LMMN algo-

ri(lriri, where iiiput, signal z k is generated from a S a S

process with a = 1.2. T h e transient helmviour of

the adaptive weights are illustrated in Fig. 1 for one

realization. T h e mixing parameter values are cho-

sen as X = 0.1, 0.3, 0.6, and 0.9 a i d the st,ep-size

is U, = 5.2 x lO-'.The LMMN algorithm experiences

divergence for all values of X even if very small s k p -

sizes are used

.5r!

2t a 0

:::/

:02p-j

B I I$ 0 0 5 O o l i r i 1-0.5 1 -0 5 0 10 20 30 40 0 10 20 30 40 lime time 0.4 2 0 -0 2 0 10 20 30 40 -0 05 0 10 20 30 40 m e time

Fig. 1. Transient behaviour of tap weights for the 1,MMN

algoritlirii for a SmS input with o = 1 . 2 . Divergence in all

cases w i t h A = 0.1. 0 . 3 , 0.5, 0.9.

111. ROBLST LMMN ADAPTIVE FILTERING

In order to ensure finite varknce, i.e.

,

of t h e gradient vector for a-stable distributions, the

L M M X algorithm is modified by using the concept of

FLOS. In particular, the fact t h a t in impulsive noise, t h e variance of the fractional lower order error

is bounded. Based on this observation we introduce

the Robust-LMMN

(RLMMN)

algorithm as :

Wl.+l = WI:

+

/ L ~ ; ' > ( A

+

(1

- A){e:a>}2)X~e

(14)

where X k = [.zk<a>,,,.z:t-fif+l<a>] . Unlike t h e LMNlN

algorithm, the RLMMN algorithm is expected to yield

well-behaved convergence in impulsive noise environ-

ments. Note t h a t , the actual weights define a stable

stationary point of iterations for the RLMMN algo-

rithm.

-

_T

In Appendix-11, it is shown t h a t a sufficient con-

dition for the validity of the condition in (12) is

111 I ,Szitt ulntzoii Stiidaes _{V . APPENDIX-I}

7 ’ 1 1 ~ FT,O5 based R L M M N algorithni is compared i o tli(. Normalizcd Least Mean-p Norm ( N L M P ) [7]

,ilgoi i t h i n The update equation of the N L M P algo-

ritliin I \ givcm h y

. < p - l >

whcrt. 1

5

1)

<

a, E

>

0 and p is choscn such t h a t t h e

YLMP

algorithm is stable.

In Fig. 2 ) tlie results of a system identification ex- pt,riiirc,nt a r c presented. Note that CY = 1.2 and X =

0.9. A 1 1 AR,(5) process with parameters bl = 0.890,

h2 1 -0.152, hr3 = 0.100, b4 = -0.197 and b5 = 0.097

is gcrierat,ed which is driven by an i.i.d. S a S ran-

I L ~ . T h e system mismatch is obtained

l l W k - W,

11;

over 100 Monte Carlo tri- a n d W, are respec1,ively the current or a.iitl t h e optimal solution. T h e per-

c‘ K L M M N algorithm is comparable

wit,li t,lic N LNIP algorithm depending on the mixture

pilralilct,(:r. Note t h a t , the step-sizes are chosen as p = 5.2 x I O - “ for both algorithms and the steady-

em iirisniatches are approximately equal.

O Y , T - - - -.. .~ ~ , . . ~ --. - _ 1000 2000 3000 4000 5000 6000 I l n X Fig. 2 .

a i i d tlir Nl,‘\dP algorithm (dashed) for CY = 1.2 arid X = 0.9.

1111. b y s k i n riiisiiiatch for R L M M N algorithm (solid)

1V. C O N C L U S I O N

Tlic- R o l ~ ~ i ~ l - L i \ i l M N adaptive algorithm is proposed for i,lii: ~ t p p l i c n t i o ~ i s wliere the signals have a n impul-

5iT.e l ~ i ~ d ~ ~ r < ~ . A Iiound is obtained on thc variance of

1,Iie grailiciit vector of‘ fhe proposed algorithm. Simu- lation rc~srilts are presented for adaptive system iden- 1, i 1 i c a 1,ioii.

The unbounded behaviour of the LMMN algo-

rithm for a-stable processes is proven below. Let

e k = d k - WkTXk = VkTXk ₍₁₇₎

where

(18) a

Vk =

w,

- WI;

is the weight error vector. Since Vk - Vk+l =

Wk+l -Wk, the weight update equation of tlie LMMN

algorithm can be written in terms of Vk as :

~ k + = l Vk -- p e k ( X

+

(1 - X)e;)Xk (19)

arid

p2E[(VrlXh)2(X

-t

(1 - X ) ( V ~ X , ) ” ) 2 ~ ~ X I ; $ ] . (20)

Having infinite value of (20) can be simply shown

by considering the term, E[(VZXI,)

I

IXk

I

l a ] ,

after ex-

panding the parerit hesis. By using the “indepeiidence

theory assumption” in

[lo]

aiid assuming t h a t VI, and

XI, are independent, we have

2 2

i = l

We have E ~ X ~ ~ , ~ ~ & / ~ ~ ] = 00, since

E[zi-,]

= 00

within this (3xpectation and therefore, (10) is proven

for T,MMN algorithm under ru-stable distributions

VI. APPENDIX-I1

In order to prove (15) the update equation of tjhc

RLMhlN algorithm can be rewritten in terms of v k

as 1

< U >

[5] M.

Shao and

C.L.

Nikias, "Signa.1 Processing with

Fractional Lower Order Moments : Stable Processes

and their Applications, " Proc. IEEE, Vol. 81, pp.

986-1009, 1993.

[6] S. A . Kassam, Signal Detection in Non-Gaussian Noise. N e w York: Springer, 1988.

[7] 0. Ariltaii, A. E. Cetin, and E. Erzin. " Adaptive

Filtering for Non-Gaussian Stable Processes," IEEE

Transaction o n Signal Processing Letters, Vol. 1, pp.

2

A

$- ( l - A ) { ( v k T x k ) < a > } ' ]

~ ~ x k l i ~ '

(23) By using t,lre Hijlder inequality,

IVkTXk/

L

llVI;lllllXkllm (24)

i n (2:3), TTC obt,aiii

1

1wktl

-

w k

11;

5

p2

I

IvI;

11;'

I

IxI;

I

I z [ X 2

+

2X( 1 -

A)

[a]

0. Tanrikulu, J . A . Chambers and A.G. Constan-

tinides, Least Mean Mixed-Norm Adaptive Filter-

ing," Electronics Letters, Vol. 30, N o . 19, pp. 1574-

1575, Sep. 1994.

[9]

0. Tanrikulu and J . A . Chambers, "Convergence

and steady-state properties of the Least Mean Mixed

Norm (LMMN) adaptive algorithm" ,to appear in IEE

Proceedings- VIS, 1996. If t8hr statist,ical independence assumption of VI; and

Xk is iiscd oiice inore, we have E[I I W ~ + I ~ W k

II;]

5

~ ~ ' [ ~ ' E [ I I V I , ~ ~ ~ ~ ] E [ ~ ~ X I ; ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~

[ l o ]

S.

Hayltin Adaptive Filter Theory

,

Preiitice Hall,

N J , 1986

A IEEE Trans. 1984, IT-30, pp.275-283.

[la ] Y. Hosoya, "Discrete Time Stable Processes and

Their Certain Properties, " Ann. Prob., Vol.

(5,

No.

Since x k - j = IlXkll,

<

00, from

(as),

we obtain

U

<

a18

(26)

as a sufficient condition for the fulfillment of the con-

tlitioii in (12). It is a.ssumed t h a t no parameter-

i z r clivergc.nce occurs in the R L M M N algorithm) i.e.,

su1)i; ~ ~ V I ; ~ ~ m

<

i-10. In this case, even if VI; is a SCYS w c t o r prowss, when (26) isvalid, we have E[I/VI,II;"]

<

x , E[ilVklI':"]

<

00 a.nd E[llV~;il:"]

<

00.

1, p p . 94-105, 1978.

V I I . REFERENCES

[I]

13.Ma.iidelbrot and J . W . V a n Ness, "Fractional Brow-

I I ~ R I I i\/lol,ions, Fractional Noises, and .4pplicatioiis,"

C U I , Vol. 10, pp. 422-437, 1968.

[;'I

S',S.Pilla.i, h/I. Harisankar,

''

Simulat,ed performance

of a. DS spread spectrum system in impulsive atmo-

spheric noise, '' IEEE Trans. Electromagmtic Com- p t . . Vol. 2 9 % p p . 80-82, 1987.

[3] hl. Uouvet a,nd S.C. Schwartz, "Comparison of

atlaptivc ancl robust receivers for signal detection in

nmhieiit underwater noise, " I E E E Trans. Acous.

,h'peech ciizd Signal Proc., Vol. 37, pp. 621-626, 1989.

[4]

D.

I\/liddletoii, " Sta.tistica1 Physical Models of Elec-

t,roiiiagnetic interference, " IEEE Transactions on Elec-

troi~iagiietzc Coinpat., Vol. 19, pp. 106-127, 1977.