A general purpose VLSI median filter and its applications for image processing

(1)

A GENERAL PURPOSE VLSI MEDIAN FILTER

AND ITS APPLICATIONS FOR

IMAGE PROCESSING

Mustafa Karaman, Levent Onural, and Abdullah Atalar

ABSTRACT

A general purpose median filter configuratioii consisting of two single-chip median filters is proposed. One of t h e chips is designed for the applications requiring variable word-length and variable window size whereas t h e other one is for real-time applications. T h e architectures of t h e chips are based on

t h e odd/even transposition sorting. T h e cliips are implemented in 3 - p m M2CMOS by using full-custom VLSI design techniques. T h e chips together with a

reasonable external hardware can be used for tlie realizations of many median filtering techniques. In

this paper, tlie VLSI design procedure of t h e chips and their applications t o different median filtering techniques for image processing are presented.

1. INTRODUCTION

T h e median of a n odd number of elements is de- fined as t h e middle element when the elenieuts are sorted. O u t p u t of a median filter is the median of its input d a t a , a n d t h e resulting nonlinear smoothing filter can filter out tlie impulsive noises from signals and images while preserving t h e edge-information

[l]. Such filters are frequently used in many sig-

iial a n d image processing applications. I n terms of impulsive noise suppressioa, edge preservation, and ease of design, t h e performance of median filters are better t h a n tlie other smoothing filters such as linear filters [2] a n d generalized mean filters [3].

l n 1-D a n d 2-D standard median filtering applications, a window of size w, w is odd, moves on t h e sampled values of t h e signal or image, and then t h e median of t h e samples within t h e window is com- puted a n d written a s t h e output element at t h e loca- tion of t h e center of tlie window. Theoretical analysis a n d applications of the median filters can be found in t h e literature [4,5]. Mostly, median filters a r e implemented in general purpose computers [6,7]. However, there a r e also hardware implementations for faster filtering purposes [8]. Because of the low VLSI cost of sorting structures, most of the hard- ware median filtering algorithms are based on sort- ing [9].

I n order t o increase tlie performance of tlie median filters for particular applications, various techniques such as weighted [IO], separable [ll], recursive [12], adaptive-length [13], generalized [14], selective [15], hybrid [lG] median filtering techniques have been de- veloped. T h e computation of the median of a group of elements is tlie fundamental operation in all those techniques. Thus, tlie standard median filters are used as tlie basic components for the realizations of other techniques.

Tlie window size of t h e median filter and t h e word- length of t h e elements are not tlie same in different applications. Also, t h e required speed of t h e filter- ing operation varies depending on tlie application. I n order t o meet these changing demands, a general purpose VLSI median filter unit which coiisists of two single-chip median filters, one extensible and one real-time, is designed. T h e extensible median filter chip is designed for t h e applications requiring variable word-lengths a n d variable window sizes whereas t h e real-time median filter chip is for tlie real-time median filtering applications. The architectures of t h e chips are bit-level pipelined systolic structures based o n tlie odd/even transposition sorting. T h e cliips are implemented in 3 - p m M2CMOS by using full-custom VLSI design techniques. I n tlie following sections, t h e architectures, VLSI implementations, and some possible applications of tlie chips are presented.

2. ARCHITECTURES

2.1 Extensible Median Filter Architecture

T h e extensible median filter is an odd/even trans- position sorting network which is a pipelined regu- lar structure consisting of 9 compare-and-swap stages (Fig.1.a). Each stage consists of 5 bitwise compare- and-swap units. Each of these units compares two one-bit numbers at its inputs and interchanges t h e m if necessary so t h a t tlie larger one is at t h e “top”. At t h e o u t p u t of t h e last stage, t h e d a t a will be sorted such t h a t t h e largest will b e a t t h e top, a n d tlie median will be in t h e middle. At each clock, one bit from each word (total of 9 bits) enter t h e network and one bit of tlie median is obtained a t t h e output. Tlie flow is from t h e most significant bits toward the least significant bits both a t t h e input and a t t h e

(2)

S. E. output. Because of t h e bitwise serial d a t a flow, this

structure allows arbitrary word-length, L .

Tlie bitwise compare-and-swap unit (CSU1) is a fi- iiite state niacliilie wliicll lias tlrree legal operatioli states: r q u n l . ~ ( I A S , and J U J ~ ~ . C S U l is set t o the equal state a t t h e end of eacli d a t a word by a reset signal. Thus t h e reset signal flows tlrrougl~ tlie stages of t h e network a t a rate of one stage per clock cycle by iiieaiis of the pipeliiied delay units. C S U l stays in equal s t a t e as long as its inputs are equal. However, it locks itself into one of tlie pass or swap states de- peiidiiig 011 its inputs aiid stays in t h a t s t a t e until

it is reset. T h e state diagram aiid tlie operations a t

different states are given iii Fig.1.b.

In t h e extensible median filter structure given in Fig.l.a, t h e upper and b u t e r e.ctcn~zora I/O’s (.rt,<,’s and

y2l0’s) are used t o extend the filter t o larger window sizes. For ti1 = 9, tlie upper arid lower exteii- sioii inputs are connected t o logic 1’s and logic 0’s so

t h a t the correspondiiig coinpare-and-swap units act

as delay units. O n tlie other hand, the design allows tlie iiitercoiiiiectioi~s of many of these chips to form mediaii filters for (11

> 9.

A i , Bi : om

-

bel Input dota

A., 8. one- bit output data COIIIPOf. R equal state (A ;B)*R unused stat. swap s l d * &&= Bi ,Bs A,) R ( b )

Figure 1: Tlie extensible median filter: a) arcllitec- ture, b) compare-and-swap unit (CSU1).

-

_{bitwise de!oy unii.}

/

-

rn,

x , y z, bits of the inputs corresponding ’he new elements In a

r”, I

’’

bit of the median 5, E,

3 x 3 sliding window

Test inputs to- testing of the blocks individually !a!

€3, €3, A,= i f So then B, else A,

B o = if Sothen A , else B,

Figure ArQ 2: T h e real-time ( b ) median filter: a) architec- S o = Si + E, A i < B i 1 Eo = E, ( A , = 8, 1

conpore

and s w o p

so Eo

ture, b) coinpare-and-swap unit (CSU2).

T h e extensible median filter generates its outputs with a delay of ti’

+

L clocks; and after tlie network

is full, it finds one L-bit iiiediaii per L clocks. Al- though, t h e resulting speed inay be sufficient for tlie real-time median filtering of 512 x 512 frames with L

<

3 , it is not enough for tlie real-time filtering of

1024 x 1024 frames with L

> 1.

2.2 Real-Time Median Filter Architecture

Tlie real-time iiiediau filter is designed by intercoii- iiectiiig 8 odd/evea transposition sorter blocks in parallel [9] (Fig.2.a). In this network, t h e d a t a enter in such a way t h a t t h e iiiost significant bits go t o tlie first block, t h e second iiiost significant bits t o tlie second block, and so on. Tlie bitwise compare-aiid- swap unit used in this network is slightly different tliaii t h a t of tlie extensible one, because t11e“swap” or ‘ipass” information flows froiii upper t o lower block

so t h a t tlie compare-and-swap unit takes this iiifor- mation, uses, updates and sends it out (Fig.2.b). For proper timing, t h e delay units are included a t tlie in- p u t aiid output of tlie network.

(3)

T h e real-time median filter has nine 8-bit d a t a inputs a n d it generates one 8-bit median per clock. At every clock, three new elements enter t h e chip, corresponding t o t h e new elements of a sliding 3 x 3 window. Since t h e clock period is determined by the delay of one compare-and swap unit (CSUS), recent VLSI technology allows t h e implementation of CSU2 a t a speed larger t h a n t h e real-time operation rate for t h e 1024 x 1024 frames with L = 8.

3. C H I P S

Both of the extensible and real-time median filter ar-

chitectures are regular arrays of the bitwise compare- and-swap units. Also, their internal communication schemes are simple and regular. This makes t h e VLSI implementations easy and straightforward [17,18]. T h e architectures are mapped t o hardware by using standard CMOS logic style [19] in 3 - p dou- ble metal n-well process. For generation of t h e chip layouts, a n d their simulations, full-custom VLSI CAD tools [20,21] are used: magic for layout editing, Spice, R n l , and Esim for simulations. T h e overall layouts of t h e chips are shown in Fig.3.

frequency up t o 40 MHz with a power dissipation less t h a n 800 m W at this frequency. I t generates one median per clock so t h a t its throughput is 40 mega medians/s. I t consists of about 22000 transistors and has a n area of 4 5 m m 2 (6.8 mmx6.6 mm) and 40 pins. T h e testing of t h e chips are easily accomplished by t h e functional test techniques [22] since t h e operations of t h e cells can b e selectively probed by using proper test vectors. T h e test vectors and t h e expected outputs are generated by using software tools written for these purposes. There are 500 test vectors for the extensible median filter chip, and 12,000 for t h e other one.

4. APPLICATIONS

In image processing applications, median filters are used mainly for noise suppression and for edge detection. For impulsive noise suppression, standard median filtering technique is a good choice. How- ever, for suppression of nonimpulsive noises other techniques such as adaptive-length, separable, recursive, and weighted median filtering techniques may be more convenient. For edge detection, generalized, hybrid, and selective median filtering techniques are frequently used. I n addition, t h e weighted median filtering can b e also used for edge detection by choos- ing the weight coefficients properly.

The designed median filter chips can b e selectively used in a processor environment by means of t h e chip enable signal t h a t each chip has. Furthermore, one can realize any median filtering technique mentioned above by using t h e extensible and/or the real-time median filter chips together with o r without a reasonable external hardware:

For t h e standard median filtering technique, the exact medians of t h e elements, in a window size w = 9 with arbitrary word length L , can be found by using only one extensible median filter chip. For w

>

9 with arbitrary L , a t most [[w/9]

1’

([[.I]’ indicates the smallest greater integer) chips are required t o find the exact medians. On t h e other hand, t h e real-time median filter chip can find t h e exact running medians of t h e elements in a window of a fixed size w = 9 with fixed word length L = 8 a t t h e real-time rate.

Figure 3: T h e layouts of the median filter chips: a ) real-time, b) extensible.

According t o t h e simulation results, t h e extensible median filter chip can run up t o a clock frequency of

30 MHz with a power dissipation less t h a n 2 5 0 m W a t this frequency. T h e throughput of the chip is about 3 0 / L mega medians/s. The chip consists of about 5000 transistors and has a n area of 11.7 mm2

(3 mmx3.9 mm) and 28 pins. O n t h e other hand, t h e real-time median filter chip can run with a clock

T h e extensible median filter is a favorable choice t o realize t h e adaptive-length median filters [13],

since one can change t h e window size from 3 to indefinitely large ones by using t h e extensible median filter chip(s) by applying logic 0’s o r 1’s t o unused inputs of t h e chip(s) appropriately. For t h e realizations of the weighted median filters

[lo],

t h e extensible median filter can b e used with a pipelined multiplier t o multiply t h e input d a t a with t h e weight coefflcients. Since all input d a t a of t h e chip are entered t o t h e chip directly at each move of t h e window, one can realize a n adaptive weighted median filter by changing t h e weight coefflcients a t each position of t h e window on t h e frame.

(4)

0 A pair of t h e extensible or t h e real-time median filter chips can be used as a selective me- dian filter [15] together with a n external control logic consisting of two full-word subtracter and a full-word comparator.

Either t h e extensible or t h e real-time median filter chip can be used as a line-recursive median Alter [13] by loading t h e window elements from t h e frame appropriately.

0 T h e chips can be used for t h e realizations of t h e

separable median filters [11] without any external hardware.

5. CONCLUDING REMARKS

A general purpose VLSI median filter unit consisting of two single-chip median filters and its applications are presented. T h e architectures of the chips are modular and have regular communication schemes which make t h e VLSI implementations rather easy a n d straightforward. Both of t h e architectures are not preferable t o be implemented at larger window sizes since t h e area is proportional t o t h e wz. We have chosen w = 9, because this is t h e most com- monly used window size in two dimensional median Altering applications.

T h e main contributions of this study are t h e architecture of t h e extensible median filter and its VLSI implementation. Another achievement of this study is t h e implementation of the real-time median fll- t e r which can operate at t h e real-time rate for t h e 1024 x 1024 resolution frames.

ACKNOWLEDGMENT

This research was sponsored by NATO’s Scientific Affairs Division in t h e framework of the Science for Stability Programme.

References

[l] J. W. W e y , “Nonlinear (nonsuperposable) methods for smoothing data,” in Conf. Rec., p. 673, EASCON 1974. [2] A. V. Oppenheim and R. W. Schafer, Digital Signal Processing, Englewood Cliffs, N.J.: Prentice-Hall, 1975. [3] A. Kundu,

S.

K. Mitra, and P. P. Vaidyanathan, “Appli- cation of two-dimensional generalized mean filtering for removal of impulse noises from images,” IEEE %ns.

Acowtic, Speech, and Signal Processing, vol. ASSP-32, NO. 3, pp. 600-609, Jun. 1984.

[4] N. C. Gallagher, Jr., and G. L. Wise, “A theoretical analysis of the properties of median filters,” IEEE B a n s . Acowtic, Speech, and Signal Processing, vol. ASSP-29, pp. 1136-1141, Dec. 1981.

[5] E. Ataman and E. Alparslan, “Application of median filtering algorithm to images,” Electronics Division, Marmara Research Institute, Gebze, Turkey, Tech. Rep.

U1 78/10, Sep. 1978.

[6] E. Ataman, V. K. Aatre, and

K.

M.

Wong,

“A

fast method for real-time median filtering,” ZEEE h n s .

Acowtic, Speech, and Signal Processing, vol. ASSP-28,

pp. 415-421, Aug. 1980.

[7] V. V. B.

Rao

and K.

S. Rao,

UA new algorithm for real- time median filtering,” IEEE h n s . Acowtic, Speech, and Signal Processing, vol. ASSP-34, pp. 1674-1675, Dec. 1986

(81 D. L. Knuth, The Art of Computer Progmmming- Searching and Sorting, vol. 3. Reading MA: Addison- Wesley, 1973.

[9] K. Oflazer, “Design and implementation of a single-chip 1-D median filter,” IEEE %ns. Acowtic, Speech and Signal Processing, vol. ASSP-31, pp. 11641168, Oct. 1983.

[lo] T. Loupas, W. N. McDicken, and P. L. Allan, Noise reduction in ultrasonic images by digital filtering, “ The

British Journal of Radiology, vol. 60, pp.389-392, Apr. 1987.

[I11 T. A. Nodes and N. C. Gallagher, Jr., “Two- dimensional root structures and convergence properties of the separable median filter,” IEEE !?+am. Acowtic, Speech, and Signal Processing, vol. ASSP-31, pp. 1350- 1365, Dec. 1983.

[12] C. G. Boncelet Jr., “Recursive algorithms and VLSI implementations for median filtering,” Proc. of IEEE ISCAS‘88, pp. 1745-1747.

[13] H. M. Lin and A. N. Willson, Jr., “Adaptive-length median filters for image processing,” Proc. of IEEE IS- CAS‘88, pp. 2557-2560.

[14] Y. H.

Lee

and

S.

A. Kassam, “Generalized median filtering and related nonlinear filtering techniques,” ZEEE B a n s . Acoustic, Speech, and Signal Processing, vol. ASSP-33, pp. 672-683, Jun. 1985.

[15]

S.

J. KO, Y. H. Lee, and A. T.

Fam,

“Selective median filters,” Proc. of IEEE ISCAS‘88, pp. 14951498. [16] Y. Neuvo, P. Heinonen, and I. Defee, “Linear-median

hybrid edge detectors,” IEEE Bans. Circuih and Sgs-

terns, vol. CAS-34, pp. 1337-1343, Nov. 1987. [17] M. J. Foster and H. T. Kung, “The design of special

purpose VLSI chips,” IEEE Computer, pp. 26-40, Jan. 1980.

(181 H. T. Kung, “Why systolic architectures?,” IEEE Com- puter, pp. 37-46, Jan. 1982.

[19] N. Weste and K. Eshraghian,Pltnciples of CMOS VLSI Design, Reading MA: Addison-Wesley, 1985.

[20] Berkeley C A D Tools User’s Manual, EECS Dep., Uni- versity of California at Berkeley, 1986.

(211 VLSZ Tools Reference Manual, TR#87-02-01, Release 3.1, NW Lab. Int. Sys., Dep. Computer Sci., University of Washington, Feb.1987.

[22] J. A. Abraham and W. K. Fuchs, “F‘ault and error mod- els for VLSI,” ZEEE Proc., vol. 74, pp. 639-654, May 1986.