Compression of images in CFA format

COMPRESSION OF IMAGES IN CFA FORMAT

Halil

L Cuce, A. Enis Cetin

Department of Electrical and Electronics Engineerin~

Bilkent University, Ankara, Turkey

E-mail:

{hic,

cetin}

@ee.bilkent.edu.tr

ABSTRACT

Inthis paper, images in Color Filter Array (CFA) format are

compressed without converting themto full-RGB color im-ages. Greenpixels are extracted from the CFAimage data andplaced in a rectangular array, and compressed using a transformbased method without estimating the correspond-ing luminance values. Inaddition, two sets ofcolor

differ-ence (orchrominance) coefficients are obtained correspond-ingto the red and blue pixels ofthe CFA data and theyare

also compressed using atransformbased method. The

pro-posed method produces betterPSNRvaluescomparedtothe standardapproach ofbilinearinterpolation followed by

com-pression.

Index Terms- Image coding, discrete cosine transforms,

quantization,Huffmancodes

1. INTRODUCTION

Mostcolorimagingsystemsand circuitsuse acolor filter ar-ray (CFA) to select different wavelength bands at different photosensor locations. TheBayerfilter withamosaicpattern

showninFig. 1iscommonly utilizedindigitalcameras[1, 2].

Camerasusing thispatternproduce only onecolor valueper

pixel. Onthe otherhand, digital image coding techniques and standards, including JPEG andJPEG-2000, weredeveloped

forfullRed, Green, and Blue (RGB) color images inwhich each pixeloftheimage consists ofthree color components [3, 4, 5]. Inorder to compress digital images produced by imaging sensors usingCFAarrays the standard procedure is

tointerpolate the CFAimage datatothe fullRGB color for-matand compressthe interpolated full color image using an

image compression standard. The interpolationprocess

ini-tially increases the data size three-folds and thisnotonly in-creasesthecomputational costbut alsorequiresanadditional

memory spaceinside thedigitalcamera.

Inthispaper, images inColor Filter Array (CFA) format are compressed without converting them to full-RGB color images. GreenpixelsareextractedfromtheCFAimage data andplaced in a rectangular array, and compressed using a transformbased method without estimating the correspond-ing luminance values. Inaddition, two sets ofcolor

differ-Mark K.

Davey

Grandeye Ltd. 6 Huxley Road,

Guildford, Surrey,

GU2 7RE, UK

E-mail: mark.davey

@

grandeye.com

Fig. 1. A 5by 5 section of a Colour FilterArray (CFA) in

Bayer format.

Gij, Rij

and

Bij

represent a green, a red, and

abluepixel, repectively.

ence (orchrominance) coefficients are obtained correspond-ingto the red and bluepixels oftheCFAdata and they are

also compressed usingatransformbased method. The main advantageofthe proposed method is that it gives more

em-phasis togreenpixels as human visionsystem [2]. Another advantage is that it doesnotrequire additionalmemory space forinterpolation insideadigitalcamera.

2. PROPOSED METHOD

The method described inthis article compresses digital im-ages in CFA formatwithoutconverting themtofull-RGB color images. Only one-third offull-color data is availablein an

image obtainedby a sensorusing theBayerfilter. InBayer pattern, thereare two greenpixels,oneredpixel andablue pixel in a twoby twoimage region as showninFig 1. The human visionsystem gets most ofits sharpness information

fromgreenlight, this is the mainreasonwhy thereare more greenpixels compared to red and bluepixels inBayer pat-tern [2]. Insteadofinterpolatingthe missing imagedatawe

rearrange the available data into other two-dimensional ar-raysrepresenting the color information andcompressthe

re-arranged data. For example, the available green color data

canbe re-ordered intoa(rectangular)two-dimensional array

asfollows:

Gi

G21 [gij] = G3 G13 G15 ...

G23

G25

...

G33 G35

...

Entries of the above matrix obey the following rule:

Gi,2j-g%j

=

Gi,2j

ifiis odd, and

if iseven

In this way, the green CFA data in the quincunx format in

theCFA arrayisrearranged intoatwo-dimensionalarray, and

aplurality ofsquare 8by 8blockscanbe obtained from the two-dimensionalarrayforDCTbased blockcompression pur-poses. Green pixel valuescanbe alsorearranged diagonally

asin[6,7]. Thisarrangementis discussedinthenextsection.

Inour scheme, the green data is compressed as is

with-outtryingtoestimate the luminance valuesasin mostcoding schemes. Since the human eye gives moreemphasis to the

greenchannel informationmoreemphasis should be givento

thegreenchannel information. This isoneof the main

differ-encesbetweenourmethod and otherCFAcompression meth-ods [1, 2, 9, 10, 11] which estimate aluminance value as a

weightedaverageof thecurrentgreenvalue andneighboring red and blue values. Estimated luminance valuesare not

per-fect because the actual red and blue values are notavailable forgreenpixelsin a CFA array. This is the mainreasonwhy

wegetbettercompression results then methods estimating the luminance value forgreenpixels.

Red and blue channel data are encodedusing color dif-ferencing approachorchrominance data compression. Color difference values are obtained from the two-by-two blocks of the CFA array as well. For example, consider the first two-by-two block of the CFA data containing the samples

(Gil,

R12, B21,

G22)

of the array shown in Fig. 1. From

these four valuestwo color difference values for the red and bluepixelsareobtainedasfollows

Ua1 R12 GII+G22₂

(3) vll B21 GII+G22

respectively. Ingeneral, acolour difference value for each red pixel is obtained using the formula:

color space. The size ofU and V matrices are half of the

G matrix as in 4:2:2 subsampling scheme. After this step,

the U and V array data are divided into plurality of blocks

(1) and the Discrete Cosine Transform of the blocks are

com-puted. Transform domain data is quantized and Huffman

en-coded using the quantization tables of the JPEG standardin our simulations studies described inSection 4. Other com-pression methods and standards including the wavelet based

JPEG 2000 standard can be also applied to thisrearranged

(2)

data forcompression.

3. QUINCUNX SCANNING OFTHECFADATA

Thereareotherwaysof rearranging thegreencolor values of the CFAdata. Green color filter of the CFA isin quincunx format. Therefore, the filterarray canbe rotated 45 degrees and two-dimensionalplurality of blockscanbe extracted from the diamondshaped partitions of theCFAdata. Forexample, thefollowing two-dimensional(rectangular)array canbe ob-tained fromFig. 1 asfollows:

[ G31 G22 G13 ... G42 G33 G24

[dij]

=

G53 G44 G35

..

(6)

Theoriginal data actuallycomesfromadiamond-shaped

re-gion from the CFA array as shown inFig. 2. In this way,

the green orthe corresponding luminance data canbe

rear-ranged into two-dimensional plurality ofrectangular blocks and compressed by an ordinary Discrete Cosine Transform basedimage coder. Greencolor filter of the CFAisin

quin-cunxformat with periodicity vectors [1 1]T and [1 1]T.

In other words, these vectors describe the quincunx array:

Q = {[n

k]T

n[1

l]T

+ k[l _1]T}. Thearray Q includes

indexpairs [n, k] = [0, ], [1, 1], [1, -1], [2,0 ], [2, 2], [1, 3],

[3, 3], ... ofa two-dimensional array but it does not include

all pairs of indices ofatwo-dimensional array for example

[0,1],

[1, 0], [2,1],

...etc. Theperiodicity matrix describing

thequincunxarray is formed from the periodicity vectors:

p I-1 1

₍₇₎

G2i-l,2j I +

G2i,2j

2

and, similarly, acolour difference value for each bluepixel is

obtained:

vij =B2i,2j-1 G2i-l,2j

I +G2i,2j

2

Inthisway,the data inBayer color representation is converted

into Green - color difference U- color difference V (GUV)

This matrixcanbediagonalizedusing the Smith-normal form as described in the article by Gunduzhan, Cetin and Tekalp [6]. Diagonalizationprocessdefinesavariety ofmeans

ofmapping the quincunxarrayintoatwo-dimensional (rect-angular) array. Inthe caseofa quincunx array, the relation definedby the matrix pTmapsthequincunxarrayintoa two-dimensionalarray.

t 1 -1 i

I 1I (8)

1142

Fig. 2. The methodto extract 8x8 blocks for Green values from CFA image.

As aresult, the following rule is a waytomapthe quincunx

arrayintoarectangular two-dimensional array,

g[i,j] =G[i+j,i -j] (9) The re-ordered datag[i,j] covers all the indices of a

two-dimensional array, i.e, g[O,0] = G[O,0], g[l, 0] = G[1, 1],

g[1,1] =

G[2,0], g[0,1]

= G[1 -1],.... Once the

origi-nalquincunx data in the CFAarray is in the formofa

two-dimensional arraythen the datacanbe divided intoplurality

of blocks and transformed intocompression domain using the DCT.

4. SIMULATION EXAMPLES

Wetesttheproposed CFA image compression method using

some well-known images: Lena, peppers, monarch, girl, by

assuming the availability of only one color value for each

pixel in the CFA format. Also, we use someother images:

lighthouse, a widely used image to test the quality of CFA demosaicking algorithms [8] andanimage from the Halocam

cameraofGrandeye Ltd [12].

Inordertocomparetheperformance of theproposed method

withJPEG, Peak Signal toNoise Ratio (PSNR) measure is

used. The PSNR iscomputedasfollows:

MSE= xh

E EZ

[O(x,y) -R(x, y)]2

(10) PSNR

(in dB)

= 20

log,

where0and Raretheoriginal and the reconstructed images,

respectively. 55F Proposed - --JPEG 50F co 45-z_Cl) 0- 40-10 20 30 CR 40 50 60

Fig. 3. Average PSNR values for sixtestimagesfor the

pro-posed method and the JPEG compression.

Table 1. PSNRvalues oftestimages forvariousCRvalues.

CR Girl Lena Monarch

CR Jpeg Ours Jpeg Ours Jpeg Ours

5 39.37 46.96 40.02 49.89 39.78 50.15 16 36.58 38.85 37.24 37.80 37.49 41.22 15 34.96 35.23 35.07 34.97 35.85 37.06 20 33.97 33.49 34.11 33.45 34.60 34.33 30 32.61 31.68 33.11 31.46 32.90 30.98 40 31.59 30.63 31.84 30.34 31.49 29.41 60 29.91 29.54 130.12 29.07 129.30 27.72 CR Lighthouse Peppers Grandeye

Jpeg Ours Jpeg Ours Jpeg Ours

5 35.23 45.98 38.56 46.91 44.62 55.48 10 33.28 36.65 35.23 36.23 41.79 43.11 15 31.96 32.32 33.72 33.67 40.72 41.09 20 30.77 30.31 32.74 32.12 40.13 40.12 30 29.43 28.22 31.49 30.40 39.16 38.78 40 28.47 27.19 30.50 29.39 38.70 37.89 60 27.06 26.82 129.02 28.10 137.79 36.97

Average PSNR valuesversusCompression Ratio (CR) val-uesfor these siximages are shown inFig. 3. GUV planes

of the images are compressed using the DCT and Huffman

tables of the JPEG standard. Theseimages are also

interpo-lated toRGB images using bilinearinterpolation and JPEG compressed. The JPEG compression results arealsoplotted

inFig. 3. PSNR values of individual test images for

vari-ous CR values are given in Table 1. The proposed method

produces better PSNR when the compression ratio is below CR=15 comparedtothe standard approach of bilinear inter-polation followed by the JPEG compression. After CR=15

one cannotice severe compression artifacts in mostnatural

images in JPEG standard. Because of this higher

compres-sion ratios arerarely usedinpractice.

Fig. 4shows differenceimages between the original

im-ages and the images decompressed by the proposed method

1143 RRGR G R G R G G GBGB G G B G G B G B

IR

G R G

IR

G G

IR

G G

LR LG

R

LG

R

GlE

R

GI

(G G Gc GBGa.BG B D G B CRG R .G RG R G R G R G R G R G PR G RG

loRIGG

RG GR~GP G GB B G BGB G1G G RI

k1R

R G R G R G R PGR .G R G

IXGI a

G

IB

G B

IG EB

G B

IG IE BA

G

v I

R G a R 6

S

R G R G R

G__

G G B G 6 B G B G~B G 0 G GB

.G

B

Lt

a Ri 0 R

WIN;

G R G G R G R

.G

R

GI

1G

x

G

Ggl

0B GBI G G8 G G.

E

s

SR

G RI R G r

tIR

C.

Y

GR

G R G R G R 0

B G XSx G iB

No

G

BW

G B

_{W,RWFWx.go.RG/GWWR.SR..G..I~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~...}

B

~R

G RU G R G R G R G RG WX..GR G IG BEG B3IEMG B G 6. G; B G+B G B G B G a .R

G__

iG

R!

G

S_,_

R

E

G R G

6. REFERENCES

(a)

(b)

(c)

Fig. 4. A portion from the lighthouse image. (a) Original. (b) and (c) Differences between original image and images reconstructed by the proposed method and JPEG atCR=5, respectively. Difference image=128+4*abs(original -

recon-structed).

andJPEG inorder to show the improvement ofourmethod since human eye system may not distinguish any difference when the actual reconstructedimages aredepicted. The dif-ferenceimagesaremultiplied by4 tomake themmorevisible. Theedges intheproposed methodare shaper than the edges

in JPEG asshowninFig. 4-b and4-c.

Although only one casefrom omnidirectional Grandeye

cameraimages is reportedin this section, extensive simula-tion studieswerecarriedoutusing Grandeye cameraimages and theproposed method outperformed the standard approach

inallcases.

5. CONCLUSION

Inthis paper, a compression scheme forCFA image data is presented. The proposed method has better performance than

JPEGwhencompression ratio is below 15in our simulation studies. The proposed method has also low computational

costandrequires lowmemorywhencomparedtothe standard methodsusing fullRGB values since theCFAimage data is one-third of theRGBimage data.

[1] Y. Tsai, K. T. Parulski, M. A. Rabbani, "Compression method and apparatus for single-sensor color imaging systems," US Patent No.5,065,229,Oct. 1, 1991.

[2] http://www.kodak.com/US/en/corp/researchDevelopment/ technology /Features/pixPic.shtml

[3] W. Pennebaker andJ. Mitchell, "JPEG: stillimage data compression standard," Van Nostrand Reinhold, NY, 1992.

[4] M.Rabbani, Rajan Joshi, "An overview of the JPEG2000

still image compression standard," Signal Processing: Image Communication, vol.17,pp. 3-48, 2002.

[5] JPEG 2000 Image Coding System, ISO/IECInternational Standard, 5444-1,2000.

[6] E. Gunduzhan, A. Enis Cetin, A. MuratTekalp, "DCT

Coding of Nonrectangularly Sampled Images,"IEEE Sig-nalProcessing Letters, vol. 1, no. 9, pp. 131-134, Sept.

1994.

[7] R. Ansari,A. E.Cetin and S. H. Lee, "SubbandCoding ofImagesusing Nonrectangular Filter Banks," in Proc.

of the SPIE 32nd Annual International Technical

Sym-posium: Applications of Digital Signal Processing, San

Diego,CA,vol.974, August 1988.

[8] B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W.

Schafer, and R. M. Mersereau, "Demosaicking: Color filterarray interpolation,"IEEESignalProcessing Mag-azine, pp.44-54,Jan.2005.

[9] S. Battiato, A. Buemi, L. DellaTorre, A. Vitali, "Fast VectorQuantization Engine forCFA DataCompression,"

inProc. of IEEE-EURASIP WorkshoponNonlinear Sig-nal andImage Processing, NSIP-2003Grado,Italy,June 2003.

[10] Sang-YongLee,Ortega, A,"Anovelapproach ofimage compression indigitalcameras withaBayer color filter array,"Proc. ofIEEEInternationalConferenceonImage

Proc., vol.3,pp.482-485, 2001.

[11] Koh, C.C. Mitra, S.K., "Compression of Bayer color

fil-ter array data," Proceedings ofIEEEInternational Con-ferenceonImage Processing., vol. 2,pp. 11-255-8,14-17

Sept.2003.

[12]

http://www.grandeye.com