Joint source-channel coding and guessing

ISIT 1997, Ulm, Germany, June 29 - J u l y 4

Joint Source-Channel Coding and Guessing1

E r d a l Arikan and Neri M e r h a v

Dept. of Electrical and Electronics Eng., Bilkent University, 06533 Ankara, Turkey. a r i k a n Q e e . b i l k e n t

.

edu. t r Dept. of Electrical Eng., Technion - I.I.T., Haifa 32000, Israel. merhavQee. te c h n i o n . ac. il

Abstract - We consider the joint source-channel guessing problem, define measures of optimum per- formance, and give single-letter characterizations. As an application, sequential decoding is considered.

I . INTRODUCTION

A N D

MAIN RESULT

Let P be a discrete memoryless source over a finite alphabet

U , U a reconstruction alphabet, and d a single-letter distortion measure defined on

U

x U . A D-admissible guessing strategy for U N is a possibly infinite ordered list Q N = {UI, U*, . . .} C

U N such t h a t for each U E U N there exists h, E Q N with

d(u,&)

5

N D . T h e guessing function G N ( . ) induced by a guessing strategy Q N is the function t h a t maps each U E U N

into the index

i

of t h e first h, E Q N such that d(u,h,)

5

ND. Thus, GN(u) is the number of guesses required t o find a reconstruction of U within distortion level N D by sequentially probing from t h e list Q N . T h e moments E[GN(U)’], p

2

0: serve as measures of complexity for the guessing effort. Arikan and Merhav [l] defined the guessing exponent as

1

E(D,

p ) = lim - inf In E [ G N ( U ) ~ ] N-CO N O N

for p

2

0, and showed t h a t it has a single-letter form given by

E ( D , P ) = m$PR(D,Q) - D(QIIP)I (2) where R ( D , Q) is t h e rate-distortion function, D(QIIP) is the relative entropy, and t h e maximum is over all probability dis- tributions on U .

T h e aim of this talk is t o consider the guessing problem in a joint source-channel setting, in which one is allowed t o send information about U t o the guesser over some discrete memoryless channel W , using the channel X times for each source symbol. We assume W has a finite input alphabet

X

and a finite output alphabet

y .

T h e source output U E U N

is encoded into a channel input block X

E

X “ , h’ = [AN], using an encoder eN : U N +. X ” , X is transmitted over W ,

and the guesser observes the channel output

Y

E

y K .

A D- admissible guessing scheme, in this situation, is a collecti?n

{ G N ( Y ) ,

y E

y“},

such t h a t for each y E

y“,

G N ( Y )

C

U N

is a D-admissible guessing scheme for

U N

in the previously

exponent as

defined sense. We define the joint sour‘echannel guessing

Here, GN(UIY) denotes the guessing function induced by BN(Y). Our

main

result is t h e following.

‘The work of N. Merhav was partially supported by the Wolfson Research Awards administered by the Israel Academy of Sciences and Humanities.

Theorem 1 The joint source-channel guessing exponent is given by

E s c ( D , p ) = max(0, E ( D , P ) - X E o ( p ) } (4) where E o ( p ) is Gallager’s function [2, p.1381 for W .

11.

LIST-ERROR

EXPONENT

Consider list-decoding in the above situation so t h a t given the channel output Y one is allowed t o generate f estimates of the source output U and suppose an error occurs only if none of the estimates is within distortion level

N D

of

U.

Let P e , ~

denote the minimum possible value of the list decoding error probability over all encoders e N and all l i s t 4 decoders. T h e asymptotic behavior of P e , ~ for f = 1 has been considered by Csiszir, but it remains only partially known. Here, we consider exponential list sizes, f = e N L , and define the joint sourcechannel Iist-error exponent as

1

N - w N

F,,(L, D ) = lim --log pe,N. _{( 5 )}

Our second result is the following.

Theorem 2 The joint source-channel list-error exponent i s given by

F,,(L, D ) = inf [ F ( R , D )

+

X E S ~ [ ( R

-

L)/X]] where F ( R , D ) i s Marton’s source coding exponent

E S P ( .) i s the sphere-packing exponent [2, p. f57].

(6) and R Z L

111.

APPLICATION

TO

SEQUENTIAL

DECODING

Koshelev [5] considered using sequential decoding in joint source-channel coding systems for the lossless case

D

= 0. Here, we prove the following converse which complements his result, and applies t o the lossy case D

>

0 as well.

Theorem 3 For any p

2

0, the pth moment of computation in sequential decoding in a joint source-channel coding system must grow exponentially with the number of symbols correctly decoded if

E ( D ,

p )

>

AEo(p).

REFERENCES

E. Arikan and N. Merhav, “Guessing subject to distortion,” Technion -I.I.T., EE Pub. No. 1015, Feb. 1996 (Also, submitted to IEEE Trans. Inform. Theory).

R. G. Gallager, Information Theory and Reliable Transmission.

New York: Wiley, 1968.

I. Csiszir, “On the error exponent of source channel transmis- sion with a distortion threshold,” IEEE Trans. Inform. Theory, vol. IT-28, no. 6, pp. 823-828, Nov. 1982.

I<. Marton, ‘‘Error exponent for source coding with a fidelity criterion,” IEEE Trans. Inform. Theory, vol. IT-20, pp. 197- 199, 1974.

V. N. Koshelev, “Direct sequential encoding and decoding for discrete sources,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 340-343, May 1973.