On The One - Dimensional Distribution Function Of Semi - Markovian Random Walk Process With Reflecting And Delaying Barriers

©BEYKENT UNIVERSITY

ON THE ONE-DIMENSIONAL DISTRIBUTION

FUNCTION OF SEMI-MARKOVIAN RANDOM

WALK PROCESS WITH REFLECTING AND

DELAYING BARRIERS

Selahattin MADEN

Ordu University, Faculty of Arts and Sciences, Department of mathematics, 52750, Ordu, TURKEY

e-mail:maden55@mynet.com

Received: 23.03.2007 Revised: 04.06.2008 Accepted: 05.01.2009

ABSTRACT

In this paper, a semi-Markovian random walk process with reflecting barrier on the zero-level and delaying barrier on the P (P > 0) -level is constructed mathematically and the one-dimensional distribution function of this process is expressed by means of the probability characteristics of a renewal process

{Tn} and a random walk {Yn }.

ÖZET

Bu çalışmada, sıfır seviyesinde yansıtan ve P( P > 0) - seviyesinde tutan bariyerli bir yarı-Markov rasgele yürüyüş süreci matematiksel olarak kurulmuş ve bu sürecin bir boyutlu dağılım fonksiyonu bir {Tn} yenileme süreci ve bir

{Yn} rasgele yürüyüş sürecinin olasılık karakteristikleri yardımıyla verilmiştir.

Keywords: Semi-Markovian random walk, reflecting barrier, delaying barrier,

distribution function, renewal process, random walk.

1. INTRODUCTION

It is known that the most of the problems of stock control theory is often given by means of random walks or random walks with delaying barriers(see References 1-5, etc.). But, for the problem considered in this study, one of the barriers is reflecting and the other one is delaying, and the process representing the quantity of the stock has been given by using a random walk and a renewal process. The practical state of the problem mentioned above is as follows.

Suppose some quantity of a stock in a certain warehouse is increasing or decreasing in random discrete portions depending to the demands at discrete times. Then, it is possible to characterize the level of stock by a process called the semi-Markovian random walk process. The processes of this type have been widely studied in (see, for example References 1, 4 and 6, etc.). But sometimes some problems occur in stock control theory such that in order to get an adequate solution we have to consider some processes which are more complex than semi-Markovian random walk processes. For example, if the borrowed quantity is demanded to be added to the warehouse immediately when the qunatity of demanded stock is more than the total quantity of stock in the warehouse then, it is possible to characterize the level of stock in the warehouse by a stochastic process called as semi-Markovian random walk processes with reflecting barrier. But for the model studied in this study an additional condition has been considered. Since the volume of warehouse is finite in real cases, the supply coming to the warehouse is stopped until the next demand when the warehouse becomes full. In order to characterize the quantity of stock in the warehouse by these conditions it is necessary to use a stochastic process called as semi-Markovian random walk process with two barriers in which one of them is reflecting and the other one is delaying. Note that semi-Markovian random walk processes with two barriers, namely reflecting and delaying, have not been considered enough in literature (see, for example References 6-11).

The Model. Assume that we observe random motion of a particle, initially at

the position Xo e [ 0 , P ] , P > 0, in a stripe bounded by two barriers; the one lying on the zero-level as reflecting and the other lying on P -level as delaying. Furthermore, assume that this motion proceeds according to the following rules: After staying at the position X0 for as much as random duration E, 1, the particle wants to reach the position X0 + n1. If

X0 + n1> P then the particle will be kept at the position X1 = P since there is delaying barrier at P -level. If X0 + n1 £ [0, P ] , then the particle will be at the position X1 = X0 + n1. Since there is a reflecting barrier at zero-level, when X0 + n1< 0 the particle will reflect from this barrier as long as

|X0 + n i | . In this case, if |X0 + m | - P then the particle will be kept at the position X1 = |X0 + n 11 and if |X0 + n 11 > P then the particle will be at the position P , so that the particle will be kept at the position

After staying at the position X1 for as much as random duration E, 2 again it will jump to the position X2 = m i n { ( ; | X j + n 2 } according to the above mentioned rules. Thus at the end of n.th jump, the particle will be at the position Xn = m i n { ( ; | Xn -1 + nn} , n > 1.

Now, we construct mathematically this process.

2. CONSTRUCTION OF THE PROCESS X(t)

Suppose { ( E , i , n i ) } , i = 1,2,.. is a sequence of identically and independently distributed pairs of random variables, defined in any probability space ( Q , 3 , P) such that E,i's are positive valued, i.e., P{E,i > 0} = 1. Also let us denote the distribution function of E, 1 and n1

0 ( t )

=

P{^

1

< t}

,

F (x)

=

P { n

1

< x}

,

t e R +

,

x e R.

(2.1)

respectively. Before stating the corresponding process let us construct the following sequences of random variables:

Tn

=

i , Yn

=

£ n i ,

n > 1

; Yo

=

To

= 0.

(2.2) i=1 i=1

Then the processes { Tn: n > 1} and { Yn: n > 1} forms a renewal process and a random walk respectively. By using the random pairs (E, i, n i ) we can construct the random walk process with two barriers in which the reflecting barrier is on the zero-level and the delaying barrier is on ( > 0 -level as follows:

Xn = min{( ;|Xn-1 + n n | } , n > 1; z = Xo e [ 0 , ( ] , (2.3) where z is the initial positionof the process. Now, let us construct the

stochastic process X( t) which has the reflecting barrier from below and the delaying barrier from above and which represents the level of stock at the moment t:

This process is called the semi-Markovian random walk with the reflecting barrier on the zero-level and the delaying barrier on P -level.

3. ONE-DIMENSIONAL DISTRIBUTION FUNCTION OF

X(t)

The main aim of this study is to express the one-dimensional distribution function of the process X ( t) by the probability characteristics of a renewal process {Tn} and a random walk {Yn }. In order to formulate the main result of this paper let us give the following notations:

a

n

(z;x) = P{z + Y

i

e[0,P];1 - i - n;z + Y

n

e[0,x]}, n > 1

b

n

(z;P) = P{z + Y, e[0,P];1 - i - n - 1;z + Y

n

> P } , n > 1

c

n

(z;v) = P{z + Y, e[0,P];1 - i - n - 1;z + Y

n

< v}, n > 1

On(t) = P{Tn < t}, n > 1,

A O n (t) = O n (t) - O n+1(t), n > 1, and a0( z ; x ) = <x - z ) ; b0( z ; P ) = c0( z ; v ) = 0, < u ) = J1 < 0 where x , z

0,ß

v < 0.

Moreover, for any two sequences ( a n) and (P n) , the convolution product of ( an) and ( Pn) as follows:

n

a

n *

ß

n =

k

*

0 a

k

ß

n - k '

n

^

0

k = 0

and

( a )m = a * a * ... * a , m-times convolution product of ( an) with

v

n * n n n

1 v n /

Let Q(t; z ; x ) denote the one-dimensional distribution function of the process X ( t ) , that is,

Q(t; z;x) = P

z

{X(t) < x} = P{X(t) < x\ X(0) = z}, x,z e [0,p] , t e R+

Now, we can formulate the following theorem.

Theorem 3.1 The one-dimensional distribution function of the process X ( t ) , in terms of the probability characteristics of a renewal process {Tn} and a random walk {Yn } , is given by

TO

Q(t; z; x) = e(x - z) [1 - O(t)] + £ q(n; z; x)

A On

(t),

(3.1) n=1 where TO u u

q (n;

z

;x)

=

g

n

(

z

;

x

)

+ £ j...(

k

)... j"

k=1 —TO —TO k

n * ( (

v

i - 1 ;

d v

i ) ) * g n (

v

k ; x )

, i=1

in{(; | v

i

|},

i

>

1,

n

>

1

k=1 —to —to (3.2)

v

0

=

z ,

v. = min

g

n

(z;x) = a

n

(z;x) + b

n

(z; p)* a

n

(p;x) + b

n

(z; p)* U

n

( ( ; ( ) * a

n

((;x),

n > 1 ,

r

n

(z; v) = c

n

(z;v) + b

n

(z; ( ) * c

n

(p;v) + b

n

(z; ( ) * U

n

(p; ( ) * c

n

(p;v),

n > 1 ,

TO k U n

(P;

P ) = Z ( b n

(p; p ) )

k ,

n >

1 k=1 and g0

(

z

;

x

) =

a0

(

z

;

x

) = s (

x -

4

r0

(

z

;

v

) =

0

.

TO

Q(t;z;x) = 2 Pz { < t < Tn

+1

;X(t)< x}

n = 0 w = 2 Pz {Tn < t < T n + ı ; X n < x } n=0 TO = 2 P z { X n < x } P { T n < t < T n + ı } n=0 TO

= 2 q(n;z;x) AOn (t), (3.3)

n=0 where

q(n;z;x) = Pz {X

n

< x}, n > 0. (3.4)

Now we express q(n; z; x) by the probability characteristics of a renewal process {Tn} and a random walk {Yn }. For this aim we define v n, n > 1, as

v

0

= 0,

v 1 = min{k > 1: Xk—1 + n k < 0},

v

n

= min{k > v

n

_1 + 1: Xk-1 + n k < 0}, n > 2.

We note that the sequence of random variables { v n}, n > 1, can be take to mean the reflection moments of the markov chain { Xn} from the reflecting barrier. Then for the probabilities q(n; z; x) , n > 1, we have

TO

q(n; z; x) = £

Pz { v k

< n <

v k + 1

; X

n

< x}.

(3.5)

k = 0

We must calculate every term in (3.5) seperately. In the special case where

k = 0

, we get

Let us denote the probability on the right hand of this equality by gn (z; x) , that is,

gn(z;x) = Pz{V1 > n ; X

n

- x} ,n > 0. (3.6)

In order to state g( n; z) in terms of the probability characteristics of random walk {Yn }, n > 0 denote by V P ( n ) the number of moments in which the Markovian chain { Xn }, n > 1, goes down from the delaying barrier during the first n-step. Now, we construct this random variable mathematically. Assume that

M

" f1 , X

k - 1

< 0

v

P

( n )

=

Y x

k ,

n > 1; x

k

H

k 1 ,

k > 1.

PV ; k

t T

k k

10 , Xk-1 + n k > 0

Using V p (n) and applying the total probability formula we can write g(n; z) as follows:

n

g (z ; x) YP z { v 1 > n ; X n - x ; VP( n ) = m }

& n \ ' / = m = 0

(3.7)

If we consider random variables V1 ve V p ( n ) we can write

Pz {

>

n ; X n

- x ;

V p (

n) = 0}

=

P{z + Y, e[0,P];1 - i - n ; z + Y

n

e[0,x]} (3.8)

n

(

z

;

x

)

= a

where x, z

_0,P

n > 0.

For m = 1, by considering the definitions random variables V1 ve V 2 and the fact that ni, i = 1,2,..., are identically and independently distibuted random variables we have

Pz { v ı

>

n ; X n

< x ; v

ß

( n) = l} =

2 P { z + Y

i

e[0,ß];1 < i < k - 1;z + Y

k

> ß}

k=1

P{ß + Yi e [ 0 , ß ] ; 0 < i < n - k ; ß + ^ e[0,x]}

2 bk (z;ß)an - k (ß;x) k=1

= 2 bk( z ; ß ) an - k( ß ; x ) , where b0( z ; ß ) = 0 for z e [ 0 , ß ] . Therefore, k=0

we can write

P

z { v 1 > n ;

X

n < X ;v p ( n ) = 1} = b n (z; ß)* an (ß;x). (3.9)

On the other hand we have

Pz { 1

>

n ; X n

< x ; v

p

( n) = 2}

=

2 P{z + Y e[0,ß];1 < i < k - 1;z + Y

k

> ß }

2 < k + r < n k,r >1

P{ß + Yi g[0,ß];1 < i < r - 1 ; ß + Y

r

> ß}

P{ß + Yi g[0,ß];1 < i < n - k - r ; ß + Y ^ e[0,x]}

2

bk

(z; ß)b

r

(ß; ß)a

n - k - r

(ß;x)

2 < k + r < n k,r >1 2 bk (z; ß)br (ß; ß)an-k - r ( ß ; x ) 0< k+r < n k,r >0 = bn (z; ß ) * bn (ß; ß ) * an ( ß ; x ) . (3.10)

Continuing in a similar way it is possible to prove that

P z { 1

> n ;

Xn

<

x

; v

p

( n ) =

m

}

=

b n ( z ; ß ) * ( bn ( ß ; ß ) ) m- 1 * an ( ß ; x )

Substituting (3.8)-(3.11) in the formula for g(n; z) is given in (3.7) we have

g n (z;x) =pz { v i > n ; x n < x )

x

= a

n

(z;x) + b

n

( z ; p ) * a

n

( p ; x ) b

n

( z ; p ) * ( b

n

( p ; p ) ) ) * a

n

( P ; x ) .

m=2

Therefore we can write gn

(

z

;

x

)

=

a

n

(z;x) + b

n

(z; p)* a

n

(p;x) + b

n

(z; p)* U

n

(p; p)* a

n

(p;x)

for

n > 1

and

go(z;x) = a

0

(z;x) = P{z e[0,x]) = P{z < x) = e(x - z).

Since n i , i = 1,2,..., are identically and independently distibuted by the definition of random variable v1 we get

Pz { V 1 < n < v 2 ; Xn < x ) = £ J Pz { V 1 = k ; X k - 1 + n k e d v ) . k=1 —x

Pv{V1 > n — k;Xn—k < x ) , (3.12)

where v = m i n { p ;|v|) and n > 1. Now we denote rk (z; v) as

rk ( z ; v ) = Pz { V 1 = k ; X k — 1 + n k < v ) , k > 1,

ro

(

z

;

v

) =

0

z e [ 0 , p ] and v < 0 . Also let us rk ( z ; d v ) = dv (rk (z; v)). Thus, we can write (3.12) as n 0 Pz {V1

<

n

<

V 2

;

Xn

<

x)

= Z j

rk

(

z

;

d v

) g

n—k

( (

x

)

k=1 —x 0 n

= J Z

rk

(

z

;

d v

) g

n—k x

)

—x k = 1

= j rn (z; dv)* gn ( v ; x ) . (3.13) - w

In order to develop the general formula we have 0 0 P z { v 2 < n < V 3 ; Xn < x } = £ j j P z { = k ;k - 1 + n k e ^ } 2 < k + m < n - w - w k,m>1 Pv , { V 1 = m;Xm - 1 + n m e d v2 } Pv2 { > n - k - m;X n- k -m < x} 0 0 = j j S rk (z;d v1 )rm (v1 ;d v2 ) gn- k -m (v2 ; x) - w - w 2 < k + m < n k,m>1 0 0

=

j j S

rk ( d v1 )rm ( ( ; d v2 )gn - k - m ( (

;

x) , -<» 0 < k + m < n k ,m>0

where r0 (z; dv) = r0 ( v1; dz) = 0 Therefore we can write

0 0

P z { < n < V 3 ;Xn < x } =

j

j r n ( z ;dv , ) * r n ( v , ;dv2) * g n ( v2; x ) (3.14)

In a similar way, it is possible to prove that k > 2 için

0 0 Pz { v k

<

n < v k + 1 ;Xn

< x}= j . . .

( k )

. . . j

r n

( z ; d v ) *

r

n ( 1 ) * . . .

- w - w . . . * rn ( ( - 1 ;d vk ) * g n (vk ;x)

= j...(k)...

j ] [ = I * ( rn( i - 1

; d v i ) ) * g

n

( ( ; x )

(3.15) - w - w i=1

for k > 2 , where v0 = z . Substituting these expressions in the formula given for q ( n ; z ; x ) n > 1 için,v0 = z ve vi = m i n { p ; vi} , i > 1 olmak üzere

q(n;z;x)= P

z

{X

n

< x}

, n 0 0 k , \ \ /

g n (z; X) + Z j - ( k ) - j n * ( r " ( ; d vi ) gn (vk ;x) -OT i=1

iin{ß; |vi I}, i

k = 1

for

n

>

1, v

0 =

z

and

v

( =

min{

i >1,and

q(0; z; x) =

go

(z; x) =

ao

(z; x) = e(x - z )

1, x > z

0, x < z

To comlete the proof of the theorem it is sufficient to express rn (z; dv) in

terms of the probability characteristics of random walk { Yn} . For this reason,

by the definition of Vp ( n ) we can write rn (z; v ) , n > 1, as follows:

rn

(

z

;v) =

Pz {V1

= n

;Xn - 1

+ n „ < v <

0}

Y Pz { V P ( „ ) = m ; V 1 = n; X„ - 1 + n n - v <0} (3.16)

m= 0

In this case, when m = 0 we have

P { vß( n ) = 0 ; V1 = n ; Xn- 1 + n n < v }

= P{z +

Y j

<= [0, ß] ;1 < i < n - 1;z + Y

n

< v < 0}

= cn (z; v ) . (3.17) Also if m = 1, then P{vp(n) = 1; V1 = n ; X n - 1 + n n < v} = § P{z + Y e[0, ß ] ^ < i < k - 1;z + Y k ß } k=1

.P{ß + Yj e [ 0 , ß ] ;1 < i < n - k - 1 ; ß + Y

n - k

< v < 0}

= Z bk (z; ß)ck = 1 n-k ( ß ; v )

= Z

bk

(z; ß)c

n

-

k

(ß;v)

k = 0

= bn (z;ß)* Cn (ß; v), (3.18)

where b0( z ; ß ) = . C0( ß ; v ) = 0 . For a general formula, we have

P{vß(n) = 2;v1 = n;X,.1 + nn < v} = 2 P{Z + Yi e [ 0 ,ß ] ; l < i < k - 1;z + Yk > ß j 2 < k + 1 < n - 1 k,1>1

P{ß + Yi e[0,ß];1 < i < 1 - 1;ß + Y, > ß}

P{ß + Yi e[0,ß];1 < i < n - k - 1 - 1;ß + Y

n

- k < v < 0 }

2 bk ( z ; ß ) b 1 ( ß ; ß ) c n _k_ i ( ß ; v ) 2 < k +1< n - 1 k ,1 >1 2 b k ( z ; ß ) b 1 ( ß ; ß ) cn- k - i ( ß ; v ) 0 < k + 1 < n k , 1 >0 = bn (z; p ) * bn ( p ; p ) * cn ( p ; v ) , (3.19)

where b0( z ; p ) = b0( p ; p ) = c0( p ; v ) = 0 . Therefore it is possible to see

that

P| vp( n ) = 2 ; v 1 = n ;X„ - 1 + nn < v} =

b

n

(z; p)*(b

n

(p;

p ) ) m- 1

* c

n

(p;v)

(3.20)

for m > 2, n > 2, v < 0 .Substituting the expressions (3.17)-(3.20) in (3.16), we have

rn(z;

v) =

cn(z;

v) +

bn(z;

p)*

cn(p;

v) + £

bn(z;

p)*

(bn(p;

p)))

1

' *

cn(p;

v)

m=2

Furthermore, since

b 0 ( z ; p ) = b 0 ( p ; p ) = G0(p ;v) = 0,

Y b n ( z ; p ) * ( b „ ( P ; P ) ) )

1

* c „ ( P ; v )

m=2 =

»

1

Y b n ( z ; p ) * ( b „ ( P ; P ) ) ) * c „ ( P ; v )

m=2

=

b

„ ( z ; P ) * ( Y (

b

„ ( P ; P ) ) ) * c „ ( P ; v )

V k = 1 •>

= b„ (z; P)* U„ (P; P)* c„ (P;v).

Thus, we can write

r„ (z;v) = c„ (z;v) + b„ (z; p)* c„ (p;v) + b„ (z; p)* U„ (p; p)* c„ (p;v)

where

U „ ( P ; P ) = Y (

b

„ (P;p)). .

k=1

On the other hand, we get

r1

(z; v) =

Pz {V1

= 1;

X 0

+ n - v < 0}

= p{z + Y

1

- v}

=

c1

(

z

;

v

)

So we have given the probabilities r „ ( z ; v ) , n > 0, in terms of the probability characteristics of random walk {Y„}, n > 1. Thus the prof is completed.

Corollary 3.1 Under conditions of Theorem 3.1, if the random variable ^ 1 is exponentially distributed with parameter 0 > 0 , then the one-dimensional distribution function of the process X ( t ) can be given in the following explicit form:

Q(t; z; x) = e(x - z) e

- e t

+ Y q(n; z; x) e

- e t ,

where s ( x - z) ve q(n; z; x) is as given in Theorem 3.1.

Proof. If the random variable is exponentially distributed with parameter 0 > 0 , then the random variable Tn has the Gamma distribution with parameter (n, 0) . That is, the density function of the random variable Tn is

0n

f

T

(t) =

tn - 1

e

- 0 t

n > 1.

Tn

(n - 1 ) !

Therefore, by using this formula, it easy to see that

AO

n

(t) = M l e

- 0 t

n!

and

AO

o(t)

= O

o(t)

= 1 - O(t) = e

- 0 t.

Thus the proof is completed.

Corollary 3. 2 Under conditions of Theorem 3.1, if the random variable has the Erlang distribution with parameter 0 > 0 by order k ( k > 1) , then the one-dimensional distribution function of the process X ( t) can be given in the following explicit form:

n k + k - 1

e t

Q(t;z;x) = £ q(n;z;x) S M - e

n=0 m=nk n*

where q ( n ; z ; x ) is as given in Theorem 3.1.

Proof. If the random variable has the Erlang distribution with parameter 0 > 0 by order k ( k > 1) , then we get

n k + k - 1 m ^ n ( t ) = S ( e ^ e

i m'

m= nk 1 1 1 • e t and

m - e t

1 - O ( t ) =

Y

^

e

m=0 m !

Therefore we can write

k - 1 ( e t )m m n k+k- 1 ( e t )n

Q(t;z;x) =

E

( x - z )

(

" " L e

- e

• + Y q ( n ; z ; x ) Y

{

~J~e

m=0 m- n=1 m=nk n

-Furthermore by considering the equality q(0; z; x ) = s ( x - z) we have

e

n k + k - 1

- e t

Q(t;z;x) = £ q(n;z;x) £ M -

e

n=0 m=nk

Thus the proof is completed.

Corollary 3. 3 Under conditions of Theorem 3.1, if the random variable ^ has the Che-square distribution with freedom order 2k (k > 1) , then the one-dimensional distribution function of the process X ( t) can be given in the following explicit form:

o> n k + k - 1 j. m

Q(t;z;x) = £ q ( n ; z ; x ) £ ^ e

-

"

n=0 m=nk

Proof. Since the Che-square distribution with freedom order n is equal to the Gamma distribution with parameter ( n / 2, 1 / 2 ) , the random variable has the Erlang distribution with parameter 1/2 by order k. Substituting 1/2 instead of 0 in Corollary 3.2 the prof is completed.

REFERENCES

Borovkov, A.A., Stochastic Processes in The Theory of Queues, Nauka, Moscow 1972. 2. Feller, W., An Introduction to Probability Theory and Its Applications, Vol. I, Wiley, New York 1968.

Khaniev, T.A., 'Distribution of a semi-Markov Walk with two delay screens', Some question of the theory of stochastic processes, Collect sci. Works, Kiev, 106-113 (1984).

Khaniev, T.A., Ünver, I., 'The study of the level zero crossing time of a semi-Markovian random walk with delaying screen', Tr. J. of Mathematics 21, 257-268 (1997).

Khaniev, T. A., Özdemir, H., and Maden, S., 'Calculating the probability characteristics of a boundary functionals of a semi-continuous random process with reflecting and delaying screens', Appl. Stochastic Models Data Anal., 14, 117-123 (1998).

Maden, S., 'On the semi-markovian random walk process with reflecting and delaying barriers', Turkish. Journal of Mathematics 25, 263-274 (2001).

Khaniev, T. A., Unver, I. and Maden, S., 'On the semi-markovian random walk process with two reflecting barriers', Stochastic Analysis and Applications, 19(5), 799-819 (2001).

8. Nasirova, T.H., Processes of Semi-Markov Walk, ELM, Baku, 1984. 9. Prabhu, N.U., Stochastic Storage Processes, New York, Heidelberg: Springer-Verlag, 1981.

Spitzer, F., Principles of Random Walk, Van Nostrand, Princeton, 1969. Unver, I.,'On the boundary functional of the semi-markovian random walk with negative drift and positive jumps and two delaying screens', Jour. of Inst. of Math. & Comp. Sci. (Math. Ser.) Vol. 14, No. 2 (2001) 97 -105.