Diffusion approximation of nonhomogeneous switching processes and its application to rate of convergence analysis of computational procedures

Cybernetics and Systems Analysis, Vol. 34, No. 2, 1998

D I F F U S I O N A P P R O X I M A T I O N O F N O N H O M O G E N E O U S S W I T C H I N G P R O C E S S E S A N D I T S A P P L I C A T I O N T O R A T E O F C O N V E R G E N C E A N A L Y S I S O F C O M P U T A T I O N A L P R O C E D U R E S

V. V. Anisimov and A. V. Naidenova UDC 519.21

The development of a broad class of stochastic systems can be described in terms of stochastic processes whose behavior spontaneously changes (switches) at certain time moments. These switching points are random functionals of the previous path. Such processes arise in the theory of queueing systems and networks, in branching and migration phenomena, in the analysis of stochastic dynamical systems with random errors, and in other applications.

The description of these models relies on a special class of discrete-event stochastic processes introduced in [1-3], where they are called switching processes.

A switching process is a two-component process (x(t), ~(t)), t > 0, with values in the space (X, R r) for which there is a sequence of time moments t t < t 2 < ... such that x(t) = x(tk) on each time interval It k, t k+ l) and the behavior of the process ~t) on this time interval depends only on the values (x(tk), ~(tk)).

The moments t k are called switching points; x(t) is the discrete switching component. Such processes are describable in terms of constructive characteristics [ 1-3]. They are useful for the analysis of the asymptotic behavior of stochastic systems with "fast" and "rare" switching events [2-7].

Switching processes are a natural generalization of classes of Markov processes homogeneous in the second component [8], processes with independent increments and semi-Markov switchings [2, 3, 9], Markov aggregates [8], Markov processes with semi-Markovian random interventions [ 10], Markovian and semi-Markovian evolutions [ 11-14].

Limit theorems on the convergence of one switching process to another (in the class of switching processes) have been proved in [2, 3]. These results have led to a theory of asymptotic state aggregation of nonhomogeneous Markov and semi-Markov processes [3].

In this article we consider limit theorems on the convergence of the path of a switching process to the solution of some ordinary differential equation (the averaging principle) and the convergence of the normalized deviation to some diffusion process (the diffusion approximation) for an important subclass of switching processes - the so-called semi-Markov recurrent processes (SMRP) with additional dependence on the current switching point.

The approach to the study of recursive algorithms with random response time is based on representation of the original process as a superposition of a recurrent embedded process and an accumulated-time counter process followed by application of limit theorems for discrete-time recursive stochastic algorithms and superpositions of random functions.

We first prove some generalizations of theorems that constitute the averaging principle and the diffusion approximation for SMRP [6].

Given are independent families of random vectors {(~k(s, t), ~-k(s, t), s E R m, t >__ 0}, k >_ 0, with values in R m • [0, ~ ) whose characteristic functions are Bern-measurable (Bern is the Borel a-algebra in Rm), and the initial value s 0.

Define the sequences

~

t o = O, S k + t, h = S,. h + h . ~ ( S , , h' t~. h)' t~ + ~. h = t,. h + h'r~,(Sk, h' t~ h)' ~ ;' O.

Consider the process Sh(t) of the form Sh(O) = s o, Sh(t) = Sk, h for tk, h <_ t < tk+ l,h, t >__ O.

Translated from Kibernetika i Sistemnyi Analiz No. 2, pp. 97-104, March-April, 1998. Original article submitted June 3, 1997.

Note that by construction the sequence Sk, h, k >_ 0, is a Markov chain in R m, and Sh(t) is an SMRP [6, 9].

Let us investigate the convergence of the path of Sh(t) to the solution of a differential equation whose coefficients are determined by the parameters of the scheme (the averaging principle) and the convergence of the normalized deviation to the diffusion process (diffusion approximation). For simplicity, we consider step-homogeneous processes (the distribution of the variables ~k('), rk(') is independent of the index k). Assume that for any h > 0, s E R m, t > 0,

Let 2 (1) g(r~(s, t) + I~k(s, t)t 2) < C. Erk(s, t) = m (s, t), E~k(s, t) = b (s, t), vat ~(~, 0 = o ~(~, t), Va," ~j,(s, 0 = o ~(s, t).

(2)

The symbol [zl is the vector (or matrix) norm if z is a vector (or a matrix). The mean, the variance, and the integral are evaluated componentwise. Here and in what follows all vectors are column vectors.

T H E O R E M 1. Given is notation (1), (2). Moreover, m(s, t) > 0, for all s, t and for some fixed C

m (s, t) + Ib(s, t)l ~ C (1 + I ( s , t ) l ) , for every L > 0 there exists C L such that for I(sl, t l)l v I(s2, t2)l < L

Im (sl, tl) -- m (s2, t2)l + Ib(sl, tl) - b(s2, t2)l ~ C L ( I S I - S21 + I t l - t21).

(3)

Then sup IS h(0 - - s(t) l _.{e 0, where s(0 satisfies the equation 0,:t.cT

ds (t) = m (s (O, t) -1 b (s (t), t)dt, s (0) = s O, the symbol P stands for convergence in probability, and T > 0 is any number such that

y ( + 0o) > T, (4)

where y(t) = 'f m(rl(u),u)du, and r/(u) satisfies the relationship drl(u) = b(r/(u), u)du, 77(0) = s O. o

Proof. Denote the pair (s, t) by the variable a. Let n = [1 / h], f ~ = ntk, h, Snk = n'Sk, h, Sn(t) = n.Sh(t). In the notation of [2], let V n = 1/h, A n = I. In this case we arrive at a scheme described in [6] for the vector sequence (Snk, t ~ ) , k >_ 0. Q.E.D.

R e m a r k . If the functions m(.) and b(.) depend on the parameter h, i.e., can be written respectively as mh(') and bh(. ), we must additionally stipulate in the assumptions of the theorem that there exist continuous functions m(-) and b(-) such that mh(') --- m(-), bh(" ) ---) b(') uniformly in (s, t) in every bounded region and that condition (3) holds.

L e t u s now investigate the behavior of the centered process ~h(t) -- (1/v/-h)(Sh -- S(t)), t >_ O. Retaining the previous notation, we take

D

b'(s, t) = b (s, t) m (s, t) - t ,

2($, l) = /E(~ 1 ($, t) -- b (s, t) - b(s, t) (r I (s, l) - rn (s, 0)) 2.

T H E O R E M 2. Assume that the conditions of Theorem 1 are satisfied, continuous vector- and matrix-valued partial derivatives (0/)(s, t)/Ot) and (0/~(s, t)/Os) = K(s, t) exist, the function D(s, t) is continuous in s, t, the squares l~k(s, 012 and rk(s, t) 7- are uniformly integrable over the pair (s, t) in any bounded region.

Then the sequence of measures induced by the sequence of processes ~h(t) weakly converges in the Skorokhod space D T for any T > 0 that satisfies conditions (4) to the measure induced by the process ~'(t) that satisfies the differential equation

ar~(t) = K (s (t), t) C(I)dt + D (s (t), t) m (s (t), t) C(O) = O,

- I /2dw(t),

where w(t) is the standard Wiener process in R m.

The proof follows directly from Theorem 2 [6] if its assumptions are rewritten in vector form.

Let us apply these results to investigate the convergence of computational algorithms for the real-time R u n g e - K u t t a scheme.

Let a Cauchy problem

y '(t) = 1" (t, y (t)), y (t o) = Yo"

(5)

be given on the interval [t o, T]. Assume that instead of the function fit, y) we compute in real time the function r(t, y) = fit, y) + e(t, y), and the computation takes r(t, y) time units. Here e(t, y) is a random error that depends only on the observation point (t, y), and r(t, y) also may be a random variable,

Ee(t, y) = O, V a t e(t, y) = (~2(t, y), (6)

Er(t, y) = m (t, y), Vat" r(t, y) = az(t, y)).

We assume that the errors and the computation times at different observation points are independent.

To construct a computational procedure that solves this problem, we introduce the set of jointly independent (for different k) nonnegative random variables {rk(t, y), t >_ O, y (E. R'n}, k >_ O, identically distributed with r(t, y).

We construct a recursive computational procedure of the form

Yk + l, h = Yk, h + h ' r (tk, h, Yk. h) " m (tk. h, Yk, h), YO, h = Yo , (7) where Yk, h is an approximate value of the ftmction y(t) (y(t) is the exact solution of the Cauchy problem (5)) at the point tk, h, and the moments tk, h are deemed by the formula

A

t k . ~, !, = t~ ~, + h . r ~ ( t z h' Yk. h)' to. h =

to"

Finally, let {ek(t, y), t >_ 0, y EE Rrn}, k >_ 0 be a set of random variables that are independent for different k and are identically distributed with e(t, y).

Denote 3~h(t) = Yt, h for t E [tk, h, tk+ 1,h).

T H E O R E M 3. Assume that the following conditions hold: 1) E(z(t, y)2 + e(t, y)2) < C;

2) Varr(t, y) = (r2(t, y) < C;

3) for every L > 0 there exists CL such that for lYll v lY2I < L lm(t,y 1) -- m(t, Y2) [ + lf(t, Yl) - - f i t , Y2) [ <_ C L

lYx

Then sup 13~h(t)- Y(t)l -" 0 for T that satisfy condition (4). t o , : t , : T

Proof. We introduce some notation:

~'~ h(t, y) = r (t, y) m (t, y),

~ = tl~ Iz' Zh(U) = Yk.. h' kh <, u < (k + l)h, vh(t ) = rain {k" tk. + l, h > t}, lth(t ) = inf {u " u > 0, oll(u ) > t}.

Then by definition vh(n~,h(t)) < t < vh(h(~,h(t) + 1)) and ~h(t) = h(~,h(t) + I). Noting that ~h(t) = yz, h(t), h, we write

yh(t) = zl,(hvit(t)) = zh(,Uh(t ) - h).

Thus, the process ~h(t) is constructed as a superposition of the processes Zh(t) and ~h(t).

Investigating further the processes Zh(U), vh(u), we obtain from Theorem 1 uniform (in probability) convergence to the corresponding processes for any T

P P

sup Iz/,(u) - z ( u ) i --0, suo loj,(t) - o(t)l --,0,

u',~ T u <_ T

where the processes z(u) and v(t) solve the system of equations t

az (u) = f-(o(u), z (u))du, u(t) = f m (o(u), z (u))du. o

(8)

Let us now return to the process #h(t). Using the results on the level crossing time (see [3]), we obtain that uniformly in probability t~h(t) - - h --, v - l ( t ) as h --- 0, where v - l ( t ) is the inverse of the function v(t) defined by Eq. (8). Thus, by the theorem of convergence of superpositions of random functions (see [3]), we have uniformly in probability

A 0

Y h ( l ) "" Z h ( h * ' h ( t ) ) ~ z ( 0 l ( t ) ) .

Denote y = z(v-l(t)). Taking the derivative of this function, we obtain that dy (t) = fit, ~ (t))dt. Thus, the function (t) solves Eq. (5). Q.E.D.

Let us now investigate the behavior of the centered process ~h(t) = ~ (Yk, h - - y(t)). Denote D2(t, y) = mZ(t, y) + f2(t, y)'o2(t, y).

T H E O R E M 4. Assume that the conditions of Theorem 3 are satisfied and moreover the following conditions hold: 1) a continuous derivative g(t, y) = f'y(t, y) exists"

2) the function D2(t, y) is continuous in t, y;

3) the squares I~l,h(t, y)[ 2 and 7"l,h(t, y)2 are uniformly integrable.

Then the sequence of measures induced by the sequence of processes ~h(t) weakly converges in the Skorokhod space D T for any T satisfying condition (4) to the measure induced by the process s"(t) that satisfies the stochastic differential equation

a~(t) = g (t, y (t)) ~(t)at + D (t, y (t)) m (t, y (t)) -~ / 2 a w (t), ~(0) = 0,

where w(t) is the standard Wiener process and y(t) satisfies differential equation (5). The proof of the theorem fully relies on Theorem 2.

Thus, the rate of convergence of the R u n g e - K u t t a scheme allowing for the time to compute the next iteration (in general, this time may be a random variable) suggests that we need to design an algorithm that instead of the function fit, y) entering, the differential equation evaluates the function f(t, y) = m(t, y).flt, y), where m(t, y) = ET"k,h(t, y). The rate of convergence of this algorithm is V~ , contrary to the classical methods. Otherwise, the algorithm converges to the solution of the Cauchy problem (5) at a biased point.

Retaining the previous assumptions about the computational error 8h(t, y), we now assume that the errors at different points are mutually independent and

Eeh(t, y) = h aa (t, y), V a t th(t, y) = h bb 2(t, y), (9)

a2(t,y) + b 2 ( / , y ) g O ( 1 + y 2 ) , 0 g t~< T. (lo)

We also assume that there exists a function ~k(t, y) such that

y (t + v) - y (t) = o / ( t . y) + o z , ( t , y (t)) + 0 (o z § ~)). (11)

where ~ > 0 and for every L > 0

- - ~)

lim sup v 2 +'sO (0 2 + < C L < ~,. (12)

o - " 0 [y[ < L t < L

In our case, ~b(t, y(t)) = (1/2)y"(t).

Under these assumptions we further assume that the time to compute each successive value )~,h may depend on the current path value 7"k,h(tk, h, Yk, h) = hT"k(tk, h, Yk, h)"

We again consider a recursive procedure of the form

Yk + 1, h = Yk, h + h" rh(lk, h' Yk., h ) m

(t~ h' YL h))' Y0, h

where rh(t, y) = fit, y) + eh(t, y) is the value of the function at the point (t, y) computed with an error, and the points tk, h

,.

are defined by the recurrence

A

tk. + l. h = t/c. h + rL h(tL h, Yk. iz), tO. h = t O.

We moreover assume that eh(t, y) are independent of ~'k(t, y).

We again introduce the process )~(t) = Yk, h for t ~. [tk, h, tk+ 1,h), k >_ 0. Denote

eh(t, 3') - Eeh(t, Y)

f~,(t, y) = q Vat ~l,(t, y)-'

Fh(z, t, y) = P { ~h(t, y) < z}, a (z, t, y) = P {r I (t, y) < z}.

T H E O R E M 5. Assume that conditions (6), (9), (10) are satisfied and moreover 1 ) f 2 ( x , y ) _< C(1 + y 2 ) , 0 _< t < T, - o o < y < oo;

2) the derivative fy'(t, y) exists and is continuous in the pair of variables t, y;

3) Ezl(t, y)2+a < ~ in any region bounded in both variables, where 6 > 0 satisfies condition (12);

4) for every L > 0 there exists C t such that for [Yz I, lY21 < L ]m(t, Yl) -- m(t, Y2)I + Io(t, Yt) - - o(t, Y2) -< CLIYl --Y2I"

5) for every L > 0

lim lim sup

f

lim sup f z 2dG (z, t, y) = O.

R--~, l yl <t-.t<L I z I >R

Then for procedure (13) with 3' = min { 1/2, a, (b + 1)/2} the sequence of measures induced by the sequence of processes ~lh(t) = h-V(yh(t) -- y(t)) weakly converges in the space D r to the measure induced by the process ~(t) that satisfies the stochastic differential equation

d ~" (t) = f 'y(t, y (t)) ~" (t)dt + 61 a (t, y (t))dt +

[

+ _{m (t. y 0'))} "J dw(t)

where w(t) is the standard Wiener process and

b+l 1

6 i = l i m h a - r 6 2 = l i m h ~ - r t 63 = l i m h ~ ' - r -

h-,,0 h--*0 h -,. 0

R e m a r k 1. If a2(t, y) - 0, the assertion of the theorem is slightly modified" the sequence of processes ~h(t) = h-'r(~h(t) -- y(t)), where 3" = min{ 1, a, (b + I)/2} weakly converges to the process O(t) that satisfies the stochastic differential equation

d ~'(t) = /'y(t, y (t))ff (t)dt + 61a (t, y (t))dt - 64m (t, y (t)).~, (t, y (t))dt + + 63b (t, y (t)) x/m (t, y (t)) dw (t), ~ (0) = O,

where if(t, y(t)) = 112 y"(t), 6 I, 63 are as specified in Theorem 5 and 6 4 = lira h 1 -"r.

h-O

R e m a r k 2. If Var r](t, y) = hC.a2(t, y) ;~ 0, c > 0, then the limit process ~(t) satisfies the equation

t

d ~(t) = / y(t, y(t))~(t)dt + 61 a (t, y(t))dt - 64m (t, y(t))~o(t, y(t))dt + + I 6 2 r n 2 ( t , y ( t ) ) b 2 ( t , y ( t ) ) + 6 ~ a 2 ( t , y ( t ) ) f 2 ( t , Y ( t ) ) l 1/2

' ", (t, y ( 0 ) ' d,,, (t),

i f ( o ) = o,

where 65 = lim h((c)+l/2)-'r, and 3' = min{1, a, (b + 1)/2, (c + 1)/2}. h-o

The proof is similar to the proof of Theorem 4.

We thus see that in the nonclassical R u n g e - K u t t a scheme allowing for the response times and random errors with mean of order h a and variance of order h b, the error does not affect the rate of convergence v/h -" if a and (b + 1)/2 are greater than 1/2. Otherwise, the rate of convergence of the scheme equals 3" = min{ 1/2, a, (b + 1)/2}. If the variance of the response time is identically zero, i.e., the response time is not a random variable in practice, the order of the scheme equals 3' = min{ 1/2, a, (b + 1)/2}. Specifically, if a and (b + 1)/2 are greater than I, then the random errors do not affect the order of the scheme, which takes the classical value 1.

Note that the convergence of computational procedures in the presence of random errors but without allowing for the response time has been considered in [1]. Stochastic optimization procedures allowing for the response time have been considered in [15].

The case when the variance of the response time is of order h c has been studied separately. In this case, 3' = min{1/2, a, (b + 1)/2, (c + 1)/2}.

R E F E R E N C E S

.

.

~

V. V. Anisimov, "Switching processes," Kibernetika, 13, No. 4, 590-595 (1977).

V. V. Anisimov, "Limit theorems for switching processes and their applications," Kibernetika, 14, No. 6, 917-929 (1978).

V. V. Anisimov, Stochastic Processes with a Discrete Component. Limit Theorems [in Russian], Izd. Kiev. Univ., Kiev (1988).

V. V. Anisimov, "Switching processes: averaging principle, diffusion approximation and applications," Acta Appl. Math., 40, 95-141 (1995).

V. V. Anisimov, "Asymptotic analysis of switching queueing systems in conditions of low and heavy loading," in: Matrix-Analytic Methods in Stochastic Models, Lect. Notes Pure Appl. Math., 183, 241-260 (1996).

V. V. Anisimov and A. O. Aliev, "Limit theorems for semi-Markov recurrent processes," Teor. Veroyatn. Mat. Stat., 31, 9-15 (1990).

. 10. 11. 12. 13. 14. 15.

V. V. Anisimov and E. A. Lebedev, Stochastic Queueing Networks. Markovian Models [in Russian], Lybid', Kiev (1992).

N. P. Buslenko, V. V. Kalashnikov, and I. N. Kovalenko, Lectures in Complex System Theory [in Russian], Sovetskoe Radio, Moscow (1973).

V. V. Anisimov, "Limit theorems for processes with semi-Markov switchings and their applications," Random Oper. Stoch. Eq., 2, No. 4, 333-352 (1994).

I. I. Gikhman and A. V. Skorokhod, Theory of Stochastic Processes [in Russian], Nauka, Moscow (1973). R. Hersh, "Random evolutions: survey of results and problems, ~ Rocky Mount. J. Math., 4, 443-475 (1974). R. Kertz, "Random evolutions with underlying semi-Markov pt:ocesses," Publ. Res. Inst. Math. Sci., 14, 589-614 (1978).

V. S. Korolyuk and A. V. Swishchuk, Random Evolutions, Kluwer, Boston (1994). M. Pinsky, "Random evolutions," Lect. Notes Math., 451, 89-100 (1975).

V. V. Anisimov and Z. P. Anisimova, "Convergence of recursive procedures with random response time," Kibernetika, No. 1, 82-84 (1985).