The iterative stein rule estimator of the disturbance variance in a linear regression model when the proxy variables are used

Selçuk J. Appl. Math. Selçuk Journal of Vol. 7. No.2. pp. 13-25, 2006 Applied Mathematics

The Iterative Stein Rule Estimator Of The Disturbance Variance In A Linear Regression Model When The Proxy Variables Are Used Deniz Ünal and Fikri Akdeniz

Department of Statistics, Faculty of Arts and Sciences,University of Cukurova, 01330 Adana, Turkey

e-mail: dunal@ cu.edu.tr, akdeniz@ m ail.cu.edu.tr

Summary.In case of the existence of the unobservable relevant regressors, a way of estimating the model is to use the proxy variables as regressors. Here we consider the estimator of the disturbance variance in a linear regression model with the proxy variables when the Stein-rule(SR) estimator of this model is used in place of the ordinary least square estimator (OLSE). Our aim is to examine the effect of the proxy variables on disturbance variance in a regression models with proxy variables. For this purpose we define the iterative SRP (SR in a linear regression model with the proxy variables) and iterative PSRP (Positive part SR in a linear regression model with the proxy variables) estimators of the disturbance variance in a regression model with proxy variable and analyze them by using MSE (Mean square error) criterion.

1. Introduction

A situation is examined here where the statistician is faced with having to make a correct interpretation of a theory which can not be observed to be included in the model to be estimated. For example, in a survey of elderly people some patients may be too sick to respond and the researcher may take approximate information from their care providers. In this situation, the problem is how to treat these proxy information. There have been some studies debating be-tween omitting the incomplete covariates or using the proxy variables instead. The statistician’s choice is supposed to lie between omitting the unobservable variable and using the resulting misspecified equation or of using a (possibly poor) proxy variable. It was shown (Wickens. M, 1972) that the bias of the estimates of the coefficients of the unobservable variable is always greater than the bias resulting from using even a poor proxy. That is to bound the effects of unobservable constructions we may use proxy variables. Krasker and Prat

(1986) studied about bounding the effects of proxy variables on regression coef-ficients. Many statisticians investigated the sampling properties of statistics in a regression model when the proxy variables are used. For example, Mc Callum (1972), Frost (1979), Ohtani (1981), Ohtani and Hasegawa (1993),Trenkler and Stahlecker (1996), and Namba and Ohtani (2006).

In this paper the iterative Stein-rule estimator for the disturbance variance in proxy model and MSE of this estimator is given, and examine its sampling properties. To make an interpretation about the MSE, its differentiation is taken into consideration. This article is organized as follows. The model and the estimators are reviewed in Section 2. Iterative estimator of the disturbance variance is discussed in section 3. Section 3 also illustrates the performance of the disturbance variances.

2. The Model and Estimators

Let us first consider the linear regression model

(1)  = 11+ 22+    ∼ (0 2)

where  :  × 1 observation vector of a dependent variable, 1 :  × 1 and 2 :  × 2 matrices of observations of non-stochastic independent variables, ₁: 1× 1 and 2: 2× 1 vectors of parameters and  :  × 1 vector of normal disturbance terms. The usual OLSE for the parameter vector  = [0₁ 0₂]0 _is

(2)  = −10

where  = (0_{) and SR estimator for the parameter vector  is}

(3) = [1 −

0_ 0_]

where  =  −  and  = 1+ 2  =  −  and  is a constant in 0 ≤  ≤ 2( − 2)( + 2). The SR estimator dominates the OLSE in terms of MSE criterion (James and Stein, 1961) and SR estimator is further dominated by Positive part SR estimator (Baranchik, 1970) defined as,

(4)  = [0 1 −

0 0_] The usual estimator of the disturbance variance is

(5) 2=( − )

0_{( − )}

Now let us assume the existence of 2∗ as the matrix of the proxy variables though 2 is unobservable. Now, let us consider the linear regression model with the proxy variable ∗

2 in place of 2:

(6)  = 11+ 2∗∗2+ ∗ ∗∼ (22− 2∗∗2 2) where ∗_{= 2}

2− 2∗∗2+ . Assume  = [1 2] and ∗ = [1 2∗] have full column rank and consider the estimators for ∗ = [0₁ ∗₂0]0 based on the model (6). The usual OLSE for the parameter vector is ∗= [0₁ ∗₂0]0 _is

(7) ∗= ∗−1∗0

where ∗_{= (}∗0_∗_{) and SRP estimator for the parameter vector }∗ _is

(8)  = [1 −

∗0

∗ ∗0

∗_∗]∗

where ∗_{=  − }∗_∗ _{and 0 ≤  ≤ 2( − 2)( + 2). For  ≥ 3, PSRP estimator} for the parameter vector ∗ is

(9)   = [0 1 −

∗0_∗ ∗0

∗_∗]∗

In the next section, we obtain the sampling properties and explicit formula for the iterative SRP and PSRP estimators of the disturbance variance.

3. Iterative Estimator of the Disturbance Variance

Now for the null hypothesis 0 : ∗ = 0 against the alternative : ∗ 6= 0, ∗₌ ∗0∗∗

∗0_∗_ is the test statistic. Now consider the following pre-test estimator

with the critical value  :

(10) ˆ∗( ) = (∗_{≥ )[1 −} ∗

0

∗ ∗0

∗_∗]∗

where () is an indicator function taking the values 1 if an event  occurs and 0 if not. Note that ∗( ) reduces to  when  = 0 and   when  = . Let the iterative estimator of the disturbance variance in the model with proxy variables be

(11) ˆ2_∗ = ( − ∗ˆ ∗

( ))0_{( − }∗_ˆ∗_{( ))}  − 

( − )ˆ2∗_ = _{( − }∗ˆ∗( ))0_{( − }∗ˆ∗( )) = 0_{ − 2}0∗ˆ∗( ) + ∗0( )∗0∗ˆ∗( ) by using ∗_{=  − }∗∗ we have, ( − )ˆ2_∗ = ∗0∗0∗∗+ ∗0∗_{− 2}0∗[(∗_{≥ )} [1 − ∗ 0 ∗ ∗0 ∗_∗]∗] + (∗ ≥  )[1 − ∗0_∗ ∗0 ∗_∗] 2_∗0 ∗0∗∗) +(∗ _{≥  )[1 −} ∗ 0 ∗ ∗0 ∗_∗] 2_∗0_∗0_∗_∗ = ∗0_∗_∗_{+ }∗0_∗_{− (}∗_{≥ )}∗0_∗_∗ (12) +2(∗_{≥ )}(∗ 0 ∗₎2 ∗0 ∗_∗

Using (A1),(A2), (A3), (A4) in Appendix, we have,

(ˆ2_∗) = 1  − [ 2_{+ 2}0_0_∗_∗−1_∗0  (13) + 2 + 200∗_{ − }2(1 0;  ) + 22_{(−1 2;  )]} and by (12), bias of ˆ2_∗ is (14) = _−1 [2_{+ 2}0_0_∗_∗−1_∗0_{ + }2_ +200_∗_{ − }2_{(1 0;  ) + }2_2_{(−1 2; )} −[∗0 ∗_∗_{+ }∗0 ∗_{− (}∗_{≥ )}∗0 ∗_∗ +2(∗_{≥ )}(_∗0∗0_∗∗_)∗2]]

Let us find the MSE of ˆ2∗

 (ˆ2_∗) = (ˆ2_∗ _{− }2)2 (15) =_(−)1 2[(∗ 0 ∗_∗₎2_{+ (}∗0_∗₎2_{+ (}∗_{≥ )(}∗0_∗_∗₎2 +4(∗_{≥ )(}∗0∗)4(∗0∗∗)2+ 2∗0∗∗∗0∗ −2(∗_{≥ )(}∗0_∗_∗₎2_{+ 2}2_(∗_{≥ )(}∗0_∗₎2 −2(∗≥ )∗0∗∗0∗∗+ 22(∗_{≥  )(}∗0∗)3(∗0∗∗) −2(∗_{≥ )(}∗0_∗₎2_{] −} 22 − ×[2+ 200∗∗−1∗0 + 2 + 20∗ −2(1 0;  ) + 22_{(−1 2; )] + }4

By equations (A6), (A7),(A12), (A13), (A14), (A15), (A16),(A17) in Appendix, MSE of the ˆ2_∗ becomes,

 (ˆ2_∗) =_−1 _{−(∗1+∗2)_4 ∞ P =0 ∞ P =0 ∗1∗2 !! ( + 2 + 2)( + 2) (16) + −(∗1+∗2)4 ∞ P =0 ∞ P =0 ∗1∗2 !! ( + 2 + 2)( + 2) + −(∗1+∗2)2_4 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2 + 2)( + 2) × [1 −Γ(2+2+)Γ(2+)Γ((+)2+++2) Z ∗₌   + 0 2++1_{(1 − )}2+−1 +4_−(∗1+∗2)24 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+ ×(+2+6)(+2+4)(+2+2)(+2)(+2−2)(+2−4) [1 −Γ(2+Γ((+)2+++2)−2)Γ(2++4) Z ∗₌   + 0 2+−3_{(1 − )}2++3 +2−(∗1+∗2)_4 ∞ P =0 ∞ P =0 ∗1∗2 !! ( + 2)( + 2) −2−(∗1+∗2)2_4 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2 + 2)( + 2) ×[1 −_{Γ(2 + 2 + )Γ(2 + )}Γ(( + )2 +  +  + 2) Z ∗₌   + 0 2++1_{(1 − )}2+−1 +22−(∗1+∗2)2_4 ∞ P =0 ∞ P =0 ∗₁ ∗₂ !!2+( + 2 + 2)( + 2) ×[1 − Γ(( + )2 +  +  + 2) Γ(2 + )Γ(2 +  + 2) Z ∗₌   + 0 2+−1_{(1 − )}2++1_ −2−(∗1+∗2)2_4 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2)( + 2) ×[1 −Γ(2++1)Γ(2++1)Γ((+)2+++2) Z ∗₌   + 0 2+_{(1 − )}2+ +22−(∗1+∗2)24 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+ ×(+2+4)(+2+2)(+2)(+2−2)

×[1 − Γ(2+Γ((+)2+++2)−1)Γ(2++3) Z ∗₌   + 0 2+−2_{(1 − )}2++2 −2−(∗1+∗2)2_4 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2 + 2)( + 2) ×[1 − Γ(2+)Γ(2++2)Γ((+)2+++2) Z ∗=   + 0 2+−1_{(1 − )}2++1 − 2 2  − [ 2_{+ 2}∗_∗−1_∗0  + 2 +2∗_{ − }2(1 0;  ) + 22_{(−1 2; )] + }4_} where ∗ _{=  − }∗_∗−1_∗0_{. Since } 1 =  ∗0_∗_∗ 2 and 2 =  ∗0_∗ 2 are

inthe quadrartic formS notice that 1 and 2 have the independent noncentral chi-square distributions that is 1 ∼ 0

2 (∗1) and 2 ∼ 0 2 (∗2) where ∗1 = ∗∗−1∗0 2 and ∗₂= ∗_

2 . By some eliminations we have

 (ˆ2_∗) =_(−)4 2{(2₁) + (₂2) − (2 0  ) (17) + 4_{(−2 4  ) + 2(}12) + 22(2 0  ) − (1 1 ) + 22_{(−1 3  ) − 2(0 2 )} −} 2 2  − [ 2 + 2∗∗−1∗0 + 2∗_{ − }2(1 0;  ) + 22_{(−1 2;  )] + }4

Taking different values for  and  in the following equation (Ünal,2006)

((1)  (2) ) =   − ∗ 1_−∗2₂− (18) ∞ P =0 ∞ P =0 ∗₁∗₂ Γ(₂ +  + )Γ(_{2+  − )} !!Γ( 2 + )Γ(  2+ ) we have,

 (ˆ2_∗) =  4 ( − )2{  −( ∗ 1+∗2)_4 ∞ P =0 ∞ P =0 ∗₁∗₂ !! (19) × [( + 2 + 2)( + 2) + ( + 2 + 2)( + 2) + 2( + 2)( + 2)]} +  4 ( − )2{−( ∗ 1+∗2)2_4 ×P∞ =0 ∞ P =0 ∗₁∗₂ !! [( + 2 + 2)( + 2) ×[1 − ∗(2 + 2 +  2 + )] + 4 ×( + 2 + 6)( + 2 + 4)( + 2 + 2)( + 2) ( + 2 − 2)( + 2 − 4) ×[1 − ∗(2 +  − 2 2 +  + 4)] − 2( + 2 + 2)( + 2) ×[1 − ∗(2 + 2 +  2 + )] − 22( + 2 + 2)( + 2) ×[1 − ∗(2 +  2 +  + 2)] − 2( + 2)( + 2) ×[1 − ∗(2 +  + 1 2 +  + 1)] +22( + 2 + 4)( + 2 + 2)( + 2) ( + 2 − 2) ×[1 − ∗(2 +  − 1 2 +  + 3)] − 2( + 2 + 2)( + 2) ×[1 − ∗(2 +  2 +  + 2)]]} − 2  4  − ( +  + 2 2 −(1 0; ) + 2(−1 2; )) + 4

Taking the partial derivative of  (ˆ2_∗_{) wrt.  and for  ≥ 6 using} the Fundamental theorem of Calculus we have,

 (ˆ2_∗)  = 4 ( − )2{−( ∗ 1+∗2)2 (20) ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+ 1 ( + )(+)2++ × ( + 2 + 2)( + 2)2++12++12+−1 − 4( + 2 + 6)( + 2 + 4)( + 2 + 2)( + 2) ( + 2 − 2)( + 2 − 4)

2+−32+−32++3 +2(1 − 2)( + 2 + 2)( + 2)2+−12+−12++1 +2( + 2)( + 2)2+2+2+_{− 2}2 ×( + 2 + 4)( + 2 + 2)( + 2) ( + 2 − 2) ×2+−22+−22++2_{]} + 2}  4 ( − ){2 −(∗1+∗2)2 ×P∞ =0 ∞ P =0 ∗₁ ∗₂ !!2+_{( + )}(+)2++ 2+−12+−12+ Γ(2 + )Γ(2 + ) ×Γ(( + )2 +  +  + 1) + 2 ×Γ(( + )2 +  +  + 2) 2 +  − 1 ]} = 4_−(∗ 1+∗2)2  −  ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+_{( + )}(+)2++ (21) × { 1  − [( + 2 + 2)( + 2) ×  2++1_2++1_2+−1 − 4( + 2 + 6)( + 2 + 4)( + 2 + 2)( + 2) ( + 2 − 2)( + 2 − 4) × 2+−32+−32++3_{+ 2(1 − }2)( + 2 + 2)( + 2) × 2+−12+−12++1+ 2( + 2)( + 2)2+2+2+ − 22( + 2 + 4)( + 2 + 2)( + 2) ( + 2 − 2) × 2+−22+−22++2] + 4 2+−1_2+−1_2+ Γ(2 + )Γ(2 + ) Γ(( + )2 +  +  + 1) + 2Γ(( + )2 +  +  + 2) 2 +  − 1 ]]}

Then the conclusion will be as the following. 4.Conclusion

As mentioned before ∗( ) reduces to  when  = 0 and   when  =  . Since infinite series in (   ) and (   ) converge absolutely (Namba and Ohtani, 2006). Then for  ≥ 6 and for the negative values of equa-tion (21), the iterative SRP estimator of the disturbance variance with PSRP estimator dominates the iterative SRP estimator of the disturbance variance

with usual SRP estimator in terms of MSE criterion when the proxy variables are used in place of the unobservable variables.

5. Appendix:

By the definition of the expectation of the non-central chi-squared distributions,

(∗0∗∗) = 2(1) = 2[ + 2∗1] (A1) = 2+ 2∗∗−1∗0 (A2) (∗0∗) = 2(2) = 2[ + 2∗2] = 2 + 2∗ (A3) ((∗_{≥ )(}∗0∗∗) = [1 2 ≥ )1] 2_{= }2_{(1 0;  )} (A4) ((∗_{≥ )}(∗ 0 ∗₎2 ∗0 ∗_∗ = [ 1 2 ≥ ) −1 1 22]2= 2(−1 2;  ) We have by (5) (A5) (  1  2 ) = −∗1−∗22− ∞ P =0 ∞ P =0 ∗1 ∗2 Γ(2+  + )Γ(  2+  − ) !!Γ(₂+ )Γ(₂ + )

Here for  = 2 and  = 0 we find

(A6) (∗0∗∗)2= −(∗1+∗2)_4 ∞ P =0 ∞ P =0 ∗₁∗₂ !! ( + 2 + 2)( + 2) for  = 0 ,  = −2 (A7) ((∗0∗)2) = −(∗1+∗2)_4 ∞ P =0 ∞ P =0 ∗₁ ∗₂ !! ( + 2 + 2)( + 2)

We know the following equations by Ohtani and Namba (2006). For the incom-plete beta function

(A8) (1 2) = [(1 2)]−1 Z  0 1−1_{(1 − )}2−1_ and for ( ;  ) = Γ(2++)Γ(2++)_{Γ(2+)Γ(2+)} (A9) × [1 − ∗(2 +  +  2 +  + )] we have, ( ;  ) = [1 2 ≥ )  1  2] (A10) 2+P∞ =0 ∞ P =0 (∗1)(∗2)( ;  ) ( ;  ) = [1 2 ≥  )  1  2 ∗_∗ 2 ] (A11) ∗₁2+P∞ =0 ∞ P =0 (∗1)(∗2)( ;  ) where ∗₌   + and () = −2_(2)

!  The infinite series in (19), (20) and (21) are absolutely convergent (Namba and Ohtani,2006). Then,

((∗_{≥  )(}∗0∗∗)2) = 4((∗_{≥ )1}2) (A12) = 4(2 0;  ) = 422P∞ =0 ∞ P =0 (∗1)(∗2)(2 0;  ) = 44P∞ =0 ∞ P =0 −∗1 2−∗2 2∗ 1∗2 !!2+ Γ( 2++1) Γ( 2+) [1 − ∗(2 + 2 +  2 + )] = 4P∞ =0 ∞ P =0 −(∗1+∗2)2∗ 1∗2 !!2+ ( + 2 + 2)( + 2) [1 − ∗(2 + 2 +  2 + )] = 4P∞ =0 ∞ P =0 −(∗1 +∗2 )2∗1 ∗2 !!2+ ( + 2 + 2)( + 2) [1 − [(2 + 2 +  2 + )]−1× Z ∗ 0 2+1+_{(1 − )}2+−1 = 4−(∗1+∗2)2 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2 + 2)( + 2) [1 −Γ(2+2+)Γ(2+)Γ(2+2+++2) × Z ∗ 0 2++1_{(1 − )}2+−1] [(∗_{≥ )} (∗0∗)4 (∗0_∗_∗₎2] =  4_[(∗_{≥ )}−2 1 42)] (A13) = 4_{(−2 4;  )} = 4− ∗_{1 +}∗₂ 2 ∞ P =0 ∞ P =0 ∗ 1∗2 !!2+ (+2+6)(+2+4)(+2+2)(+2) (+2−2)(+2−4) × [1 − Γ( + 2 +++2) Γ( 2+−2)Γ(  2++4) Z ∗ 0 2+−3_{(1 − )}2++3] (∗0∗∗∗0∗) = 4( 1 ₂−1) (A14) = 4−(∗1+∗2) ∞ P =0 ∞ P =0 ∗1∗2 !!2+( + 2)( + 2)

[(∗_{≥ )(}∗0∗)2] = 4(0 2;  ) (A15) = 4− ∗_{1 +}∗₂ 2 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2 + 2)( + 2) × [1 − Γ(+2 +++2) Γ(2+2)Γ(2++2) Z ∗ 0 2+−1_{(1 − )}2++1] [(∗_{≥  )}∗0∗(∗0∗∗)] = 4(1 1;  ) (A16) = 4− ∗_{1 +}∗₂ 2 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+( + 2)( + 2) × [1 − Γ( + 2 +++2) Γ(2++1)Γ(2++1) Z ∗ 0 2+_{(1 − )}2+] [(∗_{≥  )} (∗0∗)3 (∗0_∗_∗₎] =  4 (−1 3;  ) (A17) = 4−∗1 + ∗ 2 2 ∞ P =0 ∞ P =0 ∗₁∗₂ !!2+ (+2+4)(+2+2)(+2) (+2−2) × [1 − Γ( + 2 +++2) Γ(2++1)Γ(2++1) Z ∗ 0 2+_{(1 − )}2+] References

1. A. Baranchik, A. J., 1970. A family of Minimax Estimators of the Mean of a Multivariate Normal Distribution. Ann. Math. Statist. 41, 642-645.

2. Frost, P.A., 1979. Proxy variables and Specification bias ,Rev. Econom. Statist., 61, 323-325.

3. James, W. and Stein, C., 1961, Estimation with Quadratic Loss Proceedings, Fourth Berkeley Symposium in Mathematical Statistics and Probability, 361-379.

4. Krasker, W.S., Prat, J.W, 1986. Bounding the Effects of Proxy Variables on Regression Coefficients , Econometrica, 54 (3), 641-655.

5. McCallum,B.T, 1972. Relative Asymptotic Bias from Errors of omission and mea-surement, Econometrica ,40 ,757-758.

6. Namba, A., Ohtani, K., 2006. PMSE Performance of the Stein-rule and positive part Stein-rule Estimators in a Linear Regression Model with or without Proxy Variables. Statistics and Probability Letters.

7. Ohtani, K., 1981. On the use of a Proxy Variable in Prediction: an MSE compari-son. , Rev. Econom. Statist., 63, 627-629.

8. Ohtani, K., Hasegawa, H.,1993. On Small Sample Properties of in a Linear Re-gression Model with Multivariate errors and Proxy variables. , Econometric Theory, 9, 504-515.

9. Rencher, A. C.2000. Linear Models in Statistics, Wiley Series in Probability and Statistics, 578.

10. Trenkler, G., Stahlecker, P., 1996. Dropping Variables versus Use of Proxy Vari-ables in Linear Regression. , J. Statist. Plann. Inference, 50, 65-75.

11. Unal, D., 2006. The Doubly Non-central F Distribution in a Regression Model with Proxy Variables, Submitted.

12. Wickens, M. R., 1972. A Note on the Use of Proxy Variables , Econometrica, 40(4), 759-761.