CMARS GMM estimation for semi-parametric models by conic optimization

(1)

GMM Estimation for Semi-parametric

Models by Conic Optimization:

Special Application to Finance

Erdem Kilic, Fatma Yerlikaya-Özkurt,

Gerhard Wilhelm Weber

(2)

Introduction

Importance of GMM: GMM (Generalized Method of Moments) is a robust estimator

in that, unlike maximum likelihood estimation, it does not require information of the

exact distribution of the disturbances. In fact, many common estimators in

econometrics can be considered as special cases of GMM [Nobel Prize in Economics 2012, Hansen].

Motivation to GMM: The proposed model builds up a flexible tool to model data in

finance, ....

Earlier estimation methods: …such as MLE, continuum GMM. We apply a novel

approach, …

Estimation of GMM in our study: … estimation for semi-parametric models is done by

(3)

Introduction

1. finite number of observations

generate

finite number of moment

conditions

(empirical) and at the same time moment conditions for

infinite number of theoretical processes

can be generated

2. Then moment conditions, especially, first and second moment

conditions should be written. After that the

unknown parameters

of

the model that satisfies these

moment conditions

should be

found/determined by using

Conic Quadratic Optimization

(CQP).

3. In order to generalize our model for the process that has infinite

number of observations, we proof that our model conditions are

(4)

Methodology (General Form)

• Generalized Partial Linear Model (GPLM) which extends the Generalized Linear Model (GLM). A GPLM separates the set of explanatory variables into two

subsets and combines a linear model with a nonlinear model by addition. • The general GPLM model is given by the following function

• financial sectors, many processes expressed by stochastic differential equations can be stated by a linear submodel for the deterministic drift term and by a

nonlinear submodel for the stochastic diffusion term. • The general GPLM model is given by the following function

• Here, is a finite-dimensional parameter and is a smooth function which is introduced to estimate the nonlinear part of GPLM.

(1)



,



T

 

Y  f X T    X    T    1, 2,...,  T p     

(5)

Methodology (General Form)

• In order to represent the smooth function, nonparametric models are

preferred because they are more flexible than nonlinear models. Here, we also assume that vectors and come from a decomposition of the set of

explanatory variables.

• denotes an p-dimensional random vector which typically represents

discrete covariates,

• is a q-dimensional random vector of continuous covariates which is to be modeled in a nonparametric way

• There are different kinds of

estimation methods

for

GPLM

.

Generally, the estimation methods for the model of Eqn. (1) are

based on the idea that an estimate can be found for a known

and an estimate can be found for a known ,

especially, for .

X T

 

   ˆ

 

T X

ˆ



 ˆ 

(6)

Methodology (General Form)

• _{In our study gamma is based on a}_{special nonparametric regression technique}_,

CMARS, that contains a smoother, i.e., a mollifying operation, such as a

regularization.

• _{CMARS uses}_{expansions in piecewise linear basis functions}_{; thus, can be} represented by a linear combination of the successively built basis functions and the intercept, , such that Eqn. (1) becomes

• _{where Y is a response variable, is a vector of predictors for mth basis} function

• _{and is an}_{additive stochastic component}_{which is assumed to have zero mean} and finite variance.

• Here, are basis functions taken from a set of Mmax linearly

independent basis elements, are the unknown coefficients for the mth basis function or for the constant 1 .

(2) max 0 1 ( ) M T m m m m Y       x   



t  0   1 , ,...,2  T m m m m q t t t  t  max (m1, 2,...,M ) max (m1, 2,...,M ) (m0)    t m  m 

(7)

Methodology (General Form)

• _The_form_{of the mth}_{basis function}_{is as follows:}

• _{where, is the}_{number of truncated linear functions}_multiplied in the mth basis function, is the input variable corresponding to the jth

truncated linear function in the mth basis function, is the knot value

corresponding to the variable and is the selected sign +1 or −1.

– semi parametric: parametric (linear part) and nonparametric (CMARS)

Application areas:

Particularly, high frequency data

• dependency among dependent and independent variables. • _{parameters encompass}_linear_and_nonlinear_variables.

(3) 1 ( ) : m [ m ( m m)] j j j K m m j s_ x_ _   _  



  x m j 



m j x_ m j s_

 

[ ] : max 0,q _  q , K_m m j x_

(8)

• CMARS Basis Functions

CMARS uses expansions in piecewise linear basis functions of the form

+_{( , ) = [ (} _{)] ,}

(9)

Generalized Method of Moments (GMM)

• Classical methods in finance, such as GMM and Maximum Likelihood (ML), are prone to

yielding biased results due to these model deviations. We recall the GMM (Hansen, 1982)

parameter estimation techniques which compute the unknown values of our semi-parametric model in Eqn. (2). GMM estimation for semi-parametric models will be

established for estimating parameter values in statistical models via a robust way.

• The GMM is an estimation method commonly used in econometrics. A set of orthogonality

conditions ( ) and note that here is iid and to follow a

distribution ) is imposed such that if are true parameter values It is usually not possible to choose so that and so instead we minimize some norm of .

• The orthogonality conditions:

(4)   * *_{, ,}  i i h  h  Y  ˆ i Y Yi i    i1, 2,...,N i  0, i N  E h *, ,Yi i  0 *  *  _h _* _{ 0}  * h 









* 2 2 2 2 2 2 , , i i i i i i i i i i Y h Y Y Y Y    _ _           _         

(10)

Generalized Method of Moments (GMM)

Let be the sample mean of computed on the data

The iterative GMM finds the that minimizes the quadratic form , where W is a symmetric positive definite weighting matrix, and the inverse of

variance-covariance matrix .   1   * * 1 1 , , , 1 N i i i g h Y N        y 



*_{, ,}



i i h  Y  y = y y1, ,...,2 yN * ˆ



_J

_

_*_,_y

_{ }

_ _g _*_,_{y W}

_

T _g

_

_*_,_y

_

(11)

GMM Moments for Conic GPLM

• Our motivation is to find the expectation (1. Moment) of the estimator:

• Remember the Conic GPLM:

• Consider the moment function

• First order moment condition for the Conic GPLM estimator is given as:

• Then, will become as follows:

• Therefore, we want to find the that minimizes the quadratic form of

the the following objective function

(5)



*_,



₀ i E h_  r  _ 0 max 1 ( ) M T m i i m m i i m y       x   



t  _i __{1, 2,...,}_N

  

* *_,



i h  h  r max 0 1 ( ) M T m i i i m m i m r y - - -     x 



t



T T( )



0. i i i E _ y x   d  _



*_,



i g  r





1





* * 1 1 , , 1 N i i i g r h r N    



 





1





* 1 1 , ( ) 1 N T T i i i i i g r N      _   _ 



 y x   d  * ˆ











* * 1 * * 1 1 min , min , 1 N i i i g r h r N    



   

(12)

GMM Moments for Conic GPLM

where is an block matrix constructed by matrices x and and

• _{Where y is a vector and are matrices.}

. Reformulation gives: (6)



*



1



*

 

*



1 2 * 1 1 , _i _N N g r            T y - X y - X y - X * 2 * 1 1 min _N_   y - X   





 _ X x |  d N(p + Mmax)  d  __ _T | _T T















 



* * * * * * * * * * * * * * 2 * * , ( ) ( ) T T T T T T i T T T g r W W                    y X W W y X Wy X Wy X y A y A y A 1 N _y*__Wy _{A WX}_ * max ( ) N p M

(13)

GMM Moments for Conic GPLM

• _{We may write in three way Tikhonov regularization problem}

Now, we write our optimization problem as follows:

or, equivalently again,

. (7) (8) (9) (10) 2 2 * 2 * 2 2 min _y*  _A

_





_

* 2 2 2 * 2 minimize subject to M.   * A y  * , 2 * 2 2 2 * 2 min subject to , 0, , t t t t M        * A y * , * 2 * 2 min subject to , . t t t M       * A y

(14)

Optimality Conditions

• Correct detailed specification of distribution is not

needed

• Variance-covariance matrix, more efficient method

• Computational efficiency

• Optimal choice of weight matrix: maximum

information use

(15)

For sequence in such that .

3 necessary assumption for consistency:

unique solution provides identification because estimator function converges to a particular solution, otherwise with no solution is given and it would diverge.

Consistency

1. is a closed and bounded set A

2. h( )continuous and converges uniformly to

h( )

3. is the unique solution of h( )=0.

) ( G ) (_n _n ₀    

(16)

Then,

converges almost surely to (true parameter) where

For any form an open ball of with radius Let

Note that the minimizer is attained because P-N is compact and is continuous and has a unique minimizer over the compact set P with = 0.

Since is continuous at , we may shrink the radius of the ball to in order that for all in this smaller ball

Consistency

             



 ( , ) 1 ) ( ) , ( 1 = 1 = ' 1 =     t T t T T t T t T T f x T A A x f T min arg  0 . = : = ₂ ₂ 2 1,                P b T for some P T   _  0 >  0   ) ( 3 1 = ₀ 0   ming P 0 >  0 g 0  g(₀) 0 g * >    ) ( 0 g _O*

(17)

For sufficiently large , . Let

The sequence of functions converges uniformly almost surely to For sufficiently large T , and

It follows that

Since we are free to make the radius as small as possible, the conclusion follows.

Consistency

. , , . ) , ( 1 ) ( ) , ( 1 = ) ( 1 = ' 1 =            



   T _t t T T t T t T f x T A A x f T g .) ( ) ( 2 ) ( > ) ( ₀    ₀   _T  T g g g g      . O T   T T O*Ø  gT g₀ O P   _O*, T     

(18)

Consistency

. , , Illustration for Proof of Consistency 

(19)

Efficiency

• Theorem: Parameter estimation for

minimizing squared error

• Assumptions: compactness

(20)

Asymptotic distribution of the GMM CMARS

Tikhonov estimator

Assumptions

1. compact

2. is continuously differentiable in for any 3. and for

4. has full column (P)

5. following the central limit theorem 6. (11)

)

(

)

,

(

=

)

,

(

ˆ

_

*

_r

_arg

_inf

_h

_

_r

_

2

_L

_

g

g i



 

)

(

=

ˆ

_



max

g

n

arg

    0  ) , ( r h   t

w

h( r, )=0 E  Eh(,r)0  0       ' ) , ( =   t w h E H d h n _n(₀) N 0,        ( ,) <  g wt sup E