Vol. 132 (2017) ACTA PHYSICA POLONICA A No. 3-II Special issue of the 3rd International Conference on Computational and Experimental Science and Engineering (ICCESEN 2016)
Solution of a Class of Optimization Problems
Based on Hyperbolic Penalty Dynamic Framework
F. Evirgen
∗Balıkesir University, Department of Mathematics, Balıkesir, Turkey
In this study, a gradient-based dynamic system is constructed in order to solve a certain class of optimization problems. For this purpose, the hyperbolic penalty function is used. Firstly, the constrained optimization problem is replaced with an equivalent unconstrained optimization problem via the hyperbolic penalty function. Thereafter, the nonlinear dynamic model is defined by using the derivative of the unconstrained optimization problem with respect to decision variables. To solve the resulting differential system, a steepest descent search technique is used. Finally, some numerical examples are presented for illustrating the performance of the nonlinear hyperbolic penalty dynamic system.
DOI:10.12693/APhysPolA.132.1062
PACS/topics: 02.60.Pn, 02.70.–c, 02.30.Hq
1. Introduction
Numerous commonly encountered problems in modern science and technology involve a type of optimization pro-blem, and optimization is thus an attractive research area for many scientists in various disciplines [1–4]. In the li-terature, several efficient methods have been discussed for finding the feasible best solution to these problems. A detailed and modern discussion of these methods can be found in [5].
The gradient-based method is one of these approaches, and was first introduced by Arrow and Hurwicz [6]. In this approach, the optimization problem is transformed into a system of ordinary differential equations, which are equipped with optimality conditions, in order to find the optimal solutions of the optimization problem. Similar studies can be found in the literature [7–17].
In this paper, we construct a hyperbolic penalty dy-namic system for solving a certain class of inequality constrained optimization problem. The proposed system shows that the steady state solutions x (t) of the dyna-mic system can be approximated to the optimal solutions x∗ of the optimization problem on a continuous path as t → ∞. The steepest descent search technique is used to achieve the intended results.
2. Preliminaries
2.1. Inequality constrained problem
Consider the nonlinear programming problem with in-equality constraints:
minx∈Rnf (x) ,
s.t. g (x) ≥ 0, (1)
where f : Rn → R, g : Rn → Rm are C2 functions and
we assume that the feasible set is non-empty.
∗e-mail: fevirgen@balikesir.edu.tr
By the penalty function methods, the constrained op-timization problem is transformed into an unconstrained optimization problem. In the literature, we frequently see that the following penalty function can be used for the problem (1), Ppenalty(g (x)) = m X i=1 min (gi(x), 0) , (2)
where µ > 0 (µ → ∞) is an auxiliary penalty variable. Hence, the solutions of the inequality constrained pro-blem (1) can be obtained under some conditions from the following unconstrained optimization problem,
min Ppenalty(x, µ) = f (x) + µ m X i=1 min (gi(x), 0) s.t. x ∈ Rn, (3)
where µ > 0 (µ → ∞) is an auxiliary penalty variable. This result can be expressed briefly in the following the-orem.
Theorem 1. [5, pp. 404] Let {xk} be a sequence
ge-nerated by the penalty method. Then any limit point of the sequence is a solution to the constrained problem.
2.2. Hyperbolic penalty function method
The hyperbolic penalty method was first presented by Xavier in 1982 [18] in order to solve the constrained opti-mization problem (1). In a similar manner, the hyperbo-lic penalty methods transform constrained optimization problem (1) into an unconstrained optimization problem as follows, min f (x) + m X i=1 Phyp(gi(x)) . (4)
The second term is the hyperbolic penalty function and can be written as,
Phyp(y, α, τ ) = − 1 2tan α y (1062)
Solution of a Class of Optimization Problems Based on Hyperbolic. . . 1063 + s 1 2tan α 2 y2+ τ2, (5)
where α ∈ [0, π/2) and τ ≥ 0. For convenience, the hyperbolic penalty function can be modelled by the fol-lowing form:
Phyp(y, λ, τ ) = −λy +
p
λ2y2+ τ2, (6)
where λ ≥ 0 (λ → ∞) and τ ≥ 0 (τ → 0).
The geometric idea behind the hyperbolic penalty function can be described briefly as follows. Initially, when the constraint is violated, the parameter λ increa-ses; this term goes to infinity in order to apply a penalty to the violated constraint with the help of the hyperbo-lic penalty term (6) until the violation is over. In this manner, it works as a classical exterior penalty function. On the other hand, when the constraint is feasible, the parameter τ decreases sequentially to zero. In this way, the hyperbolic penalty function behaves like an interior penalty and pushes the iteration point to the boundary of the feasible region. The graphical representation of the hyperbolic penalty function can be found in [19].
3. Hyperbolic penalty dynamic system In this section, we construct the hyperbolic penalty dy-namic system to solve the inequality constrained optimi-zation problem (1). The hyperbolic penalty function (6) is used for the optimization problem (1), and is converted to the unconstrained optimization problem (4). The nu-merical solution of problem (4) was investigated by the following vector differential equation,
dx dt = −∇f (x) − m P i=1 ∇Phyp(gi(x)) , dλ dt = rλ, r > 0, dτ dt = −qτ, 0 < p < 1, (7)
where ∇f (x) and ∇Phyp(gi(x)) are the gradient
vec-tors of the objective function and the hyperbolic penalty function with respect to x ∈ Rn, respectively. For
sim-plicity, the hyperbolic penalty dynamic system (7) can be written as dz dt = −∇f (x) − m P i=1 ∇Phyp(gi(x)) rλ −qτ , (8) where zT = xT, λ, τ.
Definition 1. A point xeis referred to an equilibrium
point of (8) if it satisfies the right-hand side of Eq. (8). The Euler discretization scheme is used to find the stable equilibrium point of the hyperbolic penalty dy-namic system (8). The following iteration formulas can be obtained to approximate the solution of (8),
xk+1i (t) = xk i(t) + ∆t × (−∇xif (x) −Pms=1∇xiPhyp(gs(x))) , λk+1= λk+ ∆t(rλ) , τk+1= τk− ∆ t(qτ ) , (9)
where ∇xi, i = 1...n is a gradient of a given function with
respect to xi, and ∆tis the step size for every interval.
4. Numerical examples and application In order to demonstrate the effectiveness of the propo-sed hyperbolic penalty dynamic system, we test several examples using our system (8). We also compare the numerical performance of the proposed dynamic system with various initial points, both feasible and unfeasible.
Example 1. Consider the following nonlinear pro-gramming problem [20, Problem No: 12],
minimize f (x) = 0.5x21+ x22− x1x2− 7x1− 7x2,
subject to g(x) = 25 − 4x2
1− x22≥ 0.
(10) By using the hyperbolic penalty function (6), pro-blem (10) can be transformed into the unconstrained op-timization problem as follows,
F (x, λ, τ ) = 0.5x21+ x22− x1x2− 7x1− 7x2
+Phyp(g (x) , λ, τ ) .
The corresponding dynamic system from (7) is con-structed as dxi dt = −∇xiF (x, λ, τ ) , i = 1...n, dλ dt = rλ, dτ dt = −qτ, x1(0) = 0, x2(0) = 0. (11)
where x1(0) and x2(0) are the feasible initial conditions.
By utilizing the Euler discretization scheme (9) with λ = 10, τ = 0.1, r = 100, q = 0.01 and step size ∆t =
0.0001, the trajectory of the system (11) approaches the expected optimal solution x∗= (2, 3) of the optimization problem (10) (see Figs. 1 and 2).
Fig. 1. Transient behavior xi(i = 1, 2) of the dynamic
system (10).
Example 2. Consider the following nonlinear pro-gramming problem with inequality constraints [21, Pro-blem No: 337], minimize f (x) = 9x2 1+ x22+ 9x23, subject to g1(x) = x1x2− 1 ≥ 0, x2≥ 1, x3≤ 1. (12)
1064 F. Evirgen
Fig. 2. Transient behavior xi(i = 1, 2) of the dynamic
system (10) with ten random initial points.
Fig. 3. Transient behavior xi(i = 1, 2, 3) of the
dyna-mic system (13).
This is a practical problem with an unknown ex-act solution. The expected solution is x∗ =
0.5774, 1.732, −0.2026 × 10−5.
By using the hyperbolic penalty function (6), we have the following unconstrained optimization problem,
F (x, λ, τ ) = 9x21+ x22+ 9x23 + 3 X s=1 Phyp(gs(x), λ, τ ) ,
where Phyp is the hyperbolic penalty function. The
cor-responding dynamic system from (7) is defined as dxi dt = −∇xiF (x, λ, τ ) , i = 1...n, dλ dt = rλ, dτ dt = −qτ, x1(0) = 1, x2(0) = 1, x2(0) = 1, (13)
where x1(0), x2(0) and x3(0) are the feasible initial
condi-tions. Utilizing the Euler discretization scheme (9) with λ = 100, τ = 0.1, r = 10, q = 0.001 and step size
∆t = 0.0001, the trajectory of the system (13)
appro-aches the optimal solution x∗ of the optimization pro-blem (12) (see Figs. 3 and 4).
Fig. 4. Transient behavior xi (i = 1, 2, 3) of the
dyna-mic system (13) with ten different initial points.
5. Conclusions
In this study, we construct a nonlinear dynamic system using the hyperbolic penalty function for a certain class of inequality constrained optimization problems. For this purpose, the steepest descent search technique is adap-ted to the dynamic system, and the Euler discretization scheme is used to find the steady state solution of the hyperbolic penalty dynamic system. Numerical exam-ples show that the structure of the proposed dynamic system is both effective and reliable in solving a class of inequality constrained optimization problems with vari-ous initial points.
Acknowledgments
This work was financially supported by Balıkesir Uni-versity Scientific Research Grant no: BAP 2015/45. The authors would like to thank the Balıkesir University.
References
[1] A.T. Özturan, Acta Phys. Pol. A 128, B-93 (2015). [2] A. Recioui, Acta Phys. Pol. A 128, B-7 (2015). [3] M.J. Pazdanowski, Acta Phys. Pol. A 128, B-213
(2015).
[4] A.T. Özturan, Acta Phys. Pol. A 130, 14 (2016). [5] D.G. Luenberger, Y. Ye, Linear and Nonlinear
Pro-gramming, 3rd ed., Springer, New York 2008. [6] K.J. Arrow, L. Hurwicz, H. Uzawa, Studies in
Li-near and Non-LiLi-near Programming, Stanford Univer-sity Press, California 1958.
[7] A.V. Fiacco, G.P. Mccormick, Nonlinear Pro-gramming: Sequential Unconstrained Minimization Techniques, John Wiley, New York 1968.
Solution of a Class of Optimization Problems Based on Hyperbolic. . . 1065
[8] H. Yamashita, Math. Program. 18, 155 (1976). [9] C.A. Botsaris, J. Math. Anal. Appl. 63, 177 (1978). [10] A.A. Brown, M.C. Bartholomew-Biggs, J. Optim.
Theory Appl. 62, 371 (1989).
[11] Y.G. Evtushenko, V.G. Zhadan, Opt. Meth. Soft-ware 3, 237 (1994).
[12] J. Schropp, I. Singer, Numer. Funct. Anal. Optim. 21, 537 (2000).
[13] L. Jin, Appl. Math. Comput. 206, 186 (2008). [14] N. Özdemir, F. Evirgen, Bull. Malays. Math. Sci.
Soc. 33, 79 (2010).
[15] F. Evirgen, N. Özdemir, J. Comput. Nonlinear Dyn. 6, 021003 (2011).
[16] F. Evirgen, N. Özdemir, A Fractional Order Dy-namical Trajectory Approach for Optimization Pro-blem with HPM, Eds. D. Baleanu, J.A.T. Machado, A.C.J. Luo, Springer, 2012, p. 145.
[17] F. Evirgen, Int. J. Optim. Control: Th. Applicat. (IJOCTA) 6, 75 (2016).
[18] A.E. Xavier, M.Sc. Thesis, Federal University of Rio de Janerio/COPPE, Rio de Janerio 1982.
[19] A.E. Xavier, Intl. Trans. in Op. Res. 8, 659 (2001). [20] W. Hock, K. Schittkowski, Test Examples for Non-linear Programming Codes, Springer-Verlag, Berlin 1981.
[21] K. Schittkowski, More Test Examples For Nonlinear Programming Codes, Springer, Berlin 1987.
