The Second-Order Differential Equation System with the Controlled Process for Variational Inequality with Constraints

Wang, Li; Chen, Xingxu; Sun, Juhe

doi:https://doi.org/10.1155/2021/9936370

Complexity

On this page

Abstract Introduction Preliminaries Conclusions Data Availability Disclosure Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Advanced Control and Optimization for Complex Energy Systems 2021

View this Special Issue

Research Article | Open Access

Volume 2021 | Article ID 9936370 | https://doi.org/10.1155/2021/9936370

The Second-Order Differential Equation System with the Controlled Process for Variational Inequality with Constraints

Li Wang,¹Xingxu Chen,¹and Juhe Sun¹

Academic Editor: Chun Wei

Received08 Mar 2021

Revised29 Apr 2021

Accepted21 May 2021

Published01 Jun 2021

Abstract

In this paper, the variational inequality with constraints can be viewed as an optimization problem. Using Lagrange function and projection operator, the equivalent operator equations for the variational inequality with constraints under the certain conditions are obtained. Then, the second-order differential equation system with the controlled process is established for solving the variational inequality with constraints. We prove that any accumulation point of the trajectory of the second-order differential equation system is a solution to the variational inequality with constraints. In the end, one example with three kinds of different cases by using this differential equation system is solved. The numerical results are reported to verify the effectiveness of the second-order differential equation system with the controlled process for solving the variational inequality with constraints.

1. Introduction

We consider the variational inequality with constraints, denoted by VIP : find such thatwhere , is a monotone mapping, is a convex and differentiable mapping, and is a nonempty closed convex set.

Variational inequality problems arise in physics, mechanics, economics, optimization, control, equilibrium model in transportation, and so forth. The finite dimensional variational inequality is an active field with abundant intension. In the past 30 years, great progress has been made in the study of numerical algorithms for solving the variational inequality problems and many numerical methods have emerged in large numbers, such as nonsmooth equation methods, smoothing methods, projection iteration methods, interior methods, multisplitting methods, homotopy approaches, Tikhonov regularization, and proximal point methods. The book by Facchinei and Pang [1] is the most famous reference to give a detailed numerical treatment of variational inequality problems and complementarity problems in the area of mathematical programming. But this book did not include the differential equation approaches for solving variational inequality problems.

The differential equation method is used to solve constrained nonlinear optimization problems, which was firstly proposed by Arrow and Hurwicz [2]; and the equity-constrained optimization problem was considered. Fiacco and McCormick [3] studied the constraint qualification by using differential equation method. Evtushenko [4] studied the problem of equality constraint earlier. Yamashita [5], Evtushenko et al. [6–10], and Pan [11] developed and improved the differential equation methods. In particular, Evtushenko and Zhadan have carried out a great deal of research on differential equation methods for nonlinear programming problems and constraint problems on general closed sets by using the stability theory of differential equations since 1973, which enriches the differential equation method of nonlinear programming problem. There are other scholars who have done a lot of research work on differential equation methods, and they have established a variety of differential equation systems to solve the optimization problems. Brown et al. [12, 13] proposed the differential equation system based on the penalty function. Zhang and Constantinides [14] presented a differential equation system based on Lagrange function to solve the equality constraint problem. Since the nonlinear Lagrange function can be used to construct a dual algorithm for solving nonlinear optimization problems and the dual algorithm has no limitation on the feasibility of the original variables, many differential equation systems involved dual variables, such as those in the works of Zhou et al. [15] and Jin et al. [16]. Recently, Jin et al. [17] formulated the differential systems based on the first derivatives and the second derivatives of the approximate augmented Lagrangian. Under suitable conditions, the asymptotic stability of the differential systems and local convergence properties of their Euler discrete schemes are analyzed. Zhang [18] studied the differential equation method for quadratic programming.

At the same time, there are a series of neural network methods for solving variational inequality and complementarity problems. In fact, the model of neural network is often described by the system of differential equations. Zhou et al. [19] transformed the nonlinear complementarity problem into an equivalent optimization problem, and a system of differential equations is constructed to solve the optimization problem. On the basis of projection operator, Gao et al. [20] presented a system of differential equations for solving variational inequality problems with linear and nonlinear constraints. He and Yang [21] studied a new neural network model for nonsymmetric linear variational inequality. The classical neural network model for solving linear variational inequality based on projection operator and compression method is extended. Liao et al. [22] studied the differential equation system of nonlinear complementarity problem. Nazemi and Sabeghi [23] applied a gradient neural network model to efficiently solve the convex second-order cone constrained variational inequality problem. Nazemi and Sabeghi [24] considered a new neural network model to simply solve the convex second-order cone constrained variational inequality problem.

Different from the above results, Antipin [25] considered the first-order and second-order differential equation systems for solving the minimization function on a simple set. The asymptotic and exponential stability of such processes was proved without using Lyapunov function. Furthermore, Antipin [26] studied the differential equation system and the controlled proximal differential equation system for solving the minimization function. The convergence theorems were obtained. Antipin and Antipin [27, 28] established the differential equation system and the controlled proximal differential equation system for solving the saddle problem and proved that the trajectory of the system converges monotonically in norm to one of the equilibrium points. Antipin [29] considered the differential equation system with the controlled process for solving the fixed point of the extremal mapping and proved that the trajectory of the system converges to the solution with exponential rate. Antipin [30] established the differential equation systems and the differential equation system with the controlled process for optimization problem, saddle problem, and the extremal mapping, respectively. The corresponding convergence theorems were also proved. The variational inequality problem with coupled constraints and the fixed point of the extremal mapping with coupled constraints were studied by Antipin and Antipin [31, 32], where symmetric functions were introduced and the differential equation systems with the controlled process for global convergence were proposed. Wang and Wang [33] established the first-order differential equation system to solve the variational inequality with constraints problem (1).

We think that the research of differential equation method for solving variational inequality is not much. In the past, the problem is firstly transformed into a smooth unconstrained optimization problem; then the differential equation method is constructed by using negative gradient for the unconstrained optimization. Inspired by the work of Antipin, our approach is different, which directly starts from the nonsmooth system of equations to establish the differential equation method, which requires relatively weak conditions. At the same time, due to the fact that the differential equation system has mature numerical solution methods, such as Runge-Kutta method, using these pieces of numerical software to solve the problem of variational inequality may achieve good results in practical calculation. It is hoped that our research on the differential equation method for the variational inequality will contribute to the development of the neural network method for solving the variational inequality problem.

In this paper, we will establish the second-order differential equation system with the controlled process for solving the variational inequality with constraints problem (1). The remainder of this paper is organized as follows. In the next section, based on Lagrange function and projection operator, we prove a lemma that gives the equivalent operator equation under certain conditions for problem (1). In Section 3, we establish the second-order differential equation system with the controlled process for the variational inequality with constraints (1) and prove that any accumulation point of the trajectory of the second-order differential equation system is a solution to problem (1). One example with three kinds of different cases is solved by the second-order differential equation system with the controlled process in Section 4, and the transient behaviors of the trajectories of this kind of differential equation system for every case are illustrated.

2. Preliminaries

The projection operator to a convex set is quite useful in reformulating the variational inequality with constraints (1) as an equation.

Let be a convex closed set; for every , there is a unique in such that

The point is the projection of onto , denoted by . The projection operator is well defined over and it is a nonexpansive mapping.

Lemma 1. (see [34]). Let be a real Hilbert space and let be a closed convex set. For a given , satisfies the inequalityif and only if .

In order to obtain the second-order differential equation system, we need to transform the variational inequality with constraints problem (1), which can be viewed as the following optimization problem:where and .

Define the Lagrange function of the minimization problem (4):where and are the primal and dual variables, respectively.

Since is the minimum of , the pair (under certain regularity conditions) is a saddle point of . That is, and .

System (6) can be represented in an equivalent manner as

Lemma 2. Suppose that is a differentiable mapping and is a nonempty closed convex set. Then the pair is the saddle point of the Lagrange function if and only if satisfies the following operator equation:where and are the operators that project a vector onto the positive octant and the set , respectively, and is a parameter.

Proof. System (4) or system (5) implies that satisfies the following inequalities:If the function is differentiable, we havewhere is the Jacobian of the mapping at .
The variational inequality system (10) can be computed bywhere is a parameter.
With the help of projection operators and Lemma 1, the above system of variational inequalities is represented in the form of operator equation aswhere and are the operators that project a vector onto the positive octant and the set , respectively. This completes the proof.

Furthermore, if is convex, we have for the second term in the first inequality of the variational inequality (10) thatfor all . Thus, we can rewrite the variational inequality (10) as follows:

Remark 1. When the function is convex and differentiable, the relations in (6)–(14) are equivalent to each other.

3. Second-Order Differential Equation System

In this section, we consider the second-order differential equation system for solving the variational inequality with constraints problem (1).

The second-order differential equation system is constructed based on equation (8) as follows:where , and are parameters.

In order to provide the convergence of the trajectory of the second-order differential equation system to a saddle point of the Lagrange function , we introduce the following second-order differential equation system with the controlled process inspired by Antipin et al. [25–32] and Wang and Wang [33].where

If and , then system (16) coincides with (8) in the work of Wang and Wang [33].

For simplicity, we denote and . Similarly, the second-order differential equation system with the controlled process (16) can be represented in the form of variational inequalities as follows:

Subsequently, we assume that , and are Lipschitz continuous with the constants , and , respectively. In addition, we suppose that and is a constant.

With regard to these conditions and (16), it is easy to show the following inequalities since the projection operator is a nonexpansive mapping.where .

Now we discuss the convergence of the trajectories , , , and of the second-order differential equation system (16).

Theorem 1. Suppose that the set of solutions of problem (1) is nonempty, is monotone and Lipschitz continuous with the constant , and is convex, differentiable, and Lipschitz continuous with the constant . Suppose that for all , is bounded for all , is Lipschitz continuous with the constant , and is a convex closed set. Then the accumulation point of the trajectory of the second-order differential equation system (16) with , , and is the solution to the variational inequality with constraints problem (1).

Proof. Setting in the variational inequality (18), we haveLet in the variational inequality (20); we obtainwhich implies thatSumming inequalities (24) and (26), we infer thatBy using inequality (23), we compute the above inequality as follows:where .
Since is convex, we transform the above inequality by using inequality (14) as follows:Letting in the first inequality of (14), we haveAdding the above inequality into (29), we haveSetting in the variational inequality (19) and in the variational inequality (21), we deduce thatSumming the above two inequalities and taking into account (22), we obtainIn view of and , we conclude thatSince is monotone, we sum (31) and (34) to obtainThe above inequality yields thatUsing the relationsinequality (35) can be transformed into the following:According to the relationsinequality (38) means thatLet and ; inequality (40) yields thatSince , , and , inequality (41) can be integrated from to :where .
Since and are all bounded for all , we deduce that has the lower bound becauseThus, there exists a constant such thatwhich implies thatThat is,By integrating (45), we havewhere is a constant. We conclude thatwhich means that is bounded for all .
The function is strongly convex, and it is well known that each of its Lebesgue sets is bounded. Thus, the trajectory is bounded. That is, there exists a constant such thatNow we claim that and .
We firstly show that is bounded. It follows from inequality (42) thatwhere is a constant. From , the above inequality means thatIn the light of , the above inequality infers thatNoting that , we conclude that is bounded in the following:which means that is bounded. It follows, fromthat also has lower bound. Thus, there exists a constant such thatwhich yields that the integrals,converge as .
Assuming that there exists such that , , , , and for all , we obtain a contradiction to the convergence of integrals. Hence, there exists a subsequence of time moments such that , , , and . Since and are bounded, we know that and are bounded. We choose the subsequences and of and . Then there exist and such that , , , , , , and as .
Let us consider inequality (16) for all and take the limit as ; we havewhich means that is a solution of variational inequality with constraints (1) by using Lemma 2. This completes the proof.

4. Numerical Results

In this section, we test the example with three kinds of different cases by our differential equation system (16). The transient behaviors of the proposed differential equation systems are demonstrated in each case. The numerical implementation is coded by Matlab R2019a running on a PC with Intel i5 9400F of 2.9 GHz CPU and the ordinary differential equation solver adopted is ode45, which uses a Runge-Kutta (4, 5) formula.

Example 1. Consider the variational inequality with constraints problem with given by and , where is an asymmetric positive definite matrix and . The parameter is used to vary the degree of asymmetry and nonlinearity, and the data of the example are illustrated as follows:and , which have been considered in the work of Dang et al. [35]. Its solution is .
Since the parameter is used to vary the degree of asymmetry and nonlinearity, we apply the second-order differential equation system (16) to solve the variational inequalities in three cases of , , and .

Case 1. When , we take , which means that the degree of asymmetry of is higher than that of the nonlinearity of . In this experiment, we set , , and in the second-order differential equation system with the controlled process (16). Figure 1 describes the convergence behaviors of , , , and of the second-order differential equation system with the controlled process (16) from the given initial points for the example with .

Case 2. We take , which means that the degree of asymmetry of is the same as the degree of the nonlinearity of . In this experiment, we let , , and in the second-order differential equation system with the controlled process (16). The solution trajectories , , , and of the second-order differential equation system with the controlled process (16) from the given initial points for the example with are shown in Figure 2.

Case 3. When , we take , which means that the degree of asymmetry of is lower than that of the nonlinearity of . In this experiment, we set , , and in the second-order differential equation system with the controlled process (16). Figure 3 describes the convergence behaviors of , , , and of the second-order differential equation system with the controlled process (16) from the given initial points for the example with .

Remark 2. In Theorem 1, the conditions of the parameters , and with , , and are sufficient but not necessary for the convergence of the trajectory of the second-order differential equation system with the controlled process (16) for solving the variational inequality with constraints problem (1). The following lists are given to illustrate the influence on the rate of convergence of trajectory of the second-order differential equation system with the controlled process (16) for solving the variational inequality with constraints problem in the above example. From the condition with , we have that . Hence, we give a discussion in the following and take the first case as an example; that is, in our example.
Fixing , and , the convergence time will be changed as follows when and are changed (Table 1). In this experiment, when , and were fixed, we found that the smaller and are, the shorter the convergence time is. Furthermore, we also found that the change of had a great influence on the rate of the convergence when is fixed; meanwhile the change of has a little effect on the rate of the convergence when is fixed.
Fixing , and , the convergence time will be changed as follows when and are changed (Table 2). In this experiment, when , and were fixed, we found that the smaller and are, the shorter the convergence time is. Furthermore, we also found that the change of had a great influence on the rate of the convergence when is fixed; meanwhile the change of has a little effect on the rate of the convergence when is fixed.
Fixing , and , the convergence time will be changed as follows when is changed (Table 3). In this experiment, when , and were fixed, we found that the convergence time is the shortest at about .

5. Conclusions

In this paper, based on the Lagrange function and projection operator, the second-order differential equation system with the controlled process is established for solving the variational inequality with constraints problem (1). The convergence of the trajectories for this kind of differential equation system is proved. Furthermore, we test one example with three kinds of different cases by the second-order differential equation system with the controlled process and illustrate the transient behaviors of the trajectories for this differential equation system in each case, which verify the effectiveness of the second-order differential equation system with the controlled process for solving the variational inequality with constraints.

It is an interesting issue that the second-order differential equation systems are used to solve the optimization problems.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon reasonable request.

Disclosure

Some of the results in this paper were presented at the Proceeding of the 11th World Congress on Intelligent Control and Automation, 2014 (see https://ieeexplore.ieee.org/document/7052904).

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The research was supported by the National Natural Science Foundation of China under Project no. 11801381.

References

F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, Springer-Verlag Inc, New York, NY, USA, 2003.
K. J. Arrow and L. Hurwicz, “Reduction of constrained maxima to saddle point problems,” in Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability, J. Neyman, Ed., pp. 1–26, University of California Press, Berkeley, CA, USA, 1956.
View at: Google Scholar
A. V. Fiacco and G. P. McCormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, John Wiley, New York, NY, USA, 1968.
Y. G. Evtushenko, “Two numerical methods of solving nonlinear programming problems,” Soviet Mathematics Doklady, vol. 15, pp. 420–423, 1974.
View at: Google Scholar
H. Yamashita, “A differential equation approach to nonlinear programming,” Mathmatical Programming, vol. 18, pp. 115–168, 1980.
View at: Publisher Site | Google Scholar
Y. G. Evtushenko, Numerical Optimization Techniques, Optimization Software, Inc. Publication Division, New York, NY, USA, 1985.
Y. G. Evtushenko and V. G. Zhadan, “Barrier-projective methods for nonlinear programming,” Computational Mathematics and Mathematical Physics, vol. 34, pp. 579–590, 1994.
View at: Google Scholar
Y. G. Evtushenko and V. G. Zhadan, “Stable barrier-projection and barrier-Newton methods in nonlinear programming,” Optimization Methods and Software, vol. 3, no. 1–3, pp. 237–256, 1994.
View at: Publisher Site | Google Scholar
Y. G. Evtushenko and V. G. Zhadan, “Stable barrier-projection and barrier-Newton methods for linear and nonlinear programming,” in Algorithms for Continuous Optimization, E. Spedicato, Ed., pp. 255–285, Kluwer Academic Publishers, Berlin, Germany, 1994.
View at: Publisher Site | Google Scholar
Y. G. Evtushenko and V. G. Zhadan, “Space-transformation technique: the state of the art,” Nonlinear Optimization and Applications, vol. 320, pp. 101–123, 1996.
View at: Publisher Site | Google Scholar
P. Q. Pan, “New ODE methods for equality constrained optimization (1)-equations,” Journal of Computational Mathematics, vol. 10, pp. 77–92, 1992.
View at: Google Scholar
A. A. Brown and M. C. Bartholomew-Biggs, “ODE versus SQP methods for constrained optimization, the hatfield polytechnic optimization center,” Tech. Rep., Hatfield Polytechnic, England, UK, 1987, Technical report No. 179.
View at: Google Scholar
A. A. Brown and M. C. Bartholomew-Biggs, “ODE versus SQP methods for constrained optimization,” Journal of Optimization Theory and Applications, vol. 62, no. 3, pp. 371–386, 1989.
View at: Publisher Site | Google Scholar
S. Zhang and A. G. Constantinides, “Lagrange programming neural networks,” IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 39, no. 7, pp. 441–452, 1992.
View at: Publisher Site | Google Scholar
L. Zhou, Y. Wu, L. Zhang, and G. Zhang, “Convergence analysis of a differential equation approach for solving nonlinear programming problems,” Applied Mathematics and Computation, vol. 184, no. 2, pp. 789–797, 2007.
View at: Publisher Site | Google Scholar
L. Jin, L.-W. Zhang, and X.-T. Xiao, “Two differential equation systems for equality-constrained optimization,” Applied Mathematics and Computation, vol. 190, no. 2, pp. 1030–1039, 2007.
View at: Publisher Site | Google Scholar
L. Jin, H. Y. Huang, and H. Huang, “Differential equation method based on approximate augmented Lagrangian for nonlinear programming,” Journal of Industrial & Management Optimization, vol. 16, no. 5, pp. 2267–2281, 2020.
View at: Publisher Site | Google Scholar
L. F. Zhang, “Based on differential equation method for solving quadratic programming modelling,” WIT Transactions on Modelling and Simulation, vol. 60, pp. 259–267, 2014.
View at: Google Scholar
L. M. Zhou, L. W. Zhang, and S. X. He, “A differential equation approach to solving nonlinear complementarity problems,” OR Transactions, vol. 9, no. 3, pp. 8–16, 2005.
View at: Google Scholar
X.-B. Gao, L.-Z. Liao, and L. Qi, “A novel neural network for variational inequalities with linear and nonlinear constraints,” IEEE Transactions on Neural Networks, vol. 16, no. 6, pp. 1305–1317, 2005.
View at: Publisher Site | Google Scholar
B. He and H. Yang, “A neural network model for monotone linear asymmetric variational inequalities,” IEEE Transactions on Neural Networks, vol. 11, no. 1, pp. 3–16, 2000.
View at: Publisher Site | Google Scholar
L.-Z. Liao, H. Qi, and L. Qi, “Solving nonlinear complementarity problems with neural networks: a reformulation method approach,” Journal of Computational and Applied Mathematics, vol. 131, no. 1-2, pp. 343–359, 2001.
View at: Publisher Site | Google Scholar
A. Nazemi and A. Sabeghi, “A novel gradient-based neural network for solving convex second-order cone constrained variational inequality problems,” Journal of Computational and Applied Mathematics, vol. 347, pp. 343–356, 2019.
View at: Publisher Site | Google Scholar
A. Nazemi and A. Sabeghi, “A new neural network framework for solving convex second-order cone constrained variational inequality problems with an application in multi-finger robot hands,” Journal of Experimental & Theoretical Artificial Intelligence, vol. 32, no. 2, pp. 181–203, 2020.
View at: Publisher Site | Google Scholar
A. S. Antipin, “Controlled proximal differential systems for saddle problems,” Differential Equations, vol. 28, no. 11, pp. 1498–1510, 1992.
View at: Google Scholar
A. S. Antipin, “On differential prediction-type gradient methods for computing fixed points of extremel mappings,” Differential Equations, vol. 31, no. 11, pp. 1754–1763, 1995.
View at: Google Scholar
A. S. Antipin, “Feedback-controlled saddle gradient processes,” Automation and Remote Control, vol. 55, no. 3, pp. 311–320, 2003.
View at: Google Scholar
A. S. Antipin, “Minimization of convex functions on convex sets by means of differential equations,” Differential Equations, vol. 30, no. 9, pp. 1365–1375, 1994.
View at: Google Scholar
A. S. Antipin, “On finite convergence of processes to a sharp minimimum and to a smooth minimium with a sharp derivative,” Differential Equations, vol. 30, no. 11, pp. 1703–1713, 1994.
View at: Google Scholar
A. S. Antipin, “From optim to equilibria, dynamics of nonhomogeneous systems,” Proceedings Fo ISA, vol. 3, pp. 35–64, 2000.
View at: Google Scholar
A. S. Antipin, “Solving variational inequalities with coupling constraints with the use of differential equations,” Differential Equations, vol. 36, no. 11, pp. 1587–1596, 2000.
View at: Publisher Site | Google Scholar
A. Antipin, “Differential equations for equilibrium problems with coupled constraints,” Nonlinear Analysis: Theory, Methods & Applications, vol. 47, no. 3, pp. 1833–1844, 2001.
View at: Publisher Site | Google Scholar
L. Wang and S. Wang, “The differential equation method for variational inequality with constraints,” ICIC Express Letters, vol. 9, no. 10, pp. 2787–2794, 2015.
View at: Google Scholar
U. Mosco, “Implicit variational problems and quasi variational inequalities,” Lecture Notes in Mathematics, vol. 543, Springer-Verlag, Berlin, Germany, 1976.
View at: Publisher Site | Google Scholar
C. Dang, Y. Leung, X.-b. Gao, and K.-z. Chen, “Neural networks for nonlinear and mixed complementarity problems and their applications,” Neural Networks, vol. 17, no. 2, pp. 271–283, 2004.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Li Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

868

Downloads

994

Citations