The difference and unity of irregular LQ control and standard LQ control and its solution

Zhang, Huanshui; Xu, Juanjuan

ESAIM: Control, Optimisation and Calculus of Variations, Tome 27 (2021), article no. S31

Irregular linear quadratic control (LQ, was called Singular LQ) has been a long-standing problem since 1970s. This paper will show that an irregular LQ control (deterministic) is solvable (for arbitrary initial value) if and only if the quadratic cost functional can be rewritten as a regular one by changing the terminal cost x$$(T)Hx(T) to x$$(T)[H + P₁(T)]x(T), while the optimal controller can achieve P₁(T)x(T) = 0 at the same time. In other words, the irregular controller (if exists) needs to do two things at the same time, one thing is to minimize the cost and the other is to achieve the terminal constraint P₁(T)x(T) = 0, which clarifies the essential difference of irregular LQ from the standard LQ control where the controller is to minimize the cost only. With this breakthrough, we further study the irregular LQ control for stochastic systems with multiplicative noise. A sufficient solving condition and the optimal controller is presented based on Riccati equations.

Classification : 93E20, 49K15
Keywords: Irregular, LQ control, Riccati equation, Stochastic control

[1] B. D. O. Anderson and J. B. Moore. Optimal control: linear quadratic methods. Prentice Hall, Englewood Cliffs, NJ (1990). | Zbl

[2] D. J. Bell, Singular problems in optimal control-a survey. Int. J. Control 21 (1975) 319–331. | MR | Zbl | DOI

[3] R. Bellman, The theory of dynamic programming. Bull. Am. Math. Soc. 60 (1954) 503–516. | MR | Zbl

[4] R. Bellman, I. Glicksberg and O. Gross, Some aspects of the mathematical theory of control processes. Rand Corporation, R-313 (1958). | MR | Zbl

[5] J. F. Bonnans and F. J. Silva, First and second order necessary conditions for stochastic optimal control problems. Appl. Math. Optim. 65 (2012) 403–439. | MR | Zbl | DOI

[6] H.-F. Chen, Unified controls applicable to general case under quadratic index. Acta Math. Appl. Sin. 5 (1982) 45–52.

[7] S. Chen, X. Li and X. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36 (1998) 1685–702. | MR | Zbl | DOI

[8] D. Clements and B. Anderson, Singular optimal control: The linear-quadratic problem. Springer-Verlag, New York (1978). | MR | Zbl

[9] R. Gabasov and F. M. Kirillova, High order necessary conditions for optimality. SIAM J. Control 10 (1972) 127–68. | MR | Zbl | DOI

[10] V. Gurman, The method of multiple maxima and optimization problems for space maneuvers. Proc. Second Readings of K. E. Tsiolkovskii, Moscow (1968) 39–51.

[11] D. Hoehener, Variational approach to second-order optimality conditions for control problems with pure state constraints. SIAM J. Control Optim. 50 (2012) 1139–1173. | MR | Zbl

[12] Y. Ho, Linear stochastic singular control problems. J. Optim. Theory Appl. 9 (1972) 24–31. | MR | Zbl | DOI

[13] T. Hsia, On the existence and synthesis of optimal singular control with quadratic performance index. IEEE Trans. Autom. Control 12 (1967) 778–779. | DOI

[14] R. E. Kalman, Contributions to the theory of optimal control. Bol. Soc., Mat. Mexicana 5 (1960) 102–119. | MR | Zbl

[15] I. Kliger, Discussion on the stability of the singular trajectory with respect to “Bang-Bang” control. IEEE Trans. Autom. Control 9 (1964) 583–585.

[16] A. J. Krener, The high order maximal principle and its application to singular extremals. SIAM J. Control Optim. 15 (1977) 256–293.

[17] A. M. Letov, The analytical design of control systems. Autom. Remote Control 22 (1961) 363–372.

[18] F. L. Lewis, D. L. Vrabie and V. L. Syrmos, Optimal control. John Wiley & Sons, Inc. (2012). | MR | Zbl | DOI

[19] J. Moore, The singular solutions to a singular quadratic minimization problem. Int. J. Control 20 (1974) 383–393. | MR | Zbl

[20] R. Penrose, A generalized inverse of matrices. Math. Proc. Camb. Philos. Soc. 52 (1955) 17–19.

[21] L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko, The mathematical theory of optimal process. English translation. Interscience (1962).

[22] Q. Qi and H. Zhang, Time-inconsistent stochastic linear quadratic control for discrete-time systems. Science: China Inf. Sci. 60 (2017) 120204:1 –120204:13. | MR

[23] M. Ait Rami, X. Chen and X. Y. Zhou, Discrete-time indefinite LQ control with state and control dependent noise. J. Global Optim. 23 (2002) 245–265. | MR | Zbl | DOI

[24] M. A. Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized Riccati equation. SIAM J. Control Optim. 40 (2001) 1296–1311. | MR | Zbl | DOI

[25] J. Shi, G.Wang and J. Xiong, Linear-quadratic stochastic Stackelberg differential game with asymmetric information. Science: China Inf. Sci. 60 (2017) 092202:1–092202:15.

[26] J. Speyer and D. Jacobson, Necessary and sufficient conditions for optimality for singular control problems. J. Math. Anal. Appl. 33 (1971) 163–187. | MR | Zbl | DOI

[27] J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems. SIAM J. Control Optim. 54 (2016) 2274–308. | MR | Zbl | DOI

[28] J. C. Willems, A. Kitapci and L. M. Silverman, Singular optimal control: a geometric approach. IAM J. Control Optim. 24 (1986) 323–337.

[29] J. Xu, J. Shi and H. Zhang, A leader-follower stochastic linear quadratic differential game with time delay. Science: China Inf. Sci. 61 (2018) 112202.

[30] H. Zhang, L. Lin, J. Xu and M. Fu, Linear quadratic regulation and stabilization of discrete-time Systems with delay and multiplicative noise. IEEE Trans. Autom. Control 60 (2015) 2599–613. | MR | Zbl | DOI

[31] H. Zhang and J. Xu, Control for Itô stochastic systems with input delay. IEEE Trans. Autom. Control 62 (2017) 350–65. | MR | Zbl | DOI

[32] H. Zhang and J. Xu, Optimal control with irregular performance. Science China Inf. Sci. 62 (2019) 192203.

[33] H. Zhang, J. Xu, Control for Itô stochastic systems with input delay. IEEE Trans. Autom. Control 62 (2017) 350–65. | MR | Zbl | DOI

[34] H. Zhang and X. Zhang, Pointwise second-order necessary conditions for stochastic optimal controls, Part I: The case of convex control constraint. SIAM J. Control Optim. 53 (2015) 2267–96.