This work introduces a sequential convex programming framework for non-linear, finitedimensional stochastic optimal control, where uncertainties are modeled by a multidimensional Wiener process. We prove that any accumulation point of the sequence of iterates generated by sequential convex programming is a candidate locally-optimal solution for the original problem in the sense of the stochastic Pontryagin Maximum Principle. Moreover, we provide sufficient conditions for the existence of at least one such accumulation point. We then leverage these properties to design a practical numerical method for solving non-linear stochastic optimal control problems based on a deterministic transcription of stochastic sequential convex programming.
Accepté le :
Première publication :
Publié le :
DOI : 10.1051/cocv/2022060
Keywords: Nonlinear stochastic optimal controlsequential convex programmingconvergence of Pontryagin extremal snumerical deterministic reformulation
@article{COCV_2022__28_1_A64_0,
author = {Bonalli, Riccardo and Lew, Thomas and Pavone, Marco},
title = {Sequential convex programming for non-linear stochastic optimal control },
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
year = {2022},
publisher = {EDP-Sciences},
volume = {28},
doi = {10.1051/cocv/2022060},
mrnumber = {4498190},
zbl = {1500.49014},
language = {en},
url = {https://www.numdam.org/articles/10.1051/cocv/2022060/}
}
TY - JOUR AU - Bonalli, Riccardo AU - Lew, Thomas AU - Pavone, Marco TI - Sequential convex programming for non-linear stochastic optimal control JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2022 VL - 28 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/cocv/2022060/ DO - 10.1051/cocv/2022060 LA - en ID - COCV_2022__28_1_A64_0 ER -
%0 Journal Article %A Bonalli, Riccardo %A Lew, Thomas %A Pavone, Marco %T Sequential convex programming for non-linear stochastic optimal control %J ESAIM: Control, Optimisation and Calculus of Variations %D 2022 %V 28 %I EDP-Sciences %U https://www.numdam.org/articles/10.1051/cocv/2022060/ %R 10.1051/cocv/2022060 %G en %F COCV_2022__28_1_A64_0
Bonalli, Riccardo; Lew, Thomas; Pavone, Marco. Sequential convex programming for non-linear stochastic optimal control. ESAIM: Control, Optimisation and Calculus of Variations, Tome 28 (2022), article no. 64. doi: 10.1051/cocv/2022060
[1] and , Vol. 87 of Control theory from the geometric viewpoint. Springer Science & Business Media (2013). | MR
[2] and , A Fokker-Planck control framework for multidimensional stochastic processes. J. Comput. Appl. Math. 237 (2013) 487–507. | MR | Zbl | DOI
[3] , Dynamic Programming. Princeton Univ. Press, Princeton, New Jersey (1957). | MR | Zbl
[4] , and , Stochastic optimal open-loop control as a theory of force and impedance planning via muscle co-contraction. PLoS Comput. Biol. 16 (2020) e1007414. | DOI
[5] and , Constrained stochastic LQC: a tractable approach. IEEE Trans Autom. Control 52 (2007) 1826–1841. | MR | Zbl | DOI
[6] and , Solution of a class of stochastic linear-convex control problems using deterministic equivalents. J. Optim. Theory Appl. 62 (1989) 17–27. | MR | Zbl | DOI
[7] , Survey of numerical methods for trajectory optimization, J. Guid. Control Dyn. 21 (1998) 193–207. | Zbl | DOI
[8] , Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim. 14 (1976) 419–444. | MR | Zbl | DOI
[9] and , Sequential quadratic programming. Acta Numer. 4 (1995) 1–51. | MR | Zbl | DOI
[10] , Optimal control of aerospace systems with control-state constraints and delays, Ph.D. thesis. Sorbonne Université (2018).
[11] , and , Analytical initialization of a continuation-based indirect method for optimal control of endo-atmospheric launch vehicle systems, in IFAC World Congress (2017). | Zbl
[12] , and , Solving Optimal Control Problems for Delayed Control-Affine Systems with Quadratic Cost by Numerical Continuation, in American Control Conference (2017). | Zbl
[13] , and , Continuity of Pontryagin extremals with respect to delays in nonlinear optimal control. SIAM J. Control Optim. 57 (2019) 1440–1466. | MR | Zbl | DOI
[14] , and , Optimal control of endo-atmospheric launch vehicle systems: geometric and computational issues. IEEE Trans. Autom. Control 65 (2019) 2418–2433. | MR | Zbl | DOI
[15] and , Pontryagin maximum principle for finite dimensional nonlinear optimal control problem on time scales. SIAM J. Control Optim. 51 (2013) 3781–3813. | MR | Zbl | DOI
[16] , Applied optimal control: optimization, estimation and control. CRC Press (1975). | MR
[17] , vol. 1 of Lectures on BSDEs, stochastic control, and stochastic differential games with financial applications. SIAM (2016). | MR | Zbl
[18] , and , Singular trajectories of control-ane systems. SIAM J. Control Optim. 47 (2008) 1078–1095. | MR | Zbl | DOI
[19] , , and , Numerical solution of the finite horizon stochastic linear quadratic control problem. Numer. Linear Algebra Appl. 24 (2017) e2091. | MR | Zbl | DOI
[20] and , Local convergence of generalized Gauss-Newton and sequential convex programming, in Conference on Decision and Control (2019).
[21] and , Local convergence of sequential convex programming for nonconvex optimization, in Recent Advances in Optimization and its Applications in Engineering. Springer (2010) 93–102.
[22] , and , Stochastic optimal control problems with control and initial-final states constraints. SIAM J. Control Optim. 56 (2018) 1823–1855. | MR | Zbl | DOI
[23] , Principles of optimal control theory, vol. 7. Springer Science & Business Media (2013). | MR | Zbl
[24] , Monte-Carlo methods and stochastic processes: from linear to non-linear. CRC Press (2016). | MR | Zbl | DOI
[25] and , Convergence results for smooth regularizations of hybrid nonlinear optimal control problems. SIAM J. Control Optim. 49 (2011) 1498–1522. | MR | Zbl | DOI
[26] , and , Optimal covariance control for stochastic systems under chance constraints. IEEE Control Syst. Lett. 2 (2018) 266–271. | MR | DOI
[27] and , Linear convex stochastic optimal control with applications in production planning. IEEE Trans. Autom. Control 18 (1973) 56–59. | MR | Zbl | DOI
[28] , and , Primal and dual linear decision rules in stochastic and robust optimization. Math. Program. 130 (2011) 177–209. | MR | Zbl | DOI
[29] , Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim. 28 (1990) 999–1048. | MR | Zbl | DOI
[30] and , Numerical methods for stochastic singular control problems. SIAM J. Control Optim. 29 (1991) 1443–1475. | MR | Zbl | DOI
[31] and , A maximum principle for stochastic control systems. J. Math. Anal. Appl. 8 (1964) 287–302. | MR | Zbl | DOI
[32] , vol. 274 of Brownian motion, martingales, and stochastic calculus. Springer (2016). | MR | Zbl
[33] , and , Solving stochastic LQR problems by polynomial chaos. IEEE Control Syst. Lett. 2 (2018) 641–646. | MR | DOI
[34] , and , Chance-constrained sequential convex programming for robust trajectory optimization, in European Control Conference (2020).
[35] , Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part I. Commun. Partial Differ. Equ. 8 (1983) 1101–1174. | MR | Zbl | DOI
[36] , Sequential Convex Programming Methods for A Class of Structured Nonlinear Programming. Tech. rep. (2013).
[37] , , and , Successive convexification of non-convex optimal control problems with state constraints. IFAC-PapersOnLine 50 (2017) 4063–4069. | DOI
[38] , Stochastic models, estimation, and control. Academic Press (1982). | MR | Zbl
[39] and , Numerical Optimization. Springer (1999). | MR | Zbl | DOI
[40] , and , Nonlinear optimization by successive linear programming. Manag. Sci. 28 (1982) 1106–1120. | Zbl | DOI
[41] , A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28 (1990) 966–979. | MR | Zbl | DOI
[42] , Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 30 (1992) 284–304. | MR | Zbl | DOI
[43] , Mathematical theory of optimal processes. Routledge (2018). | DOI
[44] , A matrix equation arising in statistical filter theory. Rep. RE-9, Experimental Astronomy Laboratory, Massachusetts Institute of Technology (1965).
[45] and , Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Trans. Autom. Control 45 (2000) 1131–1143. | MR | Zbl | DOI
[46] and , Generalized linear-quadratic problems of deterministic and stochastic optimal control in discrete time. SIAM J. Control Optim. 28 (1990) 810–822. | MR | Zbl | DOI
[47] and , On complexity of stochastic programming problems, in Continuous optimization. Springer (2005) pp. 111–146. | MR | Zbl | DOI
[48] and , Regularity properties of optimal controls for problems with time-varying state and control constraints. Nonlinear Anal.: Theory, Methods Appl. 65 (2006) 448–474. | MR | Zbl | DOI
[49] , General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim. 42 (2003) 53–75. | MR | Zbl | DOI
[50] , Some properties of the value function and its level sets for affine control systems with quadratic cost. J. Dyn. Control Syst. 6 (2000) 511–541. | MR | Zbl | DOI
[51] , Optimal control and applications to aerospace: some results and challenges. J. Optim. Theory Appl. 154 (2012) 713–758. | MR | Zbl | DOI
[52] , , and , Exact controllability of linear stochastic differential equations and related problems. Am. Inst. Math. Sci. 7 (2017) 305–345. | MR | Zbl
[53] , and , Stochastic linear-quadratic control via semidefinite programming. SIAM J. Control Optim. 40 (2001) 801–823. | MR | Zbl | DOI
[54] and , Vol. 43 of Stochastic controls: Hamiltonian systems and HJB equations. Springer Science & Business Media (1999). | MR | Zbl
Cité par Sources :
This research was supported by the National Science Foundation under the CPS program (grant #1931815).
