Dynamic optimization problems for mean-field stochastic large-population systems

Li, Min; Li, Na; Wu, Zhen

ESAIM: Control, Optimisation and Calculus of Variations, Tome 28 (2022), article no. 49

This paper considers dynamic optimization problems for a class of control average meanfield stochastic large-population systems. For each agent, the state system is governed by a linear mean-field stochastic differential equation with individual noise and common noise, and the weight coefficients in the corresponding cost functional can be indefinite. The decentralized optimal strategies are characterized by stochastic Hamiltonian system, which turns out to be an algebra equation and a mean-field forward-backward stochastic differential equation. Applying the decoupling method, the feedback representation of decentralized optimal strategies is further obtained through two Riccati equations. The solvability of stochastic Hamiltonian system and Riccati equations under indefinite condition is also derived. The explicit structure of the control average limit and the related mean-field Nash certainty equivalence equation systems are discussed by some separation techniques. Moreover, the decentralized optimal strategies are proved to satisfy the approximate Nash equilibrium property. The good performance of the proposed theoretical results is illustrated by a practical example from the engineering field.

Classification : 60H10, 60H30, 91A10, 91A25, 93E20
Keywords: Mean-field stochastic differential equation, mean-field game, stochastic Hamiltonian system, Riccati equation, ϵ-Nash equilibrium

[1] H. Brezis, Functional analysis, Sobolev spaces and partial differential equations. Springer Science & Business Media (2010). | MR

[2] R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type. Appl. Math. Optim. 64 (2011) 197–216. | MR | Zbl | DOI

[3] R. Buckdahn, J. Li and J. Ma, A stochastic maximum principle for general mean-field systems. Appl. Math. Optim. 74 (2016) 507–534. | MR | Zbl | DOI

[4] R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: a limit approach. Ann. Probab. 37 (2009) 1524–1565. | MR | Zbl | DOI

[5] R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations. Stochastic Process Appl. 119 (2009) 3133–3154. | MR | Zbl | DOI

[6] R. Carmona and F. Delarue, Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51 (2013) 2705–2734. | MR | Zbl | DOI

[7] S. Chen, X. Li, and X. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36 (1998) 1685–1702. | MR | Zbl | DOI

[8] P. J. Graber, Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource. Appl. Math. Optim. 74 (2016) 459–486. | MR | Zbl | DOI

[9] O. Gueant, J.-M. Lasry and P.-L. Lions, Mean field games and applications. In Paris-Princeton Lectures on Mathematical Finance. Berlin, Germany: Springer (2010). | MR | Zbl

[10] Y. Hu, J. Huang and X. Li, Linear quadratic mean field game with control input constraint. ESAIM Control Optim. Calc. Var. 24 (2018) 901–919. | MR | Zbl | Numdam | DOI

[11] Y. Hu, J. Huang and T. Nie, Linear-quadratic-gaussian mixed mean-field games with heterogeneous input constraints. SIAM J. Control Optim. 56 (2018) 2835–2877. | MR | Zbl | DOI

[12] M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle. SIAM J. Control Optim. 48 (2010) 3318–3353. | MR | Zbl | DOI

[13] J. Huang and N. Li, Linear-quadratic mean-field game for stochastic delayed systems. IEEE Trans. Autom. Control. 63 (2018) 2722–2729. | MR | Zbl | DOI

[14] M. Huang and M. Zhou, Linear-quadratic mean field games: asymptotic solvability and relation to the fixed point approach. IEEE Trans. Automat. Control. 65 (2020) 1397–1412. | MR | Zbl | DOI

[15] M. Huang, R. P. Malhamé and P. E. Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst. 6 (2006) 221–252. | MR | Zbl | DOI

[16] M. Huang, P. E. Caines and R. P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: individualmass behavior and decentralized $ϵ$-Nash equilibria. IEEE Trans. Autom. Control. 52 (2007) 1560–1571. | MR | Zbl | DOI

[17] J. Huang and S. Wang, Dynamic optimization of large-population systems with partial information. J. Optim. Theory Appl. 168 (2016) 231–245. | MR | Zbl | DOI

[18] J. Huang, S. Wang and Z. Wu, Backward mean-field linear-quadratic-gaussian (LQG) games: Full and partial information. IEEE Trans. Autom. Control. 61 (2016) 3784–3796. | MR | Zbl | DOI

[19] J. Huang, B. Wang and J. Yong, Social optima in mean field linear-quadratic-gaussian control with volatility uncertainty. SIAM J. Control Optim. 59 (2021) 825–856. | MR | Zbl | DOI

[20] J. Huang and Z. Yu, Solvability of indefinite stochastic Riccati equations and linear quadratic optimal control problems. Syst. Control Lett. 68 (2014) 68–75. | MR | Zbl | DOI

[21] M. Kac, Foundations of kinetic theory. In Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability. 3 (1956) 171–197. | MR | Zbl

[22] J.-M. Lasry and P.-L. Lions, Mean field games. Jpn. J. Math. 2 (2007) 229–260. | MR | Zbl | DOI

[23] M. Li, N. Li and Z. Wu, Linear-quadratic mean-field game for stochastic large-population systems with jump diffusion. IET Control Theory Appl. 14 (2020) 481–489. | MR | Zbl | DOI

[24] M. Li, T. Nie and Z. Wu, Linear-quadratic large-population problem with partial information: Hamiltonian approach and Riccati approach. Preprint (2022). | arXiv | MR | Zbl

[25] J. Li, Stochastic maximum principle in the mean-field controls. Automatica. 48 (2012) 366–373. | MR | Zbl | DOI

[26] M. Li and Z. Wu, Linear-quadratic non-zero sum differential game for mean-field stochastic systems with asymmetric information. J. Math. Anal. Appl. 504 (2021) 125315. | MR | Zbl | DOI

[27] X. Li, J. Sun and J. Xiong, Linear quadratic optimal control problems for mean-field backward stochastic differential equations. Appl. Math. Optim. 80 (2019) 223–250. | MR | Zbl | DOI

[28] N. Li, X. Li and Z. Yu, Indefinite mean-field type linear-quadratic stochastic optimal control problems. Automatica. 122 (2020) 109267. | MR | Zbl | DOI

[29] N. Li, X. Li and Z. Yu, Indefinite mean-field type linear-quadratic stochastic optimal control problems. Preprint (2020). | arXiv | MR | Zbl

[30] H. P. Mckean, A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl. Acad. Sci. U.S.A. 56 (1966) 1907–1911. | MR | Zbl | DOI

[31] Z. Qian and X. Zhou, Existence of solutions to a class of indefinite stochastic Riccati equations. SIAM J. Control Optim. 51 (2013) 221–229. | MR | Zbl | DOI

[32] B. Wang and J. Zhang, Mean field games for large-population multiagent systems with Markov jump parameters. SIAM J. Control Optim. 50 (2012) 2308–2334. | MR | Zbl | DOI

[33] Q. Wei, J. Yong and Z. Yu, Linear quadratic stochastic optimal control problems with operator coefficients: Open-loop solutions. ESAIM Control Optim. Calc. Var. 25 (2019) 17. | MR | Zbl | Numdam | DOI

[34] J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51 (2013) 2809–2838. | MR | Zbl | DOI

[35] Z. Yu, Equivalent cost functionals and stochastic linear quadratic optimal control problems. ESAIM Control Optim. Calc. Var. 19 (2013) 78–90. | MR | Zbl | Numdam | DOI