We examine mean field control problems on a finite state space, in continuous time and over a finite time horizon. We characterize the value function of the mean field control problem as the unique viscosity solution of a Hamilton-Jacobi-Bellman equation in the simplex. In absence of any convexity assumption, we exploit this characterization to prove convergence, as N grows, of the value functions of the centralized N-agent optimal control problem to the limit mean field control problem value function, with a convergence rate of order $$. Then, assuming convexity, we show that the limit value function is smooth and establish propagation of chaos, i.e. convergence of the N-agent optimal trajectories to the unique limiting optimal trajectory, with an explicit rate.
Accepté le :
Première publication :
Publié le :
Keywords: Mean field control problem, control of Markov chains, finite state space, cooperative games, social planner, Hamilton-Jacobi-Bellman equation, viscosity solution, finite difference approximation, classical solution, propagation of chaos
@article{COCV_2021__27_1_A33_0,
author = {Cecchin, Alekos},
title = {Finite state {\protect\emph{N}-agent} and mean field control problems},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
year = {2021},
publisher = {EDP-Sciences},
volume = {27},
doi = {10.1051/cocv/2021032},
language = {en},
url = {https://www.numdam.org/articles/10.1051/cocv/2021032/}
}
TY - JOUR AU - Cecchin, Alekos TI - Finite state N-agent and mean field control problems JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2021 VL - 27 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/cocv/2021032/ DO - 10.1051/cocv/2021032 LA - en ID - COCV_2021__27_1_A33_0 ER -
Cecchin, Alekos. Finite state N-agent and mean field control problems. ESAIM: Control, Optimisation and Calculus of Variations, Tome 27 (2021), article no. 31. doi: 10.1051/cocv/2021032
[1] and , Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997).
[2] , , and Finite state mean field games with Wright-Fisher common noise as limits of N-player weighted games. Preprint 2020, . | arXiv
[3] and , Analysis of a finite state many player game using its master equation. SIAM J. Control Optim. 56 (2018) 3538–3568.
[4] , and , Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics. Trans. Am. Math. Soc. 370 (2018) 2115–2160.
[5] , and , Mean field games and mean field type control theory. Vol. 101 of Springer Briefs in mathematics. Springer–Verlag New York (2013).
[6] and , Stable solutions in potential mean field game systems. Nonlinear Differ. Equ. Appl. 25 (2018) 1.
[7] and , Semiconcave Functions Hamilton-Jacobi Equations, and Optimal Control. Birkhauser, Boston (2004).
[8] and , Approximate solutions of the Bellman equation of deterministic control theory. Appl. Math. Optim. 11 (1984) 161–181.
[9] , , and , The master equation and the convergence problem in mean field games. Vol. 201 of Annals of Mathematics Studies. Princeton University Press, Princeton, NJ (2019).
[10] , , and , Second order mean field games with degenerate diffusion and local coupling. Nonlinear Differ. Equ. Appl. 22 (2015) 1287–1317.
[11] and , Forward-backward stochastic differential equations and controlled Mckean Vlasov dynamics. Ann. Probab. 43 (2015) 2647–2700.
[12] and , Probabilistic theory of mean field games with applications: Vol. I, mean field fbsdes, control, and games. Vol. 83 of Probability Theory and Stochastic Modelling. Springer (2018).
[13] and , Probabilistic theory of mean field games with applications: Vol. II, mean field games with common noise and master equations. Vol. 84 of Probability Theory and Stochastic Modelling. Springer (2018).
[14] , , and , On the convergence problem in mean field games: a two state model without uniqueness. SIAM J. Control Optim. 57 (2019) 2443–2466.
[15] and , Selection by vanishing common noise for potential finite state mean field games. Preprint 2020, . | arXiv
[16] and , Probabilistic approach to finite state mean field games. Appl. Math. Optim. 81 (2020) 253–300.
[17] and , Convergence, fluctuations and large deviations for finite state mean field games via the master equation. Stoch. Process. Appl. 129 (2019) 4510–55.
[18] and , Two approximations of solutions of Hamilton-Jacobi equations. Math. Comput. 43 (1984) 1–19.
[19] and Selection of equilibria in a linear quadratic mean-field game. Stochastic Process. Appl. 130 (2020) 1000–1040.
[20] , Extended mean field control problem: a propagation of chaos result. Preprint 2020. | arXiv
[21] , and , McKean-Vlasov optimal control: the dynamic programming principle. Preprint 2020, . | arXiv
[22] , and , McKean-Vlasov optimal control: limit theory and equivalence between different formulations. Preprint 2020, . | arXiv
[23] , On the connection between symmetric N-player games and mean field games. Ann. Appl. Prob. 127 (2017) 757–810.
[24] and , Controlled Markov processes and viscosity solutions. Vol. 25 of Applications of Mathematics. Springer, New York, 2nd edition (2006).
[25] , , and , Mean-field optimal control as Gamma-limit of finite agent controls. Eur. J. Appl. Math. 30 (2019) 1153–1186.
[26] , and , Continuous time finite state mean field games. Appl. Math. Optim. 68 (2013) 99–143.
[27] , Nonlinear Markov Games on a Finite State Space (Mean-field and Binary Interactions). Int. J. Stat. Prob. 1 (2012) 77–91.
[28] , A general characterization of the mean field limit for stochastic differential games. Prob. Theory Related Fields 165 (2016) 581–648.
[29] , Limit theory for controlled McKean-Vlasov dynamics. SIAM J. Control Optim. 55 (2017) 1641–1672.
[30] , On the convergence of closed-loop Nash equilibria to the mean field game limit. Ann. Appl. Prob. 30 (2020) 1693–1761.
[31] and , Dynamic programming for mean field type control. Comptes Rendus Math. 352 (2014) 707–713.
[32] and , Convergence of large population games to mean field games with interaction through the controls. Preprint 2020, . | arXiv
[33] and , Mean-field Markov decision processes with common noise and open-loop controls. Preprint 2019, . | arXiv
[34] and , Bellman equation and viscosity solutions for mean field stochastic control problem. ESAIM: COCV 24 (2018) 437–461.
[35] and , Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics. SIAM J. Control Optim. 55 (2017) 1069–1101.
[36] and , Mean field games under invariance conditions for the state space. Commun. Partial Differ. Equ. 45 (2020) 146–190.
[37] and , Mass Transportation Problems. Springer-Verlag (1998).
[38] , Approximation schemes for viscosity solutions of Hamilton-Jacobi equations. J. Differ. Equ. 59 (1985) 1–43.
Cité par Sources :
This research benefited from the support of the project ANR-16-CE40-0015-01 on “Mean Field Games”, LABEX Louis Bachelier Finance and Sustainable Growth - project ANR-11-LABX-0019, under the Investments for the Future program (in accordance with Article 8 of the Assignment Agreements for Communication Assistance), ECOREES ANR Project, FDD Chair and Joint Research Initiative FiME in partnership with Europlace Institute of Finance.
The author is grateful to François Delarue and Charles Bertucci for helpful comments and suggestions, as well as to two anonymous referees for their remarks which helped to improve the paper.
