Optimal control on graphs: existence, uniqueness, and long-term behavior
ESAIM: Control, Optimisation and Calculus of Variations, Tome 26 (2020), article no. 22

The literature on continuous-time stochastic optimal control seldom deals with the case of discrete state spaces. In this paper, we provide a general framework for the optimal control of continuous-time Markov chains on finite graphs. In particular, we provide results on the long-term behavior of value functions and optimal controls, along with results on the associated ergodic Hamilton-Jacobi equation.

Reçu le :
Accepté le :
Première publication :
Publié le :
DOI : 10.1051/cocv/2019071
Classification : 05C81, 34E05, 49L20, 90C40, 93E20
Keywords: Optimal control, graphs, asymptotic analysis, Ergodic Hamilton-Jacobi equation 
@article{COCV_2020__26_1_A22_0,
     author = {Gu\'eant, Olivier and Manziuk, Iuliia},
     title = {Optimal control on graphs: existence, uniqueness, and long-term behavior},
     journal = {ESAIM: Control, Optimisation and Calculus of Variations},
     year = {2020},
     publisher = {EDP Sciences},
     volume = {26},
     doi = {10.1051/cocv/2019071},
     mrnumber = {4068303},
     zbl = {1446.49016},
     language = {en},
     url = {https://www.numdam.org/articles/10.1051/cocv/2019071/}
}
TY  - JOUR
AU  - Guéant, Olivier
AU  - Manziuk, Iuliia
TI  - Optimal control on graphs: existence, uniqueness, and long-term behavior
JO  - ESAIM: Control, Optimisation and Calculus of Variations
PY  - 2020
VL  - 26
PB  - EDP Sciences
UR  - https://www.numdam.org/articles/10.1051/cocv/2019071/
DO  - 10.1051/cocv/2019071
LA  - en
ID  - COCV_2020__26_1_A22_0
ER  - 
%0 Journal Article
%A Guéant, Olivier
%A Manziuk, Iuliia
%T Optimal control on graphs: existence, uniqueness, and long-term behavior
%J ESAIM: Control, Optimisation and Calculus of Variations
%D 2020
%V 26
%I EDP Sciences
%U https://www.numdam.org/articles/10.1051/cocv/2019071/
%R 10.1051/cocv/2019071
%G en
%F COCV_2020__26_1_A22_0
Guéant, Olivier; Manziuk, Iuliia. Optimal control on graphs: existence, uniqueness, and long-term behavior. ESAIM: Control, Optimisation and Calculus of Variations, Tome 26 (2020), article no. 22. doi: 10.1051/cocv/2019071

[1] M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Springer Science & Business Media (2008). | Zbl

[2] G. Barles and P.E. Souganidis, On the large time behavior of solutions of Hamilton-Jacobi equations. SIAM J. Math. Anal. 31 (2000) 925–939. | MR | Zbl | DOI

[3] D.P. Bertsekas, Vol. 1 of Dynamic programming and optimal control. Athena scientific Belmont, MA (2005). | MR | Zbl

[4] P. Brémaud, Vol. 50 of Point processes and queues: martingale dynamics. Springer (1981). | MR | Zbl | DOI

[5] Á. Cartea, S. Jaimungal and J. Penalva, Algorithmic and high-frequency trading. Cambridge University Press (2015).

[6] A. Fathi, Sur la convergence du semi-groupe de Lax-Oleinik. Comptes Rendus de l’Académie des Sciences-Series I-Mathematics 327 (1998) 267–270. | MR | Zbl

[7] W.H. Fleming and H. Mete Soner, Vol. 25 of Controlled Markov processes and viscosity solutions. Springer Science & Business Media (2006). | MR | Zbl

[8] O. Guéant, Existence and uniqueness result for mean field games with congestion effect on graphs. Appl. Math. Optim. 72 (2015) 291–303. | MR | Zbl | DOI

[9] O. Guéant, Vol. 33 of The Financial Mathematics of Market Liquidity: From optimal execution to market making. CRC Press (2016). | MR | DOI

[10] P.-L. Lions, G. Papanicolaou and S. Varadhan, Homogenization of Hamilton-Jacobi equations. Unpublished work, 1986.

[11] G. Namah and J.-M. Roquejoffre, Remarks on the long time behaviour of the solutions of Hamilton-Jacobi equations. Commun. Partial Differ. Equ. 24 (1999) 883–893. | MR | Zbl | DOI

[12] R.S. Sutton and A.G. Barto, Reinforcement learning: An introduction. MIT Press (2018). | MR

Cité par Sources :

The authors would like to thank Guillaume Carlier (Université Paris Dauphine), Jean-Michel Lasry (Institut Louis Bachelier), and Jean-Michel Roquejoffre (Université Paul Sabatier) for the discussions they had on the subject.