We consider an infinite-horizon discounted optimal control problem for piecewise deterministic Markov processes, where a piecewise open-loop control acts continuously on the jump dynamics and on the deterministic flow. For this class of control problems, the value function can in general be characterized as the unique viscosity solution to the corresponding Hamilton−Jacobi−Bellman equation. We prove that the value function can be represented by means of a backward stochastic differential equation (BSDE) on infinite horizon, driven by a random measure and with a sign constraint on its martingale part, for which we give existence and uniqueness results. This probabilistic representation is known as nonlinear Feynman−Kac formula. Finally we show that the constrained BSDE is related to an auxiliary dominated control problem, whose value function coincides with the value function of the original non-dominated control problem.
Keywords: Backward stochastic differential equations, optimal control problems, piecewise deterministic Markov processes, randomization of controls, discounted cost
Bandini, Elena 1
@article{COCV_2018__24_1_311_0,
author = {Bandini, Elena},
title = {Optimal control of piecewise deterministic {Markov} processes: a {BSDE} representation of the value function},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
pages = {311--354},
year = {2018},
publisher = {EDP Sciences},
volume = {24},
number = {1},
doi = {10.1051/cocv/2017006},
mrnumber = {3843187},
zbl = {1396.93131},
language = {en},
url = {https://www.numdam.org/articles/10.1051/cocv/2017006/}
}
TY - JOUR AU - Bandini, Elena TI - Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2018 SP - 311 EP - 354 VL - 24 IS - 1 PB - EDP Sciences UR - https://www.numdam.org/articles/10.1051/cocv/2017006/ DO - 10.1051/cocv/2017006 LA - en ID - COCV_2018__24_1_311_0 ER -
%0 Journal Article %A Bandini, Elena %T Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function %J ESAIM: Control, Optimisation and Calculus of Variations %D 2018 %P 311-354 %V 24 %N 1 %I EDP Sciences %U https://www.numdam.org/articles/10.1051/cocv/2017006/ %R 10.1051/cocv/2017006 %G en %F COCV_2018__24_1_311_0
Bandini, Elena. Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function. ESAIM: Control, Optimisation and Calculus of Variations, Tome 24 (2018) no. 1, pp. 311-354. doi: 10.1051/cocv/2017006
[1] , A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes. SIAM J. Control Optimiz. 40 (2000) 525–539. | MR | Zbl | DOI
[2] , Existence and uniqueness for backward stochastic differential equations driven by a random measure. Electr. Commun. Probab. 20 (2015) 1–13. | MR | Zbl
[3] and . Optimal control of semi-Markov processes with a backward stochastic differential equations approach. Math. Control Signals Syst. 29 (2017) 1–35. | MR | Zbl | DOI
[4] and , Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes. Stochastic Proc. Appl. 127 (2017) 1441–1474. | MR | Zbl | DOI
[5] and , Weak Dirichlet processes with jumps. Preprint arXiv:1512.06236 (2016). | MR
[6] and , Special weak Dirichlet processes and BSDEs driven by a random measure. Preprint arXiv:1512.06234 (2016). To appear on Bernouilli (2017). | MR
[7] , and . Backward stochastic differential equations and integral-partial differential equations. Stochastics Rep. 60 (1997) 57–83. | MR | Zbl | DOI
[8] and , Optimal control of piecewise deterministic Markov processes with finite time horizon. In Modern Trends in Controlled Stochastic Processes: Theory and Applications. Luniver Press, United Kingdom (2010) 123–143.
[9] , Bounded solutions to backward SDE’s with jumps for utility optimization and indifference hedging. Ann. Appl. Probab. 16 (2006) 2027–2054. | MR | Zbl | DOI
[10] , Point processes and queues, Martingale dynamics. Springer, Series in Statistics (1981). | MR | Zbl | DOI
[11] , Functional Analysis. Sobolev Spaces and Partial Differential Equations. Springer (2010). | MR | Zbl
[12] and , Backward SDE Representation for Stochastic Control Problems with Non Dominated Controlled Intensity. Ann. Appl. Probab. 26 (2016) 1208–1259. | MR | Zbl | DOI
[13] , and , Reflected BSDEs with nonpositive jumps, and controller-and-stopper games. Stochastic Proc. Appl. 125 (2015) 597–633. | MR | Zbl | DOI
[14] and , Existence, uniqueness and comparisons for BSDEs in general spaces. Ann. Probab. 40 (2012) 2264–2297. | MR | Zbl | DOI
[15] and , Backward stochastic differential equations and optimal control of marked point processes. SIAM J. Control Optim. 51 (2013) 3592–3623. | MR | Zbl | DOI
[16] and , Backward stochastic differential equations associated to jump Markov processes and their applications. Stochastic Proc. Appl. 124 (2014) 289–316. | MR | Zbl | DOI
[17] , and , Backward stochastic differential equations driven by a marked point process: an elementary approach, with an application to optimal control. Ann. Appl. Probab. 26 (2016) 1743–1773. | MR | Zbl | DOI
[18] , and , Long time asymptotics for fully nonlinear Bellman equations: a Backward SDE approach. Stochastic Proc. Appl. 126 (2016) 1932–1973. | MR | Zbl | DOI
[19] , and , BSDEs with diffusion constraint and viscous Hamilton−Jacobi equations with unbounded data. Ann. Inst. Henri Poincaré Probab. Stat. 53 (2017) 1528–1547. | MR | Zbl | DOI
[20] and , Continuous Average Control of Piecewise Deterministic Markov Processes. Springer Briefs in Mathematics. Springer (2013). | MR | Zbl
[21] , Markov models and optimization. In Vol. 49 of Monographs on Statistics and Applied Probability. Chapman and Hall, London (1993). | MR | Zbl
[22] and , Piecewise deterministic processes and viscosity solutions. A Volume in Honour of W. H. Fleming on Occasion of His 70th Birthday, edited by , Stochastic Analysis, Control Optimization and Applications. Birkhäuser (1999) 249–268. | MR | Zbl | DOI
[23] and , Probabilities et Potentiel, Chapters I to IV. Hermann, Paris (1978).
[24] , Optimal control of piecewise deterministic Markov processes. Applied stochastic analysis, Vol. 5 of Stochastics Monogr. Gordon and Breach, New York (1991) 303–325. | MR | Zbl
[25] and , Necessary and sufficient conditions for control of piecewise deterministic processes. Stochastics and Stochastics Reports 40 (1992) 125–145. | MR | Zbl | DOI
[26] , and , Backward stochastic differential equations in finance. Math. Finance 7 (1997) 1–71. | MR | Zbl | DOI
[27] , Stochastic Calculus and its Applications. Springer (1982). | MR
[28] and , Probabilistic representation and approximation for coupled systems of variational inequalities. Statist. Probab. Lett. 80 (2010) 1388–1396. | MR | Zbl | DOI
[29] and , BSDE representations for optimal switching problems with controlled volatility. Stoch. Dyn. 14 (2014) 1450003. | MR | Zbl | DOI
[30] and , Randomized and Backward SDE representation for optimal control of non-markovian SDEs. Ann. Appl. Probab. 25 (2015) 2134–2167. | MR | Zbl | DOI
[31] , and , Semimartingale theory and stochastic calculus, Science Press, Bejiing, New York (1992). | MR | Zbl
[32] , Multivariate point processes: predictable projection, Radon−Nikodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 31 (1974/75) 235–253. | MR | Zbl | DOI
[33] , Calcul stocastique et problèmes de martingales. Vol. 714 of Lect. Notes Math. Springer, Berlin (1979). | MR | Zbl
[34] and , Probability essentials. Springer Science and Business Media (2003). | MR | Zbl
[35] and , Limit Theorems for Stochastic Processes, 2nd edition, Springer, Berlin (2003). | MR | DOI
[36] , and , A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization. Monte Carlo Methods Appl. 20 (2014) 145–165. | MR | Zbl | DOI
[37] , and , Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps. Ann. Appl. Probab. 25 (2015) 2301–2338. | MR | Zbl | DOI
[38] and , Feynman−Kac representation for Hamilton−Jacobi−Bellman IPDE. Ann. Probab. 43 (2015) 1823–1865. | MR | Zbl
[39] , , and , Backward SDEs with contrained jumps and quasi-variational inequalities. Ann. Probab. 38 (2010) 794–840. | MR | Zbl | DOI
[40] , Monotonic limit theorem for BSDEs and non-linear Doob-Meyer decomposition. Probab. Theory Rel. 16 (2000) 225–234.
[41] and , Continuous Martingales and Brownian Motion. Grundlehren der mathematischen Wissenschaften. Springer, 3rd edition (1999). | MR | DOI
[42] , Optimal control with state-space constraint II. SIAM J. Control Optim. 24 (1986) 1110–1122. | MR | Zbl | DOI
[43] and , Multidimensional diffusion processes, Springer (2007). | MR | Zbl
[44] , Optimal control of piecewise deterministic Markov process. Stochastics 14 (1985) 165–207. | MR | Zbl | DOI
[45] , Backward stochastic differential equations with random measures. Acta Math. Appl. Sinica 16 (2000) 225–234. | MR | Zbl | DOI
Cité par Sources :
