Regression Monte Carlo for Impulse Control

Ludkovski, Mike

doi:10.5802/msia.18

Ludkovski, Mike ¹

¹ Department of Statistics and Applied Probability, University of California, Santa Barbara CA 93106-3110, USA

MathematicS In Action, Tome 11 (2022) no. 1, pp. 73-90.

Résumé

I develop a numerical algorithm for stochastic impulse control in the spirit of Regression Monte Carlo for optimal stopping. The approach consists in generating statistical surrogates (aka functional approximators) for the continuation function. The surrogates are recursively trained by empirical regression over simulated state trajectories. In parallel, the same surrogates are used to learn the intervention function characterizing the optimal impulse amounts. I discuss appropriate surrogate types for this task, as well as the choice of training sets. Case studies from forest rotation and irreversible investment illustrate the numerical scheme and highlight its flexibility and extensibility. Implementation in R is provided as a publicly available package posted on GitHub.

Publié le : 2022-09-27

DOI : 10.5802/msia.18

Classification : 93E25, 65C05, 49N25
Mots clés : Impulse Control, Statistical Surrogates, Irreversible Investment

Affiliations des auteurs :

Ludkovski, Mike ¹

¹ Department of Statistics and Applied Probability, University of California, Santa Barbara CA 93106-3110, USA

@article{MSIA_2022__11_1_73_0,
     author = {Ludkovski, Mike},
     title = {Regression {Monte} {Carlo} for {Impulse} {Control}},
     journal = {MathematicS In Action},
     pages = {73--90},
     publisher = {Soci\'et\'e de Math\'ematiques Appliqu\'ees et Industrielles},
     volume = {11},
     number = {1},
     year = {2022},
     doi = {10.5802/msia.18},
     language = {en},
     url = {http://www.numdam.org/articles/10.5802/msia.18/}
}

TY  - JOUR
AU  - Ludkovski, Mike
TI  - Regression Monte Carlo for Impulse Control
JO  - MathematicS In Action
PY  - 2022
SP  - 73
EP  - 90
VL  - 11
IS  - 1
PB  - Société de Mathématiques Appliquées et Industrielles
UR  - http://www.numdam.org/articles/10.5802/msia.18/
DO  - 10.5802/msia.18
LA  - en
ID  - MSIA_2022__11_1_73_0
ER  -

%0 Journal Article
%A Ludkovski, Mike
%T Regression Monte Carlo for Impulse Control
%J MathematicS In Action
%D 2022
%P 73-90
%V 11
%N 1
%I Société de Mathématiques Appliquées et Industrielles
%U http://www.numdam.org/articles/10.5802/msia.18/
%R 10.5802/msia.18
%G en
%F MSIA_2022__11_1_73_0

Ludkovski, Mike. Regression Monte Carlo for Impulse Control. MathematicS In Action, Tome 11 (2022) no. 1, pp. 73-90. doi : 10.5802/msia.18. http://www.numdam.org/articles/10.5802/msia.18/

Bibliographie
Cité par

[1] Aid, René; Federico, Salvatore; Pham, Huyên; Villeneuve, Bertrand Explicit investment rules with time-to-build and uncertainty, J. Econ. Dyn. Control, Volume 51 (2015), pp. 240-256 | MR | Zbl

[2] Alvarez, Luis H. R. A class of solvable impulse control problems, Appl. Math. Optim., Volume 49 (2004) no. 3, pp. 265-295 | DOI | MR | Zbl

[3] Alvarez, Luis H. R. Optimal capital accumulation under price uncertainty and costly reversibility, J. Econ. Dyn. Control, Volume 35 (2011) no. 10, pp. 1769-1788 | DOI | MR | Zbl

[4] Alvarez, Luis H. R.; Koskela, Erkki Optimal harvesting under resource stock and price uncertainty, J. Econ. Dyn. Control, Volume 31 (2007) no. 7, pp. 2461-2485 | DOI | MR | Zbl

[5] Alvarez, Luis H. R.; Koskela, Erkki Taxation and rotation age under stochastic forest stand value, J. Environ. Econ. Manage., Volume 54 (2007) no. 1, pp. 113-127 | DOI | Zbl

[6] Alvarez, Luis H. R.; Lempa, Jukka On the optimal stochastic impulse control of linear diffusions, SIAM J. Control Optim., Volume 47 (2008) no. 2, pp. 703-732 | DOI | MR | Zbl

[7] Azcue, Pablo; Muler, Nora; Palmowski, Zbigniew Optimal dividend payments for a two-dimensional insurance risk process, Eur. Actuar. J., Volume 9 (2019) no. 1, pp. 241-272 | DOI | MR | Zbl

[8] Azimzadeh, Parsiad; Bayraktar, Erhan; Labahn, George Convergence of implicit schemes for Hamilton–Jacobi–Bellman quasi-Variational inequalities, SIAM J. Control Optim., Volume 56 (2018) no. 6, pp. 3994-4016 | DOI | MR | Zbl

[9] Basei, Matteo Optimal price management in retail energy markets: an impulse control problem with asymptotic estimates, Math. Methods Oper. Res., Volume 89 (2019) no. 3, pp. 355-383 | DOI | MR | Zbl

[10] Bayraktar, Erhan; Kyprianou, Andreas E.; Yamazaki, Kazutoshi Optimal dividends in the dual model under transaction costs, Insur. Math. Econ., Volume 54 (2014), pp. 133-143 | DOI | MR | Zbl

[11] Bayraktar, Erhan; Ludkovski, Michael Inventory management with partially observed nonstationary demand, Ann. Oper. Res., Volume 176 (2010) no. 1, pp. 7-39 | DOI | MR | Zbl

[12] Belak, Christoph; Christensen, Sören; Seifried, Frank Thomas A general verification result for stochastic impulse control problems, SIAM J. Control Optim., Volume 55 (2017) no. 2, pp. 627-649 | DOI | MR | Zbl

[13] Bensoussan, Alain; Chevalier-Roignant, Benoît Sequential capacity expansion options, Oper. Res., Volume 67 (2019) no. 1, pp. 33-57 | DOI | MR | Zbl

[14] Bensoussan, Alain; Liu, R. H.; Sethi, Suresh P. Optimality of an $(s, S)$ policy with compound Poisson and diffusion demands: A quasi-variational inequalities approach, SIAM J. Control Optim., Volume 44 (2005) no. 5, pp. 1650-1676 | DOI | MR | Zbl

[15] Cadenillas, Abel; Zapatero, Fernando Classical and impulse stochastic control of the exchange rate using interest rates and reserves, Math. Finance, Volume 10 (2000) no. 2, pp. 141-156 | DOI | MR | Zbl

[16] Chen, Yann-Shin Aaron; Guo, Xin Impulse control of multidimensional jump diffusions in finite time horizon, SIAM J. Control Optim., Volume 51 (2013) no. 3, pp. 2638-2663 | DOI | MR | Zbl

[17] Christensen, Sören On the solution of general impulse control problems using superharmonic functions, Stochastic Processes Appl., Volume 124 (2014) no. 1, pp. 709-729 | DOI | MR | Zbl

[18] Czarna, Irmina; Palmowski, Zbigniew De Finetti’s dividend problem and impulse control for a two-dimensional insurance risk process, Stoch. Models, Volume 27 (2011) no. 2, pp. 220-250 | DOI | MR | Zbl

[19] Egami, Masahiko A direct solution method for stochastic impulse control problems of one-dimensional diffusions, SIAM J. Control Optim., Volume 47 (2008) no. 3, pp. 1191-1218 | DOI | MR | Zbl

[20] Egloff, Daniel; Kohler, Michael; Todorovic, Nebojsa A dynamic look-ahead Monte Carlo algorithm for pricing Bermudan options, Ann. Appl. Probab., Volume 17 (2007) no. 4, pp. 1138-1171 | MR | Zbl

[21] El Asri, Brahim; Mazid, Sehail Stochastic impulse control Problem with state and time dependent cost functions, Math. Control Relat. Fields, Volume 10 (2020) no. 4, p. 855 | DOI | MR | Zbl

[22] El Asri, Brahim; Mazid, Sehail Zero-sum stochastic differential game in finite horizon involving impulse controls, Appl. Math. Optim., Volume 81 (2020) no. 3, pp. 1055-1087 | DOI | MR | Zbl

[23] Federico, Salvatore; Rosestolato, Mauro; Tacconi, Elisa Irreversible investment with fixed adjustment costs: a stochastic impulse control approach, Math. Financ. Econ., Volume 13 (2019) no. 4, pp. 579-616 | DOI | MR | Zbl

[24] Guthrie, Graeme Uncertainty and the trade-off between scale and flexibility in investment, J. Econ. Dyn. Control, Volume 36 (2012) no. 11, pp. 1718-1728 | DOI | MR | Zbl

[25] Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome The elements of statistical learning: data mining, inference and prediction, Springer Series in Statistics, Springer, 2009 | DOI

[26] Hu, Jianqiang; Zhang, Cheng; Zhu, Chenbo $(s, S)$ inventory systems with correlated demands, INFORMS J. Comput., Volume 28 (2016) no. 4, pp. 603-611 | MR | Zbl

[27] Kohler, Michael A regression-based smoothing spline Monte Carlo algorithm for pricing American options in discrete time, AStA, Adv. Stat. Anal., Volume 92 (2008) no. 2, pp. 153-178 | DOI | MR | Zbl

[28] Longstaff, Francis A.; Schwartz, Eduardo S. Valuing American options by simulations: a simple least squares approach, Rev. Financ. Stud., Volume 14 (2001), pp. 113-148 | DOI

[29] Ludkovski, Mike mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms (2020) (https://arxiv.org/abs/2012.00729)

[30] Øksendal, Bernt Karsten; Sulem, Agnes Applied stochastic control of jump diffusions, 498, Springer, 2007 | DOI

[31] Tsitsiklis, John; Van Roy, Benjamin Regression Methods for Pricing Complex American-Style Options, IEEE Trans. Neural Netw., Volume 12 (2001) no. 4, pp. 694-703 | DOI

[32] Williams, Christopher K. I.; Rasmussen, Carl Edward Gaussian processes for machine learning, MIT Press, 2006

Cité par Sources :