This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and exploit a representation as an infimum convolution of optimal transport cost and relative entropy. Also included are examples of computation and approximation of the divergence, and the demonstration of properties that are useful when one quantifies model uncertainty.
Keywords: Relative entropy, optimal transport theory, convex duality, calculus of variation, information-theoretic divergence, risk-sensitive control
@article{COCV_2022__28_1_A10_0,
author = {Dupuis, Paul and Mao, Yixiang},
title = {Formulation and properties of a divergence used to compare probability measures without absolute continuity},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
year = {2022},
publisher = {EDP-Sciences},
volume = {28},
doi = {10.1051/cocv/2022002},
mrnumber = {4371079},
zbl = {1478.60008},
language = {en},
url = {https://www.numdam.org/articles/10.1051/cocv/2022002/}
}
TY - JOUR AU - Dupuis, Paul AU - Mao, Yixiang TI - Formulation and properties of a divergence used to compare probability measures without absolute continuity JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2022 VL - 28 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/cocv/2022002/ DO - 10.1051/cocv/2022002 LA - en ID - COCV_2022__28_1_A10_0 ER -
%0 Journal Article %A Dupuis, Paul %A Mao, Yixiang %T Formulation and properties of a divergence used to compare probability measures without absolute continuity %J ESAIM: Control, Optimisation and Calculus of Variations %D 2022 %V 28 %I EDP-Sciences %U https://www.numdam.org/articles/10.1051/cocv/2022002/ %R 10.1051/cocv/2022002 %G en %F COCV_2022__28_1_A10_0
Dupuis, Paul; Mao, Yixiang. Formulation and properties of a divergence used to compare probability measures without absolute continuity. ESAIM: Control, Optimisation and Calculus of Variations, Tome 28 (2022), article no. 10. doi: 10.1051/cocv/2022002
[1] , and , Information geometry connecting Wasserstein distance and Kullback–Leibler divergence via the entropy-relaxed transportation problem. Inf. Geometry 1 (2018) 13–37. | MR | Zbl | DOI
[2] and , A User’s Guide to Optimal Transport. Springer, Berlin, Heidelberg (2013). | MR
[3] and , Chapter 1 of Data-Driven Stochastic Programming Using Phi-Divergences (2015) 1–19.
[4] , and , Robust Wasserstein profile inference and applications to machine learning. J. Appl. Probab. 56 (2016) 10. | MR | Zbl
[5] and , Quantifying distributional model risk via optimal transport. SSRN Electr. J. (2016). | Zbl
[6] , and , Duality in Vector Optimization. Springer-Verlag, Berlin Heidelberg (2009). | MR | Zbl | DOI
[7] and , Measuring distribution model risk. Math. Finance 26 (2013) 395–411. | MR | Zbl | DOI
[8] and , Distinguishing and integrating aleatoric and epistemic variationi in uncertainty quantification. ESAIM: M2AN 47 (2013) 635–662. | MR | Zbl | Numdam | DOI
[9] and , A Weak Convergence Approach to the Theory of Large Deviations. John Wiley & Sons, New York (1997). | MR | Zbl | DOI
[10] , and , Robust properties of risk–sensitive control. Math. Control Signals Syst. 13 (2000) 318–332. | MR | Zbl | DOI
[11] , , and , Path-space information bounds for uncertainty quantification and sensitivity analysis of stochastic dynamics. SIAM/ASA J. Uncert. Quantific. 4 (2016) 80–111. | MR | Zbl | DOI
[12] and , Robust risk measurement and model risk. Quantit. Finance 14 (2014) 29–58. | MR | Zbl | DOI
[13] and , Robust control and model uncertainty. Am. Econ. Rev. 91 (2001) 60–66. | DOI
[14] , , , and , Optimal mass transport: Signal processing and machine-learning applications. IEEE Signal Process. Mag. 34 (2017) 43–59. | DOI
[15] , Robust sensitivity analysis for stochastic systems. Math. Oper. Res. 41 (2016) 1248–1275. | MR | Zbl | DOI
[16] , and , Robust intensity control with multiple levels of model uncertainty and the dual risk-sensitive problem. In 49th IEEE Conference on Decision and Control (CDC) (2010) 4305–4310.
[17] , and , Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Trans. Inform. Theory 56 (2010) 5847–5861. | MR | Zbl | DOI
[18] and , Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53 (2005) 780–798. | MR | Zbl | DOI
[19] , and , Minimax optimal control of stochastic uncertain systems with relative entropy constraints. IEEE Trans. Automatic Control 45 (2000) 398–412. | MR | Zbl | DOI
[20] and , Mass Transportation Problems. Probability and Its Applications. Springer-Verlag New York (1998). | MR
[21] , Convex Analysis. Princeton University Press, Princeton (1970). | MR | Zbl | DOI
[22] , Functional Analysis. McGraw-Hill, New York (1991). | MR | Zbl
[23] , Optimal Transport for Applied Mathematicians. Progress in Nonlinear Differential Equations and Their Applications. Birkhauser Basel (2015). | MR | Zbl | DOI
[24] and , A framework for Wasserstein-1-type metrics. J. Convex Anal. 26 (2019) 353–396. | MR | Zbl
[25] , Optimal Transport: Old and New. Springer-Verlag, Berlin Heidelberg (2009). | MR | Zbl | DOI
Cité par Sources :
Research supported in part by the National Science Foundation (NSF-DMS-1904992).
Research supported in part by the Air Force Office of Scientific Research (FA-9550-18-1-0214).
