\documentclass[JEP,XML,SOM,Unicode,NoEqCountersInSection,published]{cedram} \datereceived{2020-07-22} \dateaccepted{2021-05-21} \dateepreuves{2021-06-01} \usepackage{mathrsfs} \let\mathcal\mathscr \multlinegap0pt \newcommand\mto{\mathchoice{\longmapsto}{\mapsto}{\mapsto}{\mapsto}} \newcommand{\One}{\mathbf{1}} \newcommand{\argmin}{\arg\min} \let\sep\mid \let\epsilon\varepsilon \DeclareMathOperator{\Id}{Id} \newtheorem{Theorem}{Theorem}[section] \newtheorem{Lemma}{Lemma}[section] \newtheorem{Prop}{Proposition}[section] \newtheorem{Cor}{Corollary} \theoremstyle{definition} \newtheorem{Def}{Definition}[section] \newtheorem{Rem}{Remark}[section] \newtheorem{hyp}{Hypothesis} \datepublished{2021-06-03} \begin{document} \frontmatter \title[Monotone solutions for mean field games master equations]{Monotone solutions for mean field games master equations: finite state space and optimal stopping} \author[\initial{C.} \lastname{Bertucci}]{\firstname{Charles} \lastname{Bertucci}} \address{CMAP, UMR 7641, CNRS, Ecole Polytechnique\\ 91120 Palaiseau, France} \email{charles.bertucci@polytechnique.edu} \urladdr{https://sites.google.com/view/cbertucci/} \thanks{The author acknowledges a partial support from the Chair FDD (Institut Louis Bachelier)} \begin{abstract} We present a new notion of solution for mean field games master equations. This notion allows us to work with solutions which are merely continuous. We first prove results of uniqueness and stability for such solutions. It turns out that this notion is helpful to characterize the value function of mean field games of optimal stopping or impulse control and this is the topic of the second half of this paper. The notion of solution we introduce is only useful in the monotone case. In this article we focus on the finite state space case. \end{abstract} \subjclass{35Q89, 35F50, 91A16} \keywords{Partial differential equations, mean field games} \altkeywords{Équations aux dérivées partielles, jeux à champ moyen} \alttitle{Solutions monotones des équations maîtresses des jeux à champ moyen} \begin{altabstract} On présente une nouvelle notion de solution pour les équations maîtresses des jeux à champ moyen. Cette notion permet de travailler avec des fonctions qui sont simplement continues. On prouve en premier lieu des résultats d'unicité, d'existence et de stabilité pour de telles solutions. On montre alors dans la deuxième partie de cet article que cette notion permet de caractériser la fonction valeur de jeux à champ moyen d'arrêt optimal ou de contrôle impulsionnel. Cette notion a surtout un intérêt dans le cas monotone. On se restreint ici au cas d'un espace d'états fini. \end{altabstract} \maketitle \vspace*{-.5\baselineskip} \tableofcontents \mainmatter \section*{Introduction} This paper introduced a new notion of solution of the mean field games (MFG in short) master equation in the monotone setting and applies it to master equations of games involving optimal stopping or impulse control. Even though this paper is self contained, it falls within a series devoted to the systematic study of MFG of optimal stopping or impulse control. Here we study the master equations associated with optimal stopping or impulse control in finite state space. In \cite{bertucci2018optimal}, by considering Nash equilibria of such games without common noise, we showed that those equilibria are in general in mixed strategies and translated this statement in terms of the system of partial differential equations (PDE) characterizing them. In \cite{bertucci2020fokker}, we extended this notion to the case of impulse control. In \cite{bertucci2020remark} we presented numerical methods for such problems. \subsection*{General introduction} The MFG theory is concerned with the study of games involving an infinite number of non-atomic players interacting through mean field terms. If such games have been studied in Economics for quite some time, a general mathematical framework has only been developed around fifteen years ago by J.\nobreakdash-M.\,Lasry and P.-L.\,Lions. It is presented in \cite{lasry2007mean,lions2007cours}. This theory has known too many developments for us to present them all but we are going to indicate some of them. For the moment, uniqueness of Nash equilibria has been proved almost exclusively in two cases, either under a smallness condition (on the coupling between the players or on the duration of the game) or in the so-called monotone regime, see \cite{lasry2007mean,lions2007cours,porretta2015weak}. In this monotone regime, the study of the Nash equilibria of the game reduces to the study of the master equation, a PDE which is satisfied by the value function of a player seen as depending on its own state and on the measure describing the states of the other players. As soon as the state space is infinite, the master equation is an infinite dimensional PDE \cite{cardaliaguet2019master}, whereas in the finite state space case, this PDE reduces to a system of finite dimensional PDE \cite{lions2007cours,bayraktarcohen,cecchinpelino}. In the case in which the game is deterministic or when the randomness is distributed in an i.i.d. fashion among the players, Nash equilibria can be characterized with a system of finite dimensional forward and backward equations \cite{lasry2007mean,cardaliaguet2019master}. Several other aspects of MFG have been studied, such as their long time average \cite{cardaliaguet2019long,cardaliaguet2012long} and the convergence of the $N$ player game toward the MFG \cite{cardaliaguet2019master,lacker2018convergence,cecchinpelino}. Let us also recall that a probabilistic approach of MFG have been developed, we refer to \cite{carmona2018probabilistic,lacker2016general} for more details on this approach. Finally let us mention that numerical methods have been developed to compute equilibria of MFG, mostly in the forward-backward setting, more details can be found in \cite{achdou2010mean,achdou2020mean,briceno2019implementation}. In recent years, the study of MFG of optimal stopping or other ``singular'' controls have been the subject of a growing number of researches, namely because such games have natural applications in Economy. Concerning the case of optimal stopping, let us mention \cite{bertucci2018optimal, carmona2016mean,nutz2016mean,nutz2020convergence}, for impulse control we refer to \cite{bertucci2020fokker} and to \cite{gomes2016weakly} for optimal switching. Let us also mention that the approach of \cite{bertucci2018optimal,bertucci2020fokker} has been used by P.-L.\,Lions in \cite{lions2018cours} to study a case of MFG of singular controls. \subsection*{Regularity of the solution of the master equation} In the monotone regime, it is known since the work of J.-M.\,Lasry and P.-L.\,Lions that we can define a value function for a MFG. This is a consequence of a uniqueness property of Nash equilibria of the game. Let us recall that, formally, the monotone regime is a situation in which the players have a tendency to repel each other. Precise assumptions on the monotonicity shall be made later on. If it is smooth, this value function satisfies the master equation. However, the smoothness of this value function may be difficult to prove in general \cite{cardaliaguet2019master} and for the moment no general weak notion of solution for the master equation has been established. We provide in this paper a notion of solution which demands only for the value function to be continuous. We refer to these solutions as monotone solutions since our approach relies heavily on the monotonicity of the MFG. Let us mention that in this paper we present this solutions in the finite state space case, but that this approach can be extended. In fact the case of a continuous state space shall be treated in \cite{bertucci2020forthcomming}. Recently, several authors have worked to define weak solutions of the MFG master equation, particularly in the non-monotone regime, namely \cite{mou,gangbo,zhang} in the continuous state space case and \cite{cecchin2019convergence} in the finite state space case. Although their approaches are quite different from the one we adopt here (because monotonicity assumptions are crucial to this paper), they provide interesting results and approaches. \subsection*{Mean field games with optimal stopping or singular control} An objective of this paper is to study the master equation, in finite state space, associated to optimal stopping or impulse controls. In those situations, the players can, respectively, decide to exit the game or to change instantly their state to any other state. The main difficulty arising from the study of those games is that the evolution of a population of players using such controls is not smooth in general, independently of any regularity assumptions. This fact, already presented in \cite{bertucci2018optimal,bertucci2020fokker}, is a crucial obstacle to overcome. Indeed as the evolution of the population of players is not smooth, the formulation of the associated master equation has to take into account the fact that the measure describing the repartition of players can instantly jump from one state to another. We shall see that this singular behavior for the evolution of the density of players does not translate into a loss of regularity for the value function (\ie the solution of the master equation). We believe that, in some sense, this is reminiscent of the problem of Hamilton-Jacobi equations associated with singular controls as in \cite{lasry2000classe}. \subsection*{A comment on modeling} Let us comment on a modeling choice we make in this paper. We intend to look at the master equation set on the whole $(\mathbb{R}_+)^d$. In the continuous state space case, in the setting of \cite{cardaliaguet2019master} for instance, this would correspond to look at the master equation posed on the set of positive measures instead of on the set of probability measures. In the appendix we explain how we can pass from one setting to another in finite state space. This choice to work on $(\mathbb{R}_+)^d$ is clearly motivated in the optimal stopping case because the mass of players is not constant (it can decrease). However, we argue that this kind of approach is also meaningful in general. Indeed, in a MFG the number of player is infinite and the choice to normalize their mass to one is only an arbitrary choice. Moreover, if in a lot of cases studied in the literature the mass of player is preserved, it is also natural to consider, outside of stopping or entry game, MFG in which players leave or enter the game. For instance let us refer to \cite{claisse2019mean}. Moreover, in the case in which the mass of player is preserved, solving the master equation for any initial mass of players is a suitable strategy. To conclude this word on modeling, let us mention that to have the same point of view in the probabilistic approach of MFG, in which the measure characterizing the distribution of players is often thought as the probability measure of the state of one player, one need to consider either unnormalized measures for the state of a player or either an additional real parameter which stands for the mass of players. \subsection*{Structure of the paper} The rest of this paper is organized as follows. In Section~\ref{sec:1}, we present some preliminary results concerning the study of master equations in the finite state space case, in particular for situations in which there is a boundary. In~Section~\ref{sectionmon}, we present our notion of monotone solutions. In Section~\ref{sectstop}, we derive a characterization for the value function in the optimal stopping case, as well as we prove existence and uniqueness results for such value functions. We present in Section~\ref{sec:4} the master equation associated with an impulse control problem. Finally Section~\ref{sec:5} is concerned with a MFG model of entry and exit, in a simplified economic market which applies the concepts developed in the previous sections. \subsection*{Notations} We shall use the following notations. \begin{itemize} \item We denote by $d$ an integer greater than $1$. \item $\mathbb{O}_d = (\mathbb{R}_+)^d$. \item The Euclidean norm of $\mathbb{R}^d$ is denoted $|{\cdot}|$ and its associated operator norm $\|{\cdot}\|$. The~$\ell_p$ norms on $\mathbb{R}^d$ are denoted by $|{\cdot}|_p$. The Euclidean scalar product between \hbox{$x,y \in \mathbb{R}^d$} is denoted either $x \cdot y$ or $\langle x, y \rangle$. \item An application $A: \mathcal{O} \to \mathbb{R}^d$, defined on a subset $\mathcal{O} \subset \mathbb{R}^d$ is said to be monotone~if \begin{equation*} \langle A(x)- A(y) , x-y \rangle \geq 0, \text{ for } x,y \in \mathcal{O}. \end{equation*} \item We define for $R\geq 0$, $B^1_R:= \{ x \in \mathbb{O}_d\sep x_1+\cdots+x_d \leq R\}$. \item For a $d\times d$ matrix $A$, we say that $A > 0$ in the order of positive definite matrices if for all $\xi \in \mathbb{R}^d\setminus \{0\}$ \[ \langle \xi, A \xi \rangle > 0. \] \item For $A \in \mathcal{L}(\mathbb{R}^d)$, we define \begin{equation*} \zeta(A):= -\inf \{ \langle \xi, A\xi\rangle \sep |\xi|^2 \leq 1\}. \end{equation*} \item We denote the term by term product between matrices or vectors by $*$. \item For $x \in \mathbb{R}^d$, we note $x\leq 0$ when all the components of $x$ are non-positive. \item If $f: \mathbb{R} \to \mathbb{R}$ and $x\in \mathbb{R}^d$, then $f(x):= (f(x_1),\dots,f(x_d))$. \end{itemize} \subsubsection*{Acknowledgments} We would like to thank Pierre-Louis Lions for pointing out to us the question of optimal stopping in MFG a few years ago and for the numerous discussions we had on this topic. \section{Preliminary results}\label{sec:1} This section introduces extensions of the results of \cite{lions2007cours} to the case in which the master equation is posed on a domain of $\mathbb{R}^d$, in particular when it is $(\mathbb{R}_+)^d:= \mathbb{O}_d$. We also recall a weak form of Stegall's variational lemma at the end of this section. Let us mention the work \cite{bayraktar2019finite}, which is also concerned with the well-posedness of the master equation in finite state space. In this article the authors derived and studied a particular form of the master equation arising from the presence of a so-called Wright-Fischer common noise in the MFG. This work is completely different than ours because the authors of this paper use the structure of the noise to establish regularity in a non-monotone setting while we make an extensive use of monotonicity to avoid having to use regularity arising from the structure of the randomness. The main results of this section are concerned with the existence and uniqueness of solutions of the master equation \begin{equation}\label{me} \begin{aligned} \partial_t U+(F(x,U)\cdot \nabla_x)U+\lambda (U-T^*U(t,Tx))&= G(x,U) \quad\text{in } (0,\infty) \times \mathbb{O}_d,\\ U(0,x) &= U_0(x) \quad\text{in } \mathbb{O}_d, \end{aligned} \end{equation} or its discounted stationary counterpart \begin{equation}\label{sme} r U+(F(x,U)\cdot \nabla_x)U+\lambda (U-T^*U(t,Tx))= G(x,U) \quad\text{in } \mathbb{O}_d, \end{equation} where $U: (0,\infty)\times \mathbb{O}_d \to \mathbb{R}^d$ is the value function of the MFG which is characterized by $F, G: \mathbb{O}_d \times \mathbb{R}^d\to \mathbb{R}^{2d}$, $\lambda > 0$ and $T \in \mathcal{L}(\mathbb{R}^d)$. Let us note that these PDE are not written on an open set and that no boundary conditions are imposed. The interpretation of \eqref{me} is that it is the PDE satisfied by the value function of the MFG. The value of the game in the state $i\in \{1,\dots,d\}$, when the quantity of player in each state $j \in \{1,\dots,d\}$ is $x_j$ and the time remaining in the game is $t\geq 0$, is: $U^i(t,x)$. The term $\lambda(U-T^* U\circ T)$ stands for the modeling of common noise and we refer to \cite{bertucci2019some} for more details on this question. The function $F$ stands for the evolution of the ``density'' of players and $G$ for the evolution of the value of the MFG. That is, in the case $\lambda = 0$, the characteristics of \eqref{me} are given by $(V(t), y(t))_{0\leq t \leq t_f}$ solution~of% \begin{equation}\label{char} \begin{cases} \dot{V}(t) = G(y(t),V(t)),& 0 < t < t_f ,\ V(0) = U_0(y(0)),\\ \dot{y}(t) = F(y(t),V(t)),& 0 0$ such that for all $x \in \mathbb{O}_d$ with $|x|_1 \geq R$, for all $p \in \mathbb{R}^d$, \begin{equation*} \sum_{i = 1}^d F^i(x,p) \geq 0. \end{equation*} Moreover, $T(B^1_{R'}) \subset B^1_{R'}$ for any $R' \geq R$. \end{hyp} This assumption states that when the mass of player is sufficiently large ($|x|_1 \geq R$), it cannot grow anymore. Thus it has the effect to bound the region of interest when starting from an initial distribution of mass. Although it seems possible to treat situations in which the mass of players can always increase, it is not the objective of this paper. As already mentioned several times above, monotonicity plays a crucial role in the well-posedness of \eqref{me} and \eqref{sme}. We say that we are in the monotone regime when the following assumption is satisfied. \begin{hyp}\label{mon} The function $U_0$ is monotone on $\mathbb{O}_d$ and $(G,F)$ is monotone on $\mathbb{O}_d \times \mathbb{R}^d$. \end{hyp} In order to obtain existence of solutions of either \eqref{me} or \eqref{sme}, we shall use the following stronger assumption. \begin{hyp}\label{smooth} The functions $U_0$, $F$ and $G$ are Lipschitz continuous. Moreover, there exists $\alpha > 0$ such that \begin{align*} \langle \xi, D_x U_0(x) \xi\rangle &\geq \alpha |D_xU_0(x)\xi|^2,\quad \forall x \in \mathbb{O}_d,\ \xi \in \mathbb{R}^d, \\ \begin{pmatrix} D_x G(x,p) & D_x F(x,p) \\ D_p G(x,p) & D_p F(x,p) \end{pmatrix}&\geq \alpha \begin{pmatrix} \Id & 0 \\ 0 & 0 \end{pmatrix} \quad\text{for almost every } x \in \mathbb{O}_d,\ p \in \mathbb{R}^d, \end{align*} in the order of positive matrices. \end{hyp} When addressing the existence of solutions of equations of the type of \eqref{sme}, we~shall also make the following assumption. \begin{hyp}\label{strongstat} The discount rate $r$ satisfies \begin{equation*} r > \zeta(D_xF(x,p))\quad \forall x \in \mathbb{O}_d,\ p \in \mathbb{R}^d. \end{equation*} \end{hyp} \Subsection{Uniqueness results for the master equation in finite state space} We now present uniqueness results concerning \eqref{me} and \eqref{sme}. Those results are direct extensions of the results established in \cite{lions2007cours} and do not need any new ideas, however, as they play a crucial role in the notion of solution we introduce in the next section, we detail their proofs. \begin{Prop}\label{uniq1} Under Hypotheses~\ref{stab} and \ref{mon}, there exists at most one smooth solu\-tion of \eqref{me}. If this solution exists, it is monotone. \end{Prop} \begin{proof} Let us take $U$ and $V$ two smooth solutions of \eqref{me}. We define $W$ by \begin{equation*} W(t,x,y) = \langle U(t,x)-V(t,y) , x-y \rangle. \end{equation*} This function satisfies \begin{equation*} W(0,x,y) = \langle U_0(x)-U_0(y), x-y\rangle \quad\text{in }\mathbb{O}_d^2,\vspace*{-5pt} \end{equation*} \begin{multline*} \partial_t W+F(x,U)\cdot \nabla_x W+F(y,V)\cdot \nabla_y W+\lambda(W-W(t,Tx,Ty))\\ = \langle G(x,U)-G(y,V), x-y\rangle+\langle F(x,U)-F(y,V), U-V \rangle \quad\text{in } (0, \infty) \times \mathbb{O}_d^2. \end{multline*} From Hypothesis~\ref{mon} the right-hand side of the two previous equations are non-negative. From Lemma~\ref{max1} (in appendix), we deduce that $W$ is non-negative for all time. We~can conclude that i) $U = V$ (or otherwise $W$ should change sign around some point), ii) $U$ is monotone for all time. \end{proof} \begin{Prop}\label{uniq2} Under Hypotheses~\ref{stab} through \ref{mon}, there exists at most one smooth solution of \eqref{sme}, and if it exists it is monotone. \end{Prop} \begin{proof} Let us take $U$ and $V$ two smooth solutions of \eqref{me}. We define $W$ by \begin{equation*} W(x,y) = \langle U(x)-V(y) , x-y \rangle. \end{equation*} This function satisfies \begin{multline*} r W+F(x,U)\cdot \nabla_x W+F(y,V)\cdot \nabla_y W+\lambda(W-W(Tx,Ty))\\ = \langle G(x,U)-G(y,V), x-y\rangle+\langle F(x,U)-F(y,V), U-V \rangle \quad\text{in }\mathbb{O}_d^2. \end{multline*} We then conclude as in the previous proof, by using this time Lemma~\ref{max2} in the appendix. \end{proof} \Subsection{Existence results for the master equation in finite state space} We now turn to the questions of existence of solutions of \eqref{me} and \eqref{sme}. As the next section gives a precise definition of a solution of \eqref{me} and \eqref{sme}, we do not focus on the sense in which solutions satisfy the PDE but rather on how we can obtain a priori estimates. In the monotone setting (\ie under Hypothesis~\ref{mon}), an a priori estimate on the spatial gradient of the solution can be proved, exactly as it is already the case in \cite{lions2007cours}. \begin{Prop}\label{existme} Under Hypotheses~\ref{stab}, \ref{mon} and \ref{smooth}, there exists a Lipschitz function~$U$, solution of \eqref{me} almost everywhere, such that for any $t_f > 0$, there exists $C$ such that% \begin{equation*} \|D_x U(t,x)\| \leq C, \quad\forall x \in \mathbb{O}_d,\ t \leq t_f. \end{equation*} \end{Prop} \begin{proof} This proposition can be obtained as a consequence of an a priori estimate on the solution $U$ of \eqref{me}. This idea is mostly borrowed from the lecture \cite{lions2007cours} in which this technique is presented. The only difference between this lecture and our situation is that we consider a master equation on $\mathbb{O}_d$, that is why we are not going to enter in a lot of details here but only indicate the main differences with the case in $\mathbb{R}^d$. For $U$ a smooth solution of \eqref{me}, let us define $W$ and $Z$ with \begin{align*} W(t,x,\xi) &= \langle U(t,x), \xi \rangle,\\ Z_{\beta,\gamma}(t,x,\xi) &= \langle \nabla_x W(t,x,\xi), \xi \rangle-\beta(t) \|\nabla_x W(t,x,\xi)\|^2+\gamma(t) |\xi|^2, \end{align*} for some functions $\beta$ and $\gamma$ to be defined later on. Arguing as in \cite{lions2007cours}, we deduce that $Z_{\beta,\gamma}$ satisfies \begin{multline}\label{Zedp} \partial_t Z_{\beta, \gamma}+\langle F(x,\nabla_{\xi} W), \nabla_x Z_{\beta, \gamma}\rangle+\langle D_p F (x,\nabla_{\xi} W) \nabla_{\xi}Z_{\beta, \gamma} , \nabla_{x} W\rangle\\ \shoveright{-\langle D_p G(x, \nabla_{\xi}W)\nabla_{\xi}Z_{\beta, \gamma}, \xi\rangle + \lambda(Z_{\beta, \gamma}-Z_{\beta, \gamma}(t,Tx, T\xi))}\\ \shoveleft{= \langle D_xG(x, \nabla_x W) \xi, \xi\rangle-\langle D_pG(x, \nabla_{\xi}W) \nabla_x W, \xi\rangle}\\ \shoveright{-\langle D_x F(x, \nabla_{\xi} W) \nabla_x W, \xi\rangle + \langle D_p F(x, \nabla_{\xi}W) \nabla_x W, \nabla_x W\rangle}\\ \shoveright{- 2 \beta \langle D_x G(x, \nabla_x W)\xi, \nabla_x W\rangle+2 \beta \langle D_x F(x, \nabla_{\xi} W) \nabla_x W, \nabla_x W\rangle}\\ \shoveright{+\beta \lambda \bigl(|\nabla_x W|^2-2 \langle \nabla_x W(t, Tx, T \xi), S \nabla_x W\rangle+| \nabla_x W(t,Tx, T \xi)|^2 \bigr)} \\ \shoveright{- \frac{d}{dt}\, \beta |\nabla_x W|^2+2 \gamma(\langle D_p F (x,\nabla_{\xi} W) \xi , \nabla_{x} W\rangle-\langle D_p G(x, \nabla_{\xi}W)\xi, \xi\rangle)}\\ +\frac{d}{dt}\,\gamma |\xi|^2+\lambda \gamma (|\xi|^2-|T\xi-e|^2). \end{multline} Using the monotonicity of $U_0,G$ and $F$ and the strong monotonicity of $U_0$ and $G$, we~deduce that by choosing $\gamma = 0$ and \begin{equation*} \beta(t) = \alpha e^{-t(2\|\nabla_x F\|+2 \|\nabla_x G\|+\lambda(\|T\| -1)_+)}, \end{equation*} the right-hand side of \eqref{Zedp} is non-negative and thus that by Lemma~\ref{max1}, $Z_{\beta,\gamma}$ is non-negative for all time which yields the required a priori estimate on $U$. It may seem strange to the reader that we introduce a function $\gamma$ to take it as $0$ later on. We introduced it because it is useful in other contexts (stationary problem, different assumptions on the monotonicity). Concerning the question of existence of a Lipschitz function satisfying \eqref{me}, the main argument (as in \cite{lions2007cours}) is to remark that the previous technique to obtain a priori estimates still works when one adds particular degenerate elliptic second order terms in \eqref{me}. This is a consequence of the so-called Bernstein method. We now indicate a particular choice of such terms. Let us take $\epsilon > 0$ and consider $\sigma: \mathbb{R} \to \mathbb{R}$ a smooth real bounded function such that $\sigma (x) = x^2$ in a neighborhood of $0$. Assume that $U$ is a smooth solution of \begin{equation}\label{regme} \begin{split} &\partial_t U^i+F(x,U) \cdot \nabla_x U^i-\epsilon \biggl(\sum_{j=1}^d \sigma(x_j)\partial_{jj}U^i\biggr)-\epsilon \sigma'(x_i)\partial_i U^i\\[-3pt] &\hspace*{2.5cm}+\lambda (U^i-(T^*)^iU(t,Tx)) = G^i(x,U) \quad\text{in } \mathbb{O}_d, \forall i \in \{1,\dots,d\},\\ &U(0,x) = U_0(x) \quad\text{in } \mathbb{O}_d. \end{split} \end{equation} Let us remark that as the terms in $\epsilon$ in \eqref{regme} preserves the monotonicity, we are able to adapt the previous technique to \eqref{regme} to establish a, uniform in $\epsilon \in (0,1)$, a priori estimate on the spatial gradient of a solution of \eqref{regme}. Let us now remark that once we have this a priori estimate, from classical results on degenerate elliptic equations~\cite{oleinik2012second}, existence of solutions of \eqref{regme} can be easily obtained and that, passing to the limit $\epsilon \to 0$, we deduce the existence of a Lipschitz function, solution almost everywhere of~\eqref{me}. \end{proof} We now provide an a priori estimate for solutions of \eqref{sme}. As the proof of the following statement is very similar to the one of the previous result, we do not detail it here. \begin{Prop}\label{existstat} Under Hypotheses~\ref{stab} through \ref{strongstat}, any smooth function $U$ solution of \eqref{sme} satisfies \begin{equation*} \forall x \in \mathbb{O}_d,\quad \|D_xU(x)\| \leq C, \end{equation*} for $C > 0$ depending only on $r,G,F,\lambda$ and $T$. \end{Prop} \begin{Rem}\label{rembound} Under the assumptions of the previous proposition, existence of a Lipschitz solution of \eqref{sme} is then easy to obtain once some local boundedness can be established for a solution of \eqref{sme}. Such a property can usually be obtained easily on a case by case basis. We give such an example in the next section on optimal stopping. \end{Rem} \begin{Rem} The main difference for the proof of this statement compared with the time dependent case is that the functions $\beta$ and $\gamma$ in the previous proof must be chosen constant. \end{Rem} \begin{Rem} We firmly believe the restriction on $r$ from Hypothesis~\ref{strongstat} to be mainly technical and due to the rather abstract framework in which we are working. Let us for instance mention \cite{cardaliaguet2019long} in which a Lipschitz estimate is proved for the solution of a first order stationary master equation for any discount rate. \end{Rem} \subsection{Stegall's variational principle} We end this section on preliminary results by recalling a quite weak version of Stegall's lemma \cite{stegall1978optimization,stegall1986optimization,fabian2007stegall} that we shall use many times in the rest of the paper. Moreover, we present a proof of this result that we believe to be new. \begin{Lemma}\label{stegall} Let $\phi: \Omega \to \mathbb{R}$ be a lower semi continuous function from a compact set $\Omega \subset X$ of a separable Hilbert space $X$. Then there is a dense number of points $c$ in $X$ such that $x \mto \phi(x)+\langle c,x\rangle$ has a strict global minimum on $\Omega$. \end{Lemma} \begin{proof} This proof relies on convex analysis. For a set $E$, we denote $\mathcal{P}(E)$ the set of its subsets. Let us consider the operator $A: X \to \mathcal{P}(X)$ defined by $A(c)$ is the set of the points at which $x \mto \phi(x)+\langle c,x\rangle$ reaches its minimum over $\Omega$. Clearly for all $c$, this set is non empty and well-defined. Let us check that the operator $-A$ is cyclically monotone. We consider a finite sequence $c_0,c_2,\dots,c_n = c_0$ and for all $i$, $y_i \in A(c_i)$. For all $i$, let us remark that \begin{equation*} \phi(y_i)+\langle c_i, y_i\rangle \leq \phi(y_{i+1})+\langle c_i, y_{i+1} \rangle. \end{equation*} Rearranging we get \begin{equation*} \langle c_i, y_i-y_{i+1}\rangle \leq \phi(y_{i+1})-\phi(y_i). \end{equation*} Let us now compute \begin{equation*} \sum_{i =1}^n \langle c_i-c_{i-1}, y_i \rangle = \sum_{i =1}^n \langle c_i, y_i-y_{i+1} \rangle \leq \sum_{i = 1}^n \phi(y_{i+1})-\phi(y_i) \leq 0. \end{equation*} Therefore the operator $-A$ is cyclically monotone. Thus we deduce that $A \subset \partial \psi$, where~$\psi$ is a concave function on $X$ and $\partial \psi$ is its super-differential. From a generalization of Alexandrov theorem for separable Hilbert spaces \cite{borwein1994second}, we finally deduce the required result. \end{proof} \section{Monotone solutions of the master equation}\label{sectionmon} In this section we provide a notion of solution for the master equation which does not require the solution to be differentiable with respect to the space variable. We~define first our solutions in the easier case of the stationary master equation and then present the time dependent case. \subsection{The stationary case} The main idea we exploit in this section is somehow contained in the proof of Proposition~\ref{uniq2}. Namely let us remark that for the proof of Proposition~\ref{uniq2} to hold, we only need to have information on the solution $U$ of \eqref{sme} at points of minima of $\phi_{V,y}: x \mto \langle U(x)-V, x-y\rangle$ for $V \in \mathbb{R}^d, y \in \mathbb{O}_d$. Let us now remark that if $U$ is smooth, so is $\phi_{V,y}$ and \begin{equation*} \nabla_x \phi_{V,y}(x) = U(x)-V+D_x U(x)\cdot(x-y). \end{equation*} In particular, if $x_0$ is a point of minimum of $\phi_{V,y}$ in the interior of $\mathbb{O}_d$, then \begin{equation}\label{eq1} D_x U(x_0)\cdot(x_0-y) =V-U(x_0). \end{equation} The right-hand side of \eqref{eq1} does not depend on derivatives of $U$. This leads us to understand how we can generalize the notion of solution of \eqref{sme} for a function $U$ which is not differentiable. According to this heuristic, we introduce the following definition. \begin{Def}\label{defmons} A function $U \in \mathcal{C}(\mathbb{O}_d, \mathbb{R}^d)$ is said to be a monotone solution of~\eqref{sme} if for any $V \in \mathbb{R}^d$, $y \in \mathbb{O}_d$, $R>0$ sufficiently large and $x_0$ a point of strict minimum of $\phi_{V,y}: x \mto \langle U(x)-V, x-y\rangle$ in $B^1_R$, the following holds \begin{multline}\label{smon} r\langle U(x_0), x_0-y\rangle + \lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle \\ \geq \langle G(x_0, U(x_0)), x_0-y\rangle+\langle F(x_0,U(x_0)), U(x_0)-V\rangle. \end{multline} \end{Def} \begin{Rem} Let us remark that this notion of solution is reminiscent of the definition of viscosity solutions introduced by Crandall and Lions \cite{crandall1983viscosity}. We feel that it is useful to remark that, for the master equation \eqref{sme}, we could have defined a weak solution by the fact that the function $\phi_{V,y}$ was a viscosity super-solution of a certain PDE for all $V \in \mathbb{R}^d$, $y \in \mathbb{O}_d$. However, we are not able to generalize such a formalism to define solutions of the master equation for more general cases such as optimal stopping or impulse control, that is why we prefer the definition of solutions we just presented. \end{Rem} \begin{Rem} Let us note that we impose the minimum to be strict because we have in mind to build a stable notion of solution. Asking information for all minima raises the problem that if a sequence of continuous functions $(\phi_n)_{n \geq 0}$ converges uniformly toward a function $\phi$, then point of minima of $\phi$ are not necessary limits of sequences of point of minima of $(\phi_n)_{n \geq 0}$. This type of phenomenon raises major difficulties to obtain stability, especially in the case of optimal stopping. \end{Rem} \begin{Rem} Let us remark that this notion of solution could be stated in more general domains than $\mathbb{O}_d$, as long as we have a boundary condition of the type of Hypothesis~\ref{stab}. \end{Rem} Let us insist that in the previous definition $x_0$ may be on the boundary of $\mathbb{O}_d$. We~introduce the ball $B^1_R$ because we shall place ourselves under Hypothesis~\ref{grow}, which, as we already mentioned, has the effect to bound the trajectories. Before commenting on our definition of solution, let us state the two following results which justify this choice of definition. \begin{Prop} Under Hypotheses~\ref{stab} and \ref{grow}, a smooth solution of \eqref{sme} is also a monotone solution of \eqref{sme} in the sense of Definition~\ref{defmons}. \end{Prop} \begin{proof} This result is fairly simple so we only sketch its proof here. Let us consider a classical solution $U$ of \eqref{sme}. If the point of strict minimum of $x \mto \langle U(x)-V, x-y \rangle$ is in the interior of $B^1_R$, then thanks to \eqref{eq1}, there is equality in \eqref{smon}. If this point is on the boundary, thanks to the assumptions \ref{stab} and \ref{grow}, the inequality holds. \end{proof} \begin{Theorem}\label{uniqstat} Under Hypotheses~\ref{stab} through \ref{mon}, there exists at most one continuous monotone solution of \eqref{sme} in the sense of Definition~\ref{defmons}. If it exists, it is a monotone application. \end{Theorem} \begin{proof} Let us consider $U$ and $V$ two such solutions. Let us define $W: \mathbb{O}_d^2 \to \mathbb{R}$ by \begin{equation*} W(x,y) = \langle U(x)-V(y), x-y \rangle. \end{equation*} Thanks to Lemma~\ref{stegall}, for any $\epsilon > 0$, there exists $(a,b)\in \mathbb{R}^{2d}$, $|a|+|b| \leq \epsilon$, such that \[ (x,y) \mto \langle U(x)-V(y) , x-y \rangle+\langle a,x\rangle+\langle b, y\rangle \] has a strict minimum on $(B_R^1)^2$ (for $R\!>\!0$ chosen sufficiently large independently~of~$\epsilon$), attained at $(x_0,y_0)$. Thus because $U$ is a monotone solution the following holds. \begin{multline*} r\langle U(x_0), x_0-y_0\rangle+ \lambda\langle U(x_0)-T^*U(Tx_0), x_0-y_0\rangle \\ \geq \langle G(x_0, U(x_0)), x_0-y_0\rangle+\langle F(x_0,U(x_0)), U(x_0)-V(y_0)+a\rangle. \end{multline*} On the other hand, because $V$ is a monotone solution, we deduce that \begin{multline*} r\langle V(y_0), y_0-x_0\rangle+ \lambda \langle V(y_0) -T^*V(Ty_0), y_0-x_0\rangle \\ \geq \langle G(y_0, V(y_0)), y_0-x_0\rangle+\langle F(y_0,V(y_0)), V(y_0)-U(x_0)+b\rangle. \end{multline*} Summing the two previous equations, we obtain \begin{multline*} rW(x_0,y_0) + \lambda (W(x_0,y_0)-W(Tx_0,Ty_0))\\ \geq \langle G(x_0,U(x_0))-G(y_0,V(y_0)),x_0-y_0 \rangle+ \langle F(x_0,U(x_0)),a\rangle+\langle F(y_0,V(y_0)),b\rangle\\ + \langle F(x_0,U(x_0))-F(y_0,V(y_0)),U(x_0)-V(y_0) \rangle. \end{multline*} From this we deduce the following. \[ rW(x_0,y_0) \geq \lambda (\langle a, x_0-Tx_0\rangle+\langle b, y_0-Ty_0\rangle)+\langle F(x_0,U(x_0)),a\rangle+\langle F(y_0,V(y_0)),b\rangle. \] Because $U$ and $V$ are continuous, we deduce from the fact that $\epsilon$ can be chosen arbitrary small, that for any $R> 0$, $W \geq 0$ on $B^1_R$. Thus we conclude as in the proof of Proposition~\ref{uniq2} that $U = V$ and that $U$ is monotone. \end{proof} This previous result is a strong justification for our notion of solution. By considering the proof of this result and the equivalent result in the smooth regime, one can realize that we have simply used all the ingredients useful to prove uniqueness of solutions of the master equation in the monotone regime and used them as a definition of solutions. Let us note that we are not going to prove an existence result for monotone solutions in the stationary setting (only in the time dependent one later on). This is entirely due to the same reasons as in Remark \ref{rembound} and has nothing to do with an intrinsic difficulty associated to this stationary case. One can wonder if the notion of monotone solution is not too weak. In the next section we show how it can be sufficient to describe solutions in the optimal stopping case or in the impulse control one. Moreover we now present results of stability and consistency concerning monotone solutions. \begin{Prop}\label{stabs} Consider a sequence $(F_n,G_n)_{n \in \mathbb{N}}$ of applications from $\mathbb{O}_d\times \mathbb{R}^d$ into $\mathbb{R}^{2d}$ which converges uniformly over all compact toward $(F,G)$. If for all $n$, $U_n$ is a continuous monotone solution of the master equation \eqref{sme} associated to $F_n$ and $G_n$ and if $(U_n)_{n \in \mathbb{N}}$ converges locally uniformly toward $U$, then $U$ is a monotone solution of the master equation associated to $F$ and $G$. \end{Prop} \begin{proof} Let us consider $V \in \mathbb{R}^d$, $y \in \mathbb{O}_d$, $R>0$ and $x_0$ a point of strict minimum of $\phi: x \mto \langle U(x)-V, x-y\rangle$ in $B^1_R$. From Lemma~\ref{stegall}, we can consider a sequence $(a_n)_{n \in \mathbb{N}} \in (\mathbb{R}^d)^{\mathbb{N}}$ such that for all $n$, \begin{equation*} \begin{cases} |a_n| \leq n^{-1},\\ \phi_n: x \mto \langle U_n(x)-V, x-y\rangle+\langle a_n,x\rangle \text{ has a strict minimum $x_n$ in }B^1_R. \end{cases} \end{equation*} Let us now remark that for all $n \geq 0$: \begin{equation*} \langle U_n(x_n)- V, x_n-y\rangle+\langle a_n, x_n\rangle \leq \langle U_n(x_0)-V, x_0-y\rangle+\langle a_n,x_0\rangle. \end{equation*} From this last inequality, we deduce that $(x_n)_{n \in \mathbb{N}}$ converges toward $x_0$. Finally let us remark that because for all $n \geq 0$, $U_n$ is a monotone solution, we can write \begin{multline*} r\langle U_n(x_n), x_n-y\rangle + \lambda \langle U_n(x_n)-T^*U_n(Tx_n),x_n-y \rangle \\ \geq \langle G_n(x_n, U_n(x_n)), x_n-y\rangle+\langle F_n(x_n,U_n(x_n)), U_n(x_n)-V+a_n\rangle. \end{multline*} Passing to the limit in this last expression yields the required result. \end{proof} \begin{Rem} The same type of results can be obtained in the case in which one seeks stability for the terms $\lambda$ and $T$ in \eqref{sme}. This can be achieved by changing mildly the previous proof. \end{Rem} We now show consistency of this notion of solution under an additional monotonicity assumption. That is we show that if a smooth function $U$ is a monotone solution of \eqref{sme} in the sense of Definition~\ref{defmons} and that it satisfies an additional monotonicity assumption, then it is a classical solution of \eqref{sme}. \begin{Prop}\label{consist} Assume that $U\in W^{2,\infty}$ is a monotone solution of \eqref{sme} in the sense of Definition~\ref{defmons}. Assume furthermore that for all $x \in \mathbb{O}_d$, $D_x U(x) > 0$ in the order of positive definite matrix. Then $U$ satisfies \begin{equation}\label{35} \begin{split} r U^i+(F(x,U)\cdot \nabla_x)U^i+\lambda \left(U^i-(T^*U(t,Tx))^i\right)&= G^i(x,U) \quad\text{in } \{x_i > 0\},\\ r U^i+(F(x,U)\cdot \nabla_x)U^i+\lambda \left(U^i-(T^*U(t,Tx))^i\right)&\leq G^i(x,U) \quad\text{in } \{x_i =0\}. \end{split} \end{equation} \end{Prop} \begin{proof} Let us fix $x_0$ in the interior of $\mathbb{O}_d$ such that $U$ is twice differentiable at $x_0$ (such a set is dense in $\mathbb{O}_d$). Let us define $\phi_{V,y}$ as in Definition~\ref{defmons}. Let us remark that \begin{align}\label{gradphi} \nabla_x \phi_{V,y}(x_0) &= D_x U(x_0) (x_0-y)+U(x_0)-V,\\\label{hessphi} D^2_x \phi_{V,y}(x_0) &= 2 D_x U(x_0)+D^2_xU(x_0)(x_0-y). \end{align} Let us note that from the assumption on the monotonicity of $U$, there is a neighborhood $\mathcal{O}$ of $x_0$ such that for any point $y \in \mathcal{O}$, the right-hand side of \eqref{hessphi} is non-negative. Then, taking such a $y$ and choosing $V$ such that \eqref{gradphi} vanishes, we have that $x_0$ is a point of strict minimum of $\phi_{V,y}$. Because $U$ is a monotone solution of~\eqref{sme}, we deduce that \begin{multline*} r\langle U(x_0), x_0-y\rangle+\langle F(x_0,U(x_0)), D_x U(x_0)(x_0-y)\\ + \lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle \geq \langle G(x_0, U(x_0)), x_0-y\rangle. \end{multline*} This last inequality holds for any $y \in \mathcal{O}$. From this we easily deduce that $U$ satisfies~\eqref{35} at $x_0$. We argue in the same way when $x_0$ is on the boundary of $\mathbb{O}_d$. \end{proof} \begin{Rem} Let us remark that the assumption $D_x U(x) > 0$ in the order of positive definite matrix on $\mathbb{O}_d$ is usually verified when some strict monotonicity assumption is made on $(G,F)$. Furthermore, even though this assumption may be weakened, it does not seem to be purely technical. Indeed, as we shall see in the section on optimal stopping, the monotone solution solution of a master equation may not solve the associated PDE at every point (for instance it is the case at the boundary of~$\mathbb{O}_d$). In our opinion, and very formally, the points at which this type of behavior can occur are points at which the evolution of the density of players is not well-defined, for instance because there are no players or because several Nash equilibria can lead to the same value. \end{Rem} \begin{Rem} As it is usually the case in the MFG theory, the value function does not necessary satisfy the master equation on states where there is no player, but only an inequality. This is reminiscent of the weak solutions studied in \cite{cardaliaguet2015second} for instance. \end{Rem} \subsection{The time dependent case} In this section we present the analogue of Definition~\ref{defmons} for the case of \eqref{me}. Let us note that in general in the MFG theory, the time regularity is not necessary the main challenge and that we could easily define a notion of monotone solution for smooth function of the time. However we prefer, for completeness, to present this concept for functions which are not necessary smooth in time, even though it makes this section more technical. Following the previous part, we define a monotone solution in the time dependent setting with the following: \begin{Def}\label{defms} A continuous function $U:(0,\infty)\!\times\!\mathbb{O}_d\!\to\! \mathbb{R}^d$ is a monotone solution of \eqref{me} if \begin{itemize} \item for any $V \in \mathbb{R}^d$, $y\!\in\!\mathbb{O}_d$, $R\!>\! 0$ sufficiently large $\epsilon\!>\!0$ and $\phi: \mathbb{R}\!\to\!\mathbb{R}$ a~smooth function, for any $(t_0,x_0)\in (0,\infty) \times B^1_R$ point of strict minimum of \[ (t,x) \mto \langle U(t,x)-V, x-y\rangle-\phi(t) \] on $(t_0-\epsilon, t_0] \times B^1_R$, the following holds: \begin{multline}\label{testt} \frac{d}{dt}\, \phi(t_0) + \lambda \langle U(t_0,x_0)-T^*U(t_0,Tx_0),x_0-y \rangle \\ \geq \langle F(x,U(t_0,x)), U(t_0,x)-V\rangle+\langle G(x,U(t_0,x)), x-y \rangle. \end{multline} \item The initial condition holds: \begin{equation*} U(0,x) = U_0(x) \quad\text{on } \mathbb{O}_d. \end{equation*} \end{itemize} \end{Def} In some sense, we are treating the time derivative using techniques from viscosity solutions. As in the stationary case, we now present results of uniqueness, stability and consistency for this notion of solution. Let us insist on the fact that the following Proposition also yields an existence result. \begin{Prop} Let $U: (0,\infty)\!\times\!\mathbb{O}_d\!\to\!\mathbb{R}^d$ be a smooth function, solution of~\eqref{me} in~the classical sense. Then it is a monotone solution of \eqref{me} in the sense of definition~\ref{defms}. Moreover, under the assumptions of Proposition~\ref{existme}, there exists a monotone solution of \eqref{me}. \end{Prop} \begin{proof} We only state that for any $x,y \in \mathbb{O}_d$ and $t_0 > 0$, for any $\phi$ such that $\phi(t) < \langle U(t,x), x-y \rangle$ for $t \in (t_0-\epsilon, t_0)$ for some $\epsilon > 0$ with $\phi(t_0) = \langle U(t_0,x), x-y \rangle$, then one has $\frac{d}{dt}\phi(t_0) \geq \langle \partial_t U(t_0,x),x-y\rangle$. The rest of the proof follows easily from the proof of Proposition \ref{existme}. \end{proof} \begin{Theorem}\label{uniqt} Under Hypotheses~\ref{stab} and \ref{mon}, there exists at most one continuous monotone solution of \eqref{me} in the sense of Definition~\ref{defms}. \end{Theorem} \begin{proof} Let us denote by $U$ and $V$ two such solutions. We define \hbox{$W: [0,\infty)\!\times\!\mathbb{O}_d^2\!\to\!\mathbb{R}$} by \begin{equation*} W(t,x,y) = \langle U(t,x)-V(t,y), x- y \rangle. \end{equation*} Our aim is to proceed as usual by proving that $W \geq 0$. Let us assume that there exists $(t_0,x_0,y_0)$ such that $\kappa:= W(t_0,x_0,y_0) < 0$. Let us define the function \hbox{$Z: [0,t_0]^2 \times \mathbb{O}_d^2 \to \mathbb{R}$} by \begin{equation*} Z(t,s,x,y) = \langle U(t,x)-V(s,y),x-y \rangle+\frac{1}{\alpha}(t-s)^2+\langle a , x\rangle+\langle b,y\rangle+\delta_1 t+\delta_2 s. \end{equation*} This function is well-defined and depends on the parameters $\alpha,\delta_1,\delta_2\!>\!0$ and $a,b\!\in\!\mathbb{R}^d$. Let $R > 0$ be such that $x_0,y_0 \in B^1_R$. Evaluating $Z$ at $(t_0,t_0,x_0,y_0)$, we deduce that \[ \min_{[0,t_0]^2\times (B^1_R)^2} Z \leq \kappa+\langle a , x_0\rangle+\langle b,y_0\rangle+(\delta_1+\delta_2) t_0. \] Hence, there exists $\delta_0 > 0$ and $\epsilon > 0$ such that if $|a|+|b| < \epsilon$ and $\delta_1,\delta_2 \in(\sfrac{\delta_0}{2}, \delta_0)$, then \begin{equation}\label{hypwrong} \min_{[0,t_0]\times (B^1_R)^2} Z < -\frac{\kappa}{2}. \end{equation} From Lemma~\ref{stegall}, there exists $a,b,\delta_1, \delta_2$ satisfying $|a|+|b| < \epsilon$ and $\delta_1,\delta_2 \in(\sfrac{\delta_0}{2}, \delta_0)$ such that $Z$ has a strict minimum on $[0,t_0]^2 \times (B^1_R)^2$ at $(t_*,s_*,x_*,y_*)$. If $t_*> 0$, then \begin{equation*} \phi_1: t \to-\frac{1}{\alpha}(t-s_*)^2-\delta_1 t \end{equation*} can be chosen as a test function in \eqref{testt} for $U$. Using that $U$ is a monotone solution, we then obtain \begin{multline*} - \delta_1- \frac{2}{\alpha}(t_*-s_*)+\lambda \langle U(t_*,x_*)-T^*U(t_*,Tx_*),x_*-y_* \rangle \\ \geq \langle F(x_*,U(t_*,x_*)), U(t_*,x_*)-V(s_*,y_*)+a\rangle+\langle G(x_*,U(t_*,x_*)), x_*-y_* \rangle. \end{multline*} If $s_* > 0$ we can construct the analogue function $\phi_2$ for $V$ and then obtain an analogue relation as the previous one. Thus if both $t_* > 0 $ and $s_* > 0$, one obtains \begin{multline}\label{int47} - \delta_1 - \delta_2\\ +\lambda \bigl(\langle U(t_*,x_*)-V(s_*,y_*), x_* -y_*\rangle-\langle U(t_*,Tx_*)-V(s_*,Ty_*), T(x_*-y_*)\rangle\bigr) \\ \geq \langle F(x_*,U(t_*,x_*)),a\rangle+\langle F(y_*,V(s_*,y_*)),b\rangle. \end{multline} By construction of $(t_*,s_*,x_*,y_*)$, the following holds \begin{multline*} \langle U(t_*,x_*)-V(s_*,y_*), x_* -y_*\rangle \leq \langle U(t_*,Tx_*)-V(s_*,Ty_*), T(x_*-y_*)\rangle \\ + (\langle a, Tx_* -x_*\rangle+\langle b, Ty_* -y_*\rangle). \end{multline*} Hence we deduce that \begin{equation*} -\delta_0 \geq \langle F(x_*,U(t_*,x_*))+\lambda(x_*-Tx_*),a\rangle+\langle F(y_*,V(s_*,y_*))+\lambda(y_*-Ty_*),b\rangle, \end{equation*} where we have also used that $\delta_1,\delta_2 > \delta_0/2$. Let us note that a posteriori, choosing $\epsilon$ as small as we want, so that $a$ and $b$ are as small as necessary we can contradict \eqref{hypwrong} as $\delta_0 > 0$ and $\delta_0$ was chosen independently of $\epsilon$. Thus we have proved that if~$\delta_1$ and~$\delta_2$ are chosen in $(\sfrac{\delta_0}{2}, \delta_0)$, and $a$ and $b$ are sufficiently small, for any value of $\alpha > 0$, the point of strict minimum of $Z$ cannot be attained for $t_* > 0$ and $s_* > 0$. We now treat the case in which $t_* = 0$ (the case $s_* = 0$ is similar) and show that we also arrive at a contradiction if the parameter $\alpha$ is well chosen. Let us take $\eta > 0$ arbitrary small. We want to show that $Z(t_*,s_*,x_*,y_*) \geq-\eta$ for a certain choice of~$\alpha$. The following estimate always holds \begin{equation*} |t_*-s_*|< C \sqrt{\alpha}, \end{equation*} for some constant $C$ (let us recall that we minimize the continuous function $Z$ on a compact set). Choosing $\alpha$ sufficiently small, we can assume that if $t_* = 0$, then $s_* \leq \eta'$ for $\eta'>0$ arbitrary small. Thus using the continuity of $V$, we deduce that $|V(s_*,y_*)-U_0(y_*)|\leq \eta''$ for $\eta''>0$ arbitrary small, up to taking $\eta'$ small enough. From which we deduce using the continuity of $U$ (if $\eta''$ is small enough compared to~$\eta$, which we can always assume) that $Z(t_*,s_*,x_*,y_*) \geq-\eta$ which contradicts \eqref{hypwrong} since~$\eta$ is as small as we want. Thus the function $W$ is non-negative and we conclude as usual. \end{proof} \begin{Prop}\label{propstab} Consider a sequence $(F_n,G_n)_{n \in \mathbb{N}}$ of applications from $\mathbb{O}_d\times \mathbb{R}^d$ into $\mathbb{R}^{2d}$ which converges uniformly over all compact toward $(F,G)$. If for all $n$, $U_n$~is a continuous monotone solution of the master equation associated to $F_n$ and $G_n$ and if $(U_n)_{n \in \mathbb{N}}$ converges uniformly toward $U$, then $U$ is a monotone solution of the master equation associated to $F$ and $G$. \end{Prop} \begin{proof} The proof is very similar to the one of the stationary case. We consider \hbox{$\epsilon\!>\!0$}, $R >0$, $V \in \mathbb{R}^d$, $y \in \mathbb{O}_d$, $t_0 > 0$, $x_0 \in \mathbb{O}_d$ and a smooth function \hbox{$\phi: [t_0-\epsilon, t_0] \to \mathbb{R}$}, such that $(t_0,x_0)$ is a point of strict minimum of $(t,x) \mto \langle U(t,x)-V, x-y\rangle-\phi(t)$ on $(t_0-\epsilon, t_0] \times B^1_R$. In view of Lemma \ref{stegall}, we can consider for all $n \geq 0$, $\delta_n \in [-n^{-1},n^{-1}]$ and $a_n \in B^1_{n^{-1}}$ such that $(t,x) \mto \langle U_n(t,x)-V-a_n, x-y\rangle-\phi(t)+\delta_n t$ has a strict minimum on $[t_0-\epsilon, t_0] \times B^1_R$ at $(t_n,x_n)$. Arguing as in the stationary case, we can pass to the limit $n \to \infty$ in the relation we obtain from the fact that $U_n$ is a monotone solution of the equation. This yields the required relation for $U$ and thus established that $U$ is a monotone solution of the problem. \end{proof} \subsection{A generalization of this method} As already mentioned above, the aim of this paper is to present a new notion of solution for MFG master equations and not necessary to enter into too much details on these solutions. However we believe the next remark to be worth mentioning. It has been pointed out to us by Pierre-Louis Lions. Let us consider the case $\lambda = 0$. The main argument to establish uniqueness of monotone solutions is to consider two such solutions $U$ and $V$ and to prove that $W$ defined by \begin{equation*} W(x,y) = \langle U(x)-V(y), x-y\rangle \end{equation*} is non-negative. Instead, for instance, we could have defined the function $W$ with \begin{equation*} W(x,y) = \langle U(x)-V(y), \phi(x)-\phi(y)\rangle \end{equation*} for some $\phi: \mathbb{O}_d \to \mathbb{R}^d$. Now let us remark that if $D_x\phi(x)$ is an invertible matrix for any $x$ in the interior of $\mathbb{O}_d$, the property \begin{equation*} W \geq 0 \implies U = V \end{equation*} still holds. This remark immediately generalizes the result of this section to a wider class of systems. Indeed by replacing the condition $(G,F)$ is monotone by \begin{multline*} \langle G(x,U)-G(y,V), \phi(x)-\phi(y) \rangle+\langle F(x,U)-F(y,V), U\cdot D_x \phi(x)-V\cdot D_x \phi(y) \rangle \geq 0\\ \forall x,y \in \mathbb{O}_d,\ U,V \in \mathbb{R}^d, \end{multline*} we obtain the uniqueness of the associated monotone solutions. This concept of solution depending on $\phi$ can be defined (in the stationary case) by \begin{Def}\label{defmongen} A function $U \in \mathcal{C}(\mathbb{O}_d, \mathbb{R}^d)$ is said to be a $\phi$-monotone solution of~\eqref{sme} if for any $V \in \mathbb{R}^d, y \in \phi(\mathbb{O}_d)$, $R>0$ sufficiently large and $x_0$ a point of strict minimum of $\psi_{V,y}: x \mto \langle U(x)-V, \phi(x)-y\rangle$ in $B^1_R$, the following holds \[ r\langle U(x_0), \phi(x_0)-y\rangle \geq \langle G(x_0, U(x_0)), x_0-y\rangle+\langle F(x_0,U(x_0)), (U(x_0)- V)D_x\phi(x_0) \rangle. \] \end{Def} For instance, a remarkable choice of $\phi$ could be $x \mto-x$. In such a situation, one could study master equations with a sort of anti-monotonicity properties. \section{The master equation for mean field games with optimal stopping}\label{sectstop} This section introduces the formulation of the master equation modeling a MFG in which the players have the possibility to leave the game. As already mentioned in the introduction, let us recall that MFG of optimal stopping have been the subject of several works but that the case of the master equation for such MFG has not been treated up to now. In this section we are interested with a MFG, similar to the ones represented by~\eqref{me} and~\eqref{sme}, except for the fact that the players are allowed to leave the game whenever they decide, by paying a certain exit cost. Moreover, once they have left the game, they do not interact anymore with the other players. Let us briefly recall some results from \cite{bertucci2018optimal} on this kind of MFG. In this paper, we shew that existence of Nash equilibria for MFG of optimal stopping (in the absence of a common noise) holds in general only if the players are allowed to play mixed strategies. That is, the players are allowed to play a random exit time (\ie they control their probability to leave the game). This leads to the existence of Nash equilibria in which there are part of the state space where some players are leaving and some are staying. Even though uniqueness of Nash equilibria (or at least of the well posedness of a value function for the MFG) can be established through standard monotonicity assumptions, the characterization of the evolution is highly non trivial in the states in which some players are leaving and some are staying. We propose here to address this problem using the notion of monotone solutions, proceeding as we did in \cite{bertucci2018optimal}, by first considering a penalization of the MFG in which the players cannot exit the game as freely as they may want, and then by passing to the limit. \subsection{The penalized master equation} In the penalized version of the MFG of optimal stopping, the players cannot decide to leave instantly the game, they can only control the intensity of a Poisson process which give their exit time, and the intensity of this process is bounded by $\epsilon^{-1}$ for $\epsilon > 0$. Even though we do not want to enter into the precise formulation of this penalized game, let us insist on the fact that those aforementioned Poisson processes are supposed to be independent from one player to the other. The penalized master equation is then of the following form in the time dependent setting: \begin{multline}\label{stopme} \partial_t U+\frac{1}{\epsilon}\,\beta(U)+\Bigl(\frac{1}{\epsilon}\,\beta'(U)*x+F(x,U)\Bigr)\cdot \nabla_xU\\ \shoveright{+ \lambda (U-T^*U(t,Tx))= G(x,U) \quad\text{in } (0,\infty) \times \mathbb{O}_d,}\\ \shoveleft{U(0,x) = U_0(x) \quad\text{in } \mathbb{O}_d,\hfill} \end{multline} where $\beta$ is an increasing and convex function that we would like to take as $\beta = \hat{\beta}:= (\cdot)_+$. We recall that $*$ stands for the term by term product and that $\beta(U)$ is understood component wise. Because $\hat{\beta}'$ is not well-defined but we shall shall work with smooth $\beta$ approximating $\hat{\beta}$. However we shall focus first on the case of $\hat{\beta}$ for the sake of argument. In the stationary setting, the form of the penalized master equation is given by \begin{multline}\label{stopsme1} r U+\frac{1}{\epsilon}\,\beta(U)+\Bigl(\frac{1}{\epsilon}\,\beta'(U)*x+F(x,U)\Bigr)\cdot \nabla_xU\\ +\lambda (U-T^*U(t,Tx))= G(x,U) \quad\text{in } \mathbb{O}_d. \end{multline} In the case $\lambda = 0$, the form of the characteristics of the previous equations is given~by% \begin{equation}\label{charstop} \begin{cases} \dot{V}(t) = G(y(t),V(t))-\dfrac{1}{\epsilon}\,\beta(V),\\[5pt] \dot{y}(t) = F(y(t),V(t))+\dfrac{1}{\epsilon}\,\beta'(V)*y. \end{cases} \end{equation} This type of characteristics is clearly what we expect from the study in \cite{bertucci2018optimal} when $\beta = \hat{\beta}$. Let us now come back on the case of $\hat{\beta}$, which is not well-defined. In this paper, we want to understand this derivative in the same manner as we did in \cite{bertucci2018optimal}. This technique relies mainly on understanding $\hat{\beta}'(0)$ as the segment $[0,1]$ and the previous master equations as differential inclusions. Instead of looking for solutions~$U$ of \eqref{stopsme}, we are going to look for a couple $(U,\alpha)$ solution of \begin{equation}\label{stopsme} \begin{cases} r U+\dfrac{1}{\epsilon}\,\hat{\beta}(U)+\Bigl(\dfrac{\alpha}{\epsilon}*x+F(x,U)\Bigr)\cdot \nabla_xU+\lambda (U-T^*U(t,Tx))\\ \hspace*{2.2cm}= G(x,U) \text{ in } \mathbb{O}_d,\quad 0 \leq \alpha^i(x) \leq 1,\ \forall i,\, 1 \leq i \leq d,\ x \in \mathbb{O}_d,\\ U^i(x) < 0 \implies \alpha^i(x) = 0,\\ U^i(x) > 0 \implies \alpha^i(x) = 1. \end{cases} \end{equation} The same type of reformulation can be carried on for the time dependent case. In the previous master equations, the exit cost paid by the players to leave the game is $0$. This leaving cost is necessary an upper bound for any candidate for a value function of the game, as a player can always pay the exit cost and leave the game. Thus we naturally expect that the value function $U$ satisfies $U \leq 0$ everywhere. Moreover, seeing $U \leq 0$ as a constraint helps to understand the function $\hat{\beta}$ as a sort of penalization. The case in which the exit cost depends on the state of the player which is leaving the game, but not on the density of all the players, can easily be treated in a similar fashion. The case in which the exit cost depends on the density of players is much more involved as structural assumptions have to be made on the form of the exit cost to ensure the propagation of monotonicity. We refer to \cite{bertucci2018optimal} for more detailed on this topic and we leave this case for future research. For the rest of this study we focus on the stationary case and we mention the time dependent setting at the end of this part on optimal stopping. \subsection{Results on the stationary penalized equations} This section presents three, quite simple, results on the penalized master equations. We start by proving a uniqueness result for solutions of \eqref{stopsme}, which justify in our opinion the well posedness of the penalized MFG, even though this property would not prove to be essential to build a solution of the limit master equation. We do not present in full details results of existence and uniqueness of solutions of \eqref{stopsme}, as it is quite technical and not strictly needed to construct a solution of the limit problem. The second result is a, uniform in $\epsilon$, a priori estimate for the solution of \eqref{stopsme1}. The third result is concerned with monotone solutions of \eqref{stopsme1}. \begin{Prop} Under Hypotheses~\ref{stab} and \ref{grow}, there exists at most one smooth function $U$ such that $(U,\alpha)$ is a solution of \eqref{stopsme}. If it exists, this solution is monotone. \end{Prop} \begin{proof} The proof is very similar to the one of Proposition~\ref{uniq2}. Let us take $(U,\alpha_1)$ and $(V,\alpha_2)$ two smooth solutions. We introduce $W: \mathbb{O}_d^2 \to \mathbb{R}$ defined by: \begin{equation*} W(x,y) = \langle U(x)-V(y), x- y\rangle. \end{equation*} Let us remark that $W$ satisfies \begin{equation*} \begin{aligned} r W&+\Bigl(\frac{\alpha_1(x)}{\epsilon} *x+F(x,U(x))\Bigr)\cdot \nabla_x W+\Bigl(\frac{\alpha_2(y)}{\epsilon}*y+F(y,V(y))\Bigr)\cdot \nabla_y W\\ &=\langle G(x,U(x))-G(y,V(y)),x-y\rangle+\langle F(x,U(x))-F(y,V(y)), x-y\rangle\\ &\hspace*{1cm}+ \frac{1}{\epsilon} \bigl(\langle \alpha_1(x)*x-\alpha_2(y)*y, U(x)-V(y)\rangle-\langle \hat{\beta}(U(x))-\hat{\beta}(V(y)), x-y \rangle \bigr)\\ &= \langle G(x,U(x))-G(y,V(y)),x-y\rangle+\langle F(x,U(x))-F(y,V(y)), x-y\rangle\\ &\hspace*{1cm}+ \frac{1}{\epsilon} \bigl(\langle x, \hat{\beta}(V(y))-\alpha_1(x)V(y)\rangle+\langle y, \hat{\beta}(U(x))-\alpha_2(y)U(x)\rangle \bigr)\\ &\geq \langle G(x,U(x))-G(y,V(y)),x-y\rangle+\langle F(x,U(x))-F(y,V(y)), x-y\rangle. \end{aligned} \end{equation*} The rest of the proof follows as the one of Proposition~\ref{uniq2}. \end{proof} \begin{Rem} Several conclusions can be drawn from this result. The first one is that the terms arising from the modeling of optimal stopping only reinforce the monotonicity of the equation. The second one is that the singularity $\hat{\beta}'(\cdot)$ can be understood quite freely, this is justified by this uniqueness property. Finally, it is not clear that such a uniqueness result holds for any increasing convex function $\beta$. \end{Rem} \begin{Rem} Let us briefly insist on the fact that the addition of the term \hbox{``$\hat{\beta}'(U)*x$''} in the dynamics does not alter the assumptions \ref{stab} and \ref{grow}. \end{Rem} The following result is the main argument why we are able to pass to the limit $\epsilon \to 0$ in \eqref{stopsme}. \begin{Prop}\label{estimatestop} Under the assumptions of Proposition~\ref{existstat}, for any smooth, increasing and convex function $\beta$, there exists $C > 0$ independent of $ \epsilon$ and $\beta$ such that if $U$ is a classical solution of \eqref{stopsme1}, then \begin{equation}\label{estint} \|\nabla_x U\| \leq C. \end{equation} \end{Prop} \begin{proof} This proof is similar to the one of Proposition~\ref{existstat}. Let us define $W$ and~$Z$~by \begin{align*} W(x,\xi) &= \langle U(x),\xi\rangle,\\ Z(x,\xi) &= \langle \nabla_x W (x, \xi), \xi\rangle-\delta |\nabla_x W|^2+\gamma |\xi|^2, \end{align*} for some constants $\beta$ and $\gamma$. Let us remark that, arguing as if $\beta$ was a smooth function, $Z$ satisfies\vspace*{-5pt} \begin{multline}\label{Zedppenal} r Z+\langle \epsilon^{-1}\beta'(U)*x+F(x,\nabla_{\xi} W), \nabla_x Z\rangle\\ \shoveright{ +\langle (\epsilon^{-1}\beta''(U)*x+D_p F (x,\nabla_{\xi} W)) \nabla_{\xi}Z , \nabla_{x} W\rangle}\\ \shoveright{- \langle D_p G(x, \nabla_{\xi}W)\nabla_{\xi}Z, \xi\rangle+\lambda(Z- Z(t,Tx, T\xi))}\\ = \langle D_xG(x, \nabla_x W) \xi, \xi\rangle-\langle D_pG(x, \nabla_{\xi}W) \nabla_x W, \xi\rangle-\langle D_x F(x, \nabla_{\xi} W) \nabla_x W, \xi\rangle \\ \shoveright{+ \langle D_p F(x, \nabla_{\xi}W) \nabla_x W, \nabla_x W\rangle+\langle \epsilon^{-1} \beta''(U)*x*\nabla_x W, \nabla_x W\rangle}\\ \shoveright{- 2 \delta \langle D_x G(x, \nabla_x W)\xi, \nabla_x W\rangle +2 \delta \langle D_x F(x, \nabla_{\xi} W) \nabla_x W, \nabla_x W\rangle}\\ \shoveright{+2 \delta \langle \beta'(U)*\nabla_x W, \nabla_x W\rangle}\\ \shoveright{+\delta \lambda \bigl(|\nabla_x W|^2-2 \langle \nabla_x W(Tx, T \xi), T \nabla_x W\rangle+| \nabla_x W(Tx, T \xi)|^2 \bigr)} \\ \shoveright{+r \delta |\nabla_x W|^2+2 \gamma(\langle D_p F (x,\nabla_{\xi} W) \xi , \nabla_{x} W\rangle-\langle D_p G(x, \nabla_{\xi}W)\xi, \xi\rangle)}\\ -r\gamma |\xi|^2+\lambda \gamma (|\xi|^2-|T\xi|^2). \end{multline} Let us remark that, since $\beta$ is an increasing and convex function, all the terms in the right-hand side involving $\epsilon$ are non-negative. Thus we conclude that the announced a priori estimate holds (indeed independently of $\epsilon$). \end{proof} As already mentioned in the previous section, existence of solution of a stationary master equation can be established from estimates such as \eqref{estint} if some local boundedness holds. We here give an assumption for which such a property can be proved. \begin{hyp}\label{existstop} For any $p\in \mathbb{R}^d$, $F(0,p) = 0$. Moreover there is a unique solution $V \in \mathbb{R}^d$, $V \leq 0$ of\vspace*{-5pt} \begin{equation*} rV+\lambda (V-T^*V) = -G(0,-V). \end{equation*} \end{hyp} We believe this assumption to be quite mild as it only assumes that i) once the mass of players reaches $0$, it stays at $0$, ii) the MFG with a $0$ mass of player (which is thus an optimal control problem) is well-defined and players remaining do not exit~it. Let us remark that there can indeed still be player in the MFG, even if the mass of players is $0$. In such a case, the remaining players do not ``see'' each other. \begin{Prop}\label{resume} For $\epsilon > 0$ and $\beta$ a smooth increasing and convex function such that $\beta =0 $ on $\{x \leq 0\}$, under the assumptions of Proposition~\ref{existstat} and Hypothesis~\ref{existstop}, there exists a monotone solution $U_{\epsilon,\beta}$ of \eqref{stopsme1}, Lipschitz continuous, with a Lipschitz constant independent of $\epsilon$ and $\beta$, and such that $U_{\epsilon,\beta}(0)$ is bounded uniformly in $\epsilon$ and~$\beta$. \end{Prop} The proof of this result follows exactly the argument of the previous part. \subsection{The limit master equation} We show in this section how we can characterize the value function of a MFG of optimal stopping using the notion of monotone solutions. The main idea consists in characterizing the limit of the sequence $(U_{\epsilon})_{\epsilon > 0}$ of solutions of \eqref{stopsme} when $\epsilon \to 0$. Let us take $(U_{\epsilon},\alpha_{\epsilon})$ such that $U_{\epsilon}$ is a monotone solution of \eqref{stopsme}, for any $V \in \mathbb{R}^d$, $y \in \mathbb{O}_d$ and $x$ point of strict local minimum of $\phi_{V,y}: x \mto \langle U_{\epsilon}(x)-V, x-y\rangle$, the following holds. \begin{equation*} \begin{aligned} r \langle &U_{\epsilon} (x), x-y \rangle+\lambda \langle U_{\epsilon}(x)-T^*U_{\epsilon}(Tx),x-y\rangle\\ &\geq\langle G(x,U_{\epsilon}(x)), x- y \rangle+\langle F(x,U_{\epsilon}(x)), U_{\epsilon}(x)-V \rangle\\ &\hspace*{3.5cm}+ \frac{1}{\epsilon}\bigl(\langle \alpha_{\epsilon}*x, U_{\epsilon}(x)-V\rangle-\langle \hat{\beta}(U_{\epsilon}(x)), x-y \rangle \bigr)\\ &= \langle G(x,U_{\epsilon}(x)), x- y \rangle\!+\!\langle F(x,U_{\epsilon}(x)), U_{\epsilon}(x)-V \rangle- \frac{1}{\epsilon}\bigl(\langle \alpha_{\epsilon}*x, V\rangle\!+\!\langle \hat{\beta}(U_{\epsilon}(x)), y \rangle \bigr). \end{aligned} \end{equation*} From this computation, we deduce that if $V \in \mathbb{R}^d$ is such that $V \leq 0$, then for any $y \in \mathbb{O}_d$ and $x$ point of strict local minimum of $\phi_{V,y}: x \mto \langle U_{\epsilon}(x)-V, x-y\rangle$, we obtain that \begin{multline}\label{remstop} r \langle U_{\epsilon} (x), x-y \rangle+\lambda \langle U_{\epsilon}(x)-T^*U_{\epsilon}(Tx),x-y\rangle \\ \geq \langle G(x,U_{\epsilon}(x)), x- y \rangle+\langle F(x,U_{\epsilon}(x)), U_{\epsilon}(x)-V \rangle. \end{multline} As we clearly expect that the limit of $(U_{\epsilon})_{\epsilon> 0} $ (if it exists) is non-positive, this leads us to the following definition. \begin{Def}\label{defmonstop} A function $U \in \mathcal{C}(\mathbb{O}_d, \mathbb{R}^d)$ is said to be a monotone solution of the master equation for the MFG of optimal stopping if \begin{itemize} \item $U \leq 0$, \item for any $V \in \mathbb{R}^d$ such that $V \leq 0$, for any $y \in \mathbb{O}_d$, $R>0$ sufficiently large and $x_0$ a point of strict minimum of $\phi_{V,y}: x \mto \langle U(x)-V, x-y\rangle$ in $B^1_R$, the following holds \begin{multline*} r \langle U (x_0), x_0-y \rangle+\lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle \\ \geq \langle G(x_0,U(x_0)), x_0- y \rangle+\langle F(x_0,U(x_0)), U(x_0)-V \rangle. \end{multline*} \end{itemize} \end{Def} Let us insist that the only thing which differs from the non-optimal stopping case is that $U$ has to be non-positive component-wise and that we only have information for $V$ which are also non-positive component wise. The existence of such a solution is stated in the next result. \begin{Theorem} Under the assumptions of Proposition~\ref{existstat} and Hypothesis~\ref{existstop}, there exists a monotone solution $U$ of the MFG of optimal stopping in the sense of Definition~\ref{defmonstop}. \end{Theorem} \begin{proof} Let us take a sequence of functions $(\beta_n)_{n \geq 0}$ such that for all $n \geq 0$, $\beta_n$~is a smooth increasing, convex and non negative function such that $\beta_n(0)= 0$ and $(\beta_n)_{n \geq 0}$ converges locally uniformly toward $\hat{\beta}$. For $\epsilon > 0$ we denote by $U_{\epsilon,n}$ a monotone solution of \eqref{stopsme1} with $\beta = \beta_n$. From Proposition~\ref{resume}, we know that, $(U_{\epsilon,n})_{\epsilon> 0, n \geq 0}$ is uniformly continuous and locally bounded. From the Ascoli-Arzelà theorem, we can build two sequences $(U_p)_{p \geq 0}$ and $(\epsilon_p)_{p \geq 0}$ such that: \begin{itemize} \item For all $p \geq 0$, $\epsilon_p > 0$ and $\epsilon_p \to 0$ as $p \to \infty$. \item For all $p \geq 0$, $(U_{\epsilon_p,n})_{n \geq 0}$ converges locally uniformly toward $U_p$. \item $(U_p)_{p \geq 0}$ is a family of uniformly Lipschitz functions, locally bounded, and converges locally uniformly toward a function $U$. \end{itemize} Thanks to Proposition \ref{stabs}, recalling \eqref{remstop}, we deduce that for all $p \geq 0$, $V \in \mathbb{R}^d$, $V \leq 0$, $y \in \mathbb{O}_d$ and $x_0$ point of strict minimum of $x \mto \langle U_p(x)-V, x-y\rangle$, the following holds \begin{multline*} r \langle U_p(x_0), x_0-y \rangle+\lambda \langle U_p(x_0)-T^*U(Tx_0),x_0-y\rangle \\ \geq \langle G(x_0,U_p(x_0)), x_0- y \rangle+\langle F(x_0,U_p(x_0)), U_p(x_0)-V \rangle. \end{multline*} We do not detail this fact as it follows exactly the same argument as in the proof of Proposition \ref{stabs} except for the fact that one has to be careful to choose $a_n$ (using the notation of the aforementioned proof) so that $V-a_n \leq 0$, which we can do due to Stegall's variational principle. The same property is of course also obtained immediately for $U$ the limit of $(U_p)_{p \geq 0}$. Thus it remains only to show that $U \leq 0$. Let us take $R > 0$ sufficiently large $\epsilon > 0$ and $n \geq 0$. Consider $D:= [R+1,R+2]^d$. There exists $y \in D$ and $a \in \mathbb{R}^d, a \leq 0$ arbitrary small such that $x \mto \langle U_{\epsilon,n}(x) -a, x -y\rangle$ has a strict minimum on $B^1_R$ at $x_0$. Since $U_{\epsilon,n}$ is a monotone solution of \eqref{stopsme1}, we deduce that \begin{multline*} r\langle U_{\epsilon,n}(x_0),x_0-y\rangle+\lambda \langle U_{\epsilon,n}(x_0)-T^*U_{\epsilon,n}(Tx_0),x_0 -y\rangle \\ \geq \langle G(x_0,U_{\epsilon,n}(x_0)),x_0-y\rangle+\langle F(x_0,U_{\epsilon,n}(x_0)), U_{\epsilon,n}(x_0) -a\rangle +\frac{1}{\epsilon} \langle \beta_n'(U_{\epsilon,n}(x_0))*x_0, a\rangle\\ +\frac{1}{\epsilon} \langle \beta_n(U_{\epsilon,n}(x_0)), y\rangle+\frac{1}{\epsilon}\langle \beta_n'(U_{\epsilon,n}(x_0))*U_{\epsilon,n}(x_0)-\beta_n(U_{\epsilon,n}(x_0)),x_0\rangle. \end{multline*} From which we deduce (using that $\beta_n'(z)z \geq \beta_n(z)$) that $\beta_n(U_{\epsilon,n}(x_0)) = o(\epsilon)$. Because~$y$ is chosen such that for any $x \in B^1_R$, $x^i-y^i \leq 0$, we deduce by construction of $x_0$ that, on $B^1_R$, $\beta_n(U_{\epsilon,n})$ is bounded by a $o(\epsilon)$. Hence, we obtain that $U \leq 0$. \end{proof} We now present a result of uniqueness for this type of solutions. \begin{Theorem} Under Hypotheses~\ref{stab} through \ref{mon}, there exists at most one monotone solution of the MFG of optimal stopping in the sense of Definition~\ref{defmonstop}. \end{Theorem} \begin{proof} The proof of this statement follows exactly the one of Theorem~\ref{uniqstat} by remarking that thanks to Lemma~\ref{stegall}, using the notation of the aforementioned proof, $a$ and $b$ can be chosen such that $V-a \leq 0$ and $U-b \leq 0$. \end{proof} \subsection{Comments on this notion of solution in the optimal stopping case} In this section we want to discuss how the knowledge of the value function of the MFG is helpful to understand the behavior of the population of agents. In particular we make an analogy with the notion of mixed solutions introduced in \cite{bertucci2018optimal}. First, we expect that when no player is leaving the game, $U$ solves a certain PDE, which is the master equation characterizing the MFG without optimal stopping. The set which corresponds to the fact that no player is leaving is the set $\bigcap_{i =1}^d \{ U^i < 0\}$. The fact that $U$ satisfies this property can be obtained by two arguments. First, from the PDE satisfied by $U_{\epsilon}$ for $\epsilon > 0$ and then by passing to the limit $\epsilon \to 0$. Secondly by remarking that if $U^i(x) < 0$ for all $i$ and $D_xU(x) > 0$ in the order of positive definite matrix, then the proof of Proposition~\ref{consist} can be easily adapted to show that~$U$ indeed satisfies \begin{align*} r U^i+(F(x,U)\cdot \nabla_x)U^i+\lambda \left(U^i-(T^*U(t,Tx))^i\right)&= G^i(x,U) \ \ \text{in } \{x_i > 0\}\cap \{ U < 0 \}, \\ r U^i+(F(x,U)\cdot \nabla_x)U^i+\lambda \left(U^i-(T^*U(t,Tx))^i\right)&\leq G^i(x,U) \ \ \text{in } \{x_i =0\} \cap \{ U < 0\}. \end{align*} Thus in the set $\{U < 0\}$, we can infer the evolution of the population of agents as in the case without optimal stopping, at least when $D_x U > 0$. On the other hand, when players are actually leaving the game, we would like to gain information on how they are leaving the game. Although the question of describing the precise evolution of the population is not the central question of this paper, we indicate formally what happens for the density of players. We refer to \cite{bertucci2018optimal} for more details on this question in the deterministic case. Let us consider a distribution of players $x \in \mathbb{O}_d$ such that it is optimal for some players to leave the game. For this to happen, one must have $ I(x)\ne \emptyset $, where $I$ is defined by \begin{equation}\label{defI} I(x) = \{ i \in \{1,\dots,d\} \sep U^i(x) = 0 \}. \end{equation} A natural requirement could be to expect that starting from $x$, the density of players should instantly become $\tilde{x}$ defined by \begin{equation}\label{pure} \tilde{x}^i = \begin{cases} x^i &\text{for } i \notin I(x),\\ 0 &\text{for } i \in I(x). \end{cases} \end{equation} This type of behavior corresponds to considering only symmetric Nash equilibria in pure strategies, \ie equilibria in which if one player is leaving, then all the other players are leaving. From \cite{bertucci2018optimal}, we know that we have to consider Nash equilibria in mixed strategies and that players can play a certain leaving rate (a probability to exit the game or not). Hence, even though some players are leaving the game in the state $i \in I(x)$, this does not necessary mean that $\tilde{x}^i = 0$. However, because some players are leaving and some are staying in this situation, there is a way to determine $\tilde{x}^i$. Following \cite{bertucci2018optimal}, we expect that $\tilde{x}$ is characterized by \begin{equation}\label{mixed} \begin{cases} \tilde{x}^i = x^i &\text{for } i \notin I(x),\\ U(\tilde{x}) = U(x),\\ G^i (\tilde{x}, U(\tilde{x}))\tilde{x}^i = 0 &\text{for } i \in I(x). \end{cases} \end{equation} The last line of \eqref{mixed} stands for the fact that either all the players in state $i$ have left the game, or it is also optimal to stay in the game and thus one must have $G^i (\tilde{x}, U(\tilde{x})) = 0$ (recall that the exit cost is $0$). This relation is the analogue of an integral relation in the formulation of Nash equilibria in \cite{bertucci2018optimal}. Finally, let us remark that when there is a strict monotonicity assumption on $G$, for any $x \in \mathbb{O}_d$, there is at most one solution $\tilde{x}$ of \eqref{mixed}. Thus, if we are given the solution of the master equation, we can reconstruct the trajectories by using \eqref{mixed}. \subsection{The time dependent case} As we already mentioned above, we are not going to treat in full details the time dependent case. However we indicate the natural generalizations of the results above in this situation. The definition of a solution of the problem is straightforward from definitions \ref{defms} and \ref{defmonstop}. It is given by \begin{Def}\label{defmstopt} A continuous function $U: (0,\infty)\times \mathbb{O}_d \to \mathbb{R}^d$ is a monotone solution of the time dependent MFG of optimal stopping if \begin{itemize} \item $U \leq 0$, \item for any $V \in \mathbb{R}^d$, $V \leq 0$, $y \in \mathbb{O}_d$, $R> 0$ sufficiently large, $\epsilon > 0$ and $\phi: \mathbb{R} \to \mathbb{R}$ a~smooth function, for any $(t_0,x_0)\in (0,\infty) \times B^1_R$ point of strict minimum of $(t,x) \mto \langle U(t,x)-V, x-y\rangle-\phi(t)$ on $(t_0-\epsilon, t_0] \times B^1_R$, the following holds: \begin{multline}\label{testtstop} \frac{d}{dt} \phi(t_0) + \lambda \langle U(t_0,x_0)-T^*U(t_0,Tx_0),x_0-y \rangle \\ \geq \langle F(x,U(t_0,x)), U(t_0,x)-V\rangle+\langle G(x,U(t_0,x)), x-y \rangle, \end{multline} \item $U(0,x) = \min(U_0(x),0) \text{ on } \mathbb{O}_d$. \end{itemize} \end{Def} The fact that the initial condition is only satisfied when $U_0$ is smaller than $0$ is classical feature of optimal stopping problem. The uniqueness of such solutions is a direct adaptation of the proof of Theorem~\ref{uniqt}. It can be summarized as follows. \begin{Theorem} Under Hypotheses~\ref{stab} and \ref{mon}, for any continuous function $U_0$, there exists at most one continuous function solution of the time dependent MFG of optimal stopping in the sense of Definition~\ref{defmstopt}. \end{Theorem} The question of existence of such a solution is more involved that in the stationary case. We expect that it can also be proved by penalization by considering the sequence $(U_{\epsilon})_{\epsilon > 0}$ of solutions of \eqref{stopme} and taking the limit $\epsilon \to 0$. The main argument to consider this method is that, as in the stationary case, a, uniform in $\epsilon$, estimates on $\|D_x U_{\epsilon}\|$ can be established. However, unlike in the case of \eqref{me}, because of the terms in $\epsilon^{-1}$ in \eqref{stopme}, this does not automatically translate into uniform estimates on the time regularity of~$U_{\epsilon}$. \section{The master equation for mean field games of impulse control}\label{sec:4} This section generalizes the results of the previous section to a case in which the players have the possibility to use impulse controls. Let us briefly insist on the fact that, in finite state space, the evolution of the state of a player is necessary discontinuous. Hence it could be thought that all MFG in finite state space are games of impulse controls. This is not the case. Indeed if one were to detail the game modeled by a master equation of the type of \eqref{me}, then the transition rates between the states, which are controlled by the players, would be either bounded or get a dissuasive cost as they become higher. In this section, we consider the possibility for a player to change instantly its state by paying a certain (finite) cost, this is what we call a~MFG of impulse control. We refer to \cite{bensoussan1984impulse} for a complete presentation of impulse control problems and to \cite{bertucci2020fokker} for the study of MFG of impulse control without common noise. As the formulation of impulse control is very general, and thus difficult to work with, we shall focus at some point in our study on a particular instance of MFG of impulse control. We hope that the forthcoming results convince the reader of the generality of this method. As we did in the optimal stopping case, we shall also focus on the stationary case in this section. \subsection{Description of the model} We consider a MFG in finite state space which is described by $F,G,\lambda$ and $T$ as in \eqref{sme}. We add the possibility for a player in the state $i$ to instantly jump to the state $j$ by paying a cost $k_{ij}\geq 0$. The so-called jump operator is then defined by \begin{equation}\label{defM} (Mp)^i = \min \{ k_{ij}+p^j \sep j \in \{1,\dots,d\}\}, \quad\text{for } p \in \mathbb{R}^d,\ i \in \{1,\dots,d\}. \end{equation} We shall use the notation $M^i p = (Mp)^i$ for $i \in \{1,\dots,d\}$. Given the value function~$U$ of the MFG of impulse control, the function $MU$ plays formally the same role as an exit cost. Indeed when a player is in a state $i$ and the distribution of other players is $x \in \mathbb{O}_d$, such that \begin{equation}\label{impnot} U^i(x) < M^iU(x), \end{equation} then it is strictly sub-optimal to use an impulse control. On the other hand, when there is equality in \eqref{impnot}, then it is optimal for players in state $i$ to jump to a state $j$ which reaches the minimum of $M^iU(x)$ in \eqref{defM}. As in the optimal stopping case, we introduce first a penalized version of the master equation and then explain how we can pass to the limit. \Subsection{The penalized master equation and monotone solutions of the limit game} Following the previous section, we are interested in a penalized version of the impulse control MFG in which the players cannot exactly jump instantly from one state to another but only control an intensity of jump which is bounded by $\epsilon^{-1}$ for $\epsilon > 0$ (in particular this penalized game is a ``classical'' MFG in finite state space, it only has a particular structure). The master equation associated to such a MFG is \begin{multline}\label{penalimp} rU^i + \frac{1}{\epsilon} \hat{\beta} (U^i-M^iU) +\frac{1}{\epsilon}\bigl((\boldsymbol{\alpha}\cdot\One)* x-x\cdot \boldsymbol{\alpha} \bigr)\cdot \nabla_x U^i+ (F(x,U))\cdot \nabla_x U^i\\ +\lambda \bigl(U^i-(T^*)^i U(Tx)\bigr) = G^i(x,U) \quad\text{in } \mathbb{O}_d,\ i \in \{1,\dots,d\}, \end{multline} where $\hat{\beta}$ is the positive part, $\One = (1,\dots,1)$ and $\alpha = (\alpha_{ij})_{1 \leq i,j\leq d}$ is a matrix made of~$d^2$ real valued functions on $\mathbb{O}_d$ which satisfy \begin{equation}\label{alpha} \begin{cases} \alpha_{ij}(x) = 0 &\text{if } U^i(x) < U^j(x)+k_{ij} \text{ on } \mathbb{O}_d,\\ \sum_{j = 1}^d \alpha_{ij}(x) = 1 &\text{if } \exists\, j,\ U^i(x) \geq U^j(x)+k_{ij} \text{ on } \mathbb{O}_d,\\ 0 \leq \alpha_{ij} \leq 1,\\ \sum_{j = 1}^d \alpha_{ij} \leq 1. \end{cases} \end{equation} Formally, for $i, j \in \{1,\dots,d\}$, the function $\alpha_{ij}$ indicates the proportion of players in state $i$ which choose to jump (using the impulse control) to state $j$. Let us note in particular that different jumps may be optimal to use from a single state. Thus that we are forced to use this family of functions $(\alpha_{ij})_{1\leq i,j \leq d}$. Let us insist on the fact established in \cite{bertucci2018optimal} for the optimal stopping case, that if we restrict ourselves to situations in which only a single behavior is optimal in every state, then a Nash equilibrium may not exist. From \cite{bertucci2020fokker}, we expect that some monotonicity property holds for \eqref{penalimp}--\eqref{alpha}. This following result gives a precise statement of this idea. \begin{Prop} Under Hypotheses \ref{stab} through \ref{mon}, there exists at most one smooth function $U$ such that $(U,\alpha)$ is a solution of \eqref{penalimp}--\eqref{alpha}. Moreover if such a solution exists, $U$ is monotone. \end{Prop} \begin{proof} As usual we take $(U,\alpha)$ and $(V, \gamma)$ two such solutions and we define \hbox{$W: \mathbb{O}_d^2 \to \mathbb{R}$} by \begin{equation*} W(x,y) = \langle U(x)- V(y), x-y\rangle. \end{equation*} Let us remark that $W$ is a solution of \begin{multline*} r W + \Bigl(F(x,U)+\frac{1}{\epsilon} ((\boldsymbol{\alpha}\cdot\One)* x-x\cdot \boldsymbol{\alpha}) \Bigr)\nabla_x W+\Bigl(F(y,V)+\frac{1}{\epsilon} ((\gamma\cdot\One)* y-y\cdot \gamma) \Bigr)\nabla_y W\\ = \langle G(x,U)-G(y,V), x-y \rangle+\langle F(x,U)-F(y,V), U-V \rangle\\ \shoveright{- \lambda (W(x,\xi)-W(Tx,T\xi)) +\frac{1}{\epsilon}\langle (V-MV)_+-(U- MU)_+, x-y \rangle}\\ +\frac{1}{\epsilon} \langle(\alpha\cdot\One)* x-x\cdot \alpha -(\gamma\cdot\One)* y +y\cdot \gamma, U-V \rangle. \end{multline*} Let us remark that \begin{equation*} \begin{aligned} \langle (V-MV)_+&-(U- MU)_+, x-y \rangle+\langle (\alpha\cdot\One)* x-x\cdot \alpha -(\gamma\cdot\One)* y +y\cdot \gamma, U-V \rangle\\ &=\langle x, (V-MV)_+-(U- MU)_++(U- V)* (\alpha\cdot\One)-\alpha \cdot (U-V) \rangle\\ &\hspace*{.5cm}+\langle y, (U-MU)_+-(V- MV)_++(V- U)* (\gamma\cdot\One)-\gamma \cdot (V-U) \rangle\\ &\geq\langle x, (V-MV)_+-V* (\alpha\cdot\One)+\alpha \cdot V+(\alpha*k)\cdot\One \rangle\\ &\hspace*{.5cm}+\langle y, (U-MU)_+-U* (\gamma\cdot\One)+\gamma \cdot U+(\alpha*k)\cdot \One \rangle\geq 0, \end{aligned} \end{equation*} where $k = (k_{ij})_{1\leq i,j\leq d}$. Thus we deduce that $W$ satisfies \begin{multline*} r W +\Bigl(F(x,U)+\frac{1}{\epsilon} ((\boldsymbol{\alpha}\cdot\One)* x-x\cdot \boldsymbol{\alpha}) \Bigr)\nabla_x W\\ +\Bigl(F(y,V)+\frac{1}{\epsilon} ((\gamma\cdot\One)* y-y\cdot \gamma) \Bigr)\nabla_y W + \lambda(W(x,\xi)-W(Tx,T\xi)) \geq 0. \end{multline*} Thus we conclude as we did in Proposition~\ref{uniq2}, first that $W \geq 0$, and then the required results. \end{proof} This result clearly suggests that the master equation for the impulse control MFG is well posed. Indeed, as we shall see, as the solution of the problem is monotone, the notion of monotone solution introduced in Section~\ref{sectionmon} shall be helpful. As in the case of optimal stopping, let us consider a monotone solution $U$ of the penalized equation~\eqref{penalimp} for $\epsilon > 0$ satisfying \eqref{alpha} for a family of functions $(\alpha_{ij})_{1\leq i,j\leq d}$. Thus for any $V \in \mathbb{R}^d$, $y \in \mathbb{O}_d$ and $x_0$ a point of strict local minimum of $ x \mto \langle U(x)-V, x-y\rangle$, the following holds:\vspace*{-5pt} \begin{multline*} r\langle U(x_0), x_0-y\rangle + \lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle\\ \geq \langle G(x_0,U(x_0))-\hat{\beta}(U(x_0)-MU(x_0)), x_0-y \rangle\\ + \langle F(x_0,U(x_0))+\frac{1}{\epsilon} ((\boldsymbol{\alpha}(x_0)\cdot\One)* x_0-x_0\cdot \boldsymbol{\alpha}(x_0)), U(x_0)-V\rangle. \end{multline*} From the same calculations of the previous proof, because \eqref{alpha} holds, the previous equation can be rewritten to obtain the following.\vspace*{-5pt} \begin{multline*} r\langle U(x_0), x_0-y\rangle+\lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle\geq \langle G(x_0,U(x_0)), x_0-y \rangle\\ + \langle F(x_0,U(x_0)), U(x_0)-V\rangle-\frac{1}{\epsilon}\langle x_0, ((\boldsymbol{\alpha}(x_0)\cdot\One)* V+\boldsymbol{\alpha}(x_0))\cdot V-(\alpha*k)\cdot \One\rangle. \end{multline*} Hence, if $V$ satisfies $V \leq MV$, then we obtain\vspace*{-5pt} \begin{multline*} r\langle U(x_0), x_0-y\rangle + \lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle \\ \geq \langle G(x_0,U(x_0)), x_0-y \rangle+\langle F(x_0,U(x_0)), U(x_0)-V\rangle. \end{multline*} This remark leads us to the following definition. \begin{Def}\label{defmonimp} A function $U \in \mathcal{C}(\mathbb{O}_d, \mathbb{R}^d)$ is said to be a monotone solution of the master equation for the MFG of impulse control if \begin{itemize} \item $U \leq MU$, \item for any $V \in \mathbb{R}^d$ such that $V \leq MV$, for any $y \in \mathbb{O}_d$, $R>0$ sufficiently large, and~$x_0$ a point of strict minimum of $\phi_{V,y}: x \mto \langle U(x)-V, x-y\rangle$ in $B^1_R$, the following holds\vspace*{-5pt} \begin{multline*} r\langle U(x_0), x_0-y\rangle + \lambda \langle U(x_0)-T^*U(Tx_0),x_0-y\rangle \\ \geq \langle G(x_0,U(x_0)), x_0-y \rangle+\langle F(x_0,U(x_0)), U(x_0)-V\rangle. \end{multline*} \end{itemize} \end{Def} \begin{Rem} As we can see, this definition is similar to the one of the optimal stopping case, as they only differ in the fact that the constraint of being non-positive component wise has been replaced by satisfying an inequality associated with the jump operator $M$. \end{Rem} \subsection{Results for particular mean field games of impulse control} In the present section, we assume a particular form for the jump operator $M$ and we prove a result of uniqueness for monotone solutions in the impulse control case. Let us assume that not all impulse jumps are feasible, that is we assume that $k_{ij} =+\infty$ for some $i,j \in \{1,\dots,d\}$. More precisely we assume the following. \begin{hyp}\label{clique} The cost to jump along a non trivial cycle is always infinite. That~is, for all $i \in \{1,\dots,d\}$, there exists no non constant sequence $i = i_0, i_1,\dots,i_{n+1} = i$ such that for all $k \in \{0,\dots,n\}$,\vspace*{-5pt} \begin{equation*} k_{i_{k}i_{k+1}} <+\infty. \end{equation*} Moreover, for all $i,j \in \{1,\dots,d\}$, $k_{ij} > 0$. \end{hyp} We comment on this assumption after the following result. \begin{Theorem} Under Hypotheses~\ref{stab} through \ref{mon} and \ref{clique}, there exists at most one solution of the master equation of the MFG of impulse control in the sense of Definition~\ref{defmonimp}. If it exists, this solution is monotone. \end{Theorem} \begin{proof} From Assumption~\ref{clique}, there exists an open set $\mathcal{O} \subset \mathbb{R}^d$ such that $0 \in \bar{\mathcal{O}}$ and for any $V \in \mathbb{R}^d, a \in \mathcal{O}$:\vspace*{-5pt}\enlargethispage{\baselineskip}% \begin{equation*} V \leq MV \implies V+a \leq M(V+a). \end{equation*} The rest of the proof follows exactly the one of Theorem~\ref{uniqstat} by remarking that thanks to Lemma~\ref{stegall}, using the notations of the aforementioned proof, $a$ and $b$ can be chosen in $\mathcal{O}$. \end{proof} \begin{Rem} Hypothesis \ref{clique} restricts the jumps such that it is not possible to do a sequence of successive jumps to exit one state and enter it again. We firmly believe this assumption is purely technical, even though we have not been able to prove it. Let us also mention that no particular assumption of this form is made on $F$. In particular, using ``usual'' controls, the players can come back to state from which they have jumped. \end{Rem} \section{A mean field game of entry and exit}\label{sec:5} In this section we consider an application of the tools developed in this paper. We~consider a market which agents can enter or exit by paying a certain cost, and in which the revenue of the agents in the market is a function of the total number of agents in the market. The simple model we are about to present is closely related to a model for cryptocurrency mining introduced in \cite{bertucci2020mean}, namely the version of the model with no friction at entry. The number of agents in the market is denoted by the variable $K$. The cost to exit the market is denoted by $s \in \mathbb{R}$ and the cost to enter the market is denoted by $b \in \mathbb{R}$. We assume\vspace*{-5pt} \begin{equation*} b < s. \end{equation*} Knowing the evolution of the number of agents in the market $(K_t)_{t \geq 0}$, the revenue of an agent in the market is given by\vspace*{-5pt} \begin{equation*} \int_0^{\tau} e^{-rt}g(K_t)dt, \end{equation*} where $g$ is a real valued function, $r> 0$ is a constant which takes into account the inter-temporal preference rate of the agents and $0\leq \tau \leq+\infty$ is the time at which the agent exits the market. We want to characterize the value function $U$ of this MFG which gives for a number of agents $K$ in the market, the value of the game $U(K)$ for agents in the market. Following Section~\ref{sectstop}, we introduce first a penalized version of the game in which the player cannot exit or enter the market freely. For $\epsilon > 0$, a plausible penalization of the MFG leads to the following penalized master equation:\vspace*{-3pt} \begin{equation}\label{penales} rU+\frac{1}{\epsilon} \left(\beta'(U-s)K-\beta(b-U)\right)\partial_K U+\frac{1}{\epsilon} \beta(U -s) = g(K) \quad\text{in } \mathbb{R}_+, \end{equation} where $\beta(\cdot) = (\cdot)_+$. The following result holds true. \begin{Prop} Let $U$ be a smooth solution of \eqref{penales}. Assume that $g$ is an increasing and Lipschitz function. There exists $C > 0$ depending only on $g$ and $r$ such that \begin{equation*} \|\partial_K U\|_{\infty} \leq C. \end{equation*} \end{Prop} The proof of this statement follows exactly the one of Proposition~\ref{estimatestop} so we do not detail it here. From this result, we easily deduce that, if $g$ is Lipschitz and increasing, there exists a monotone solution of \eqref{penales}. Such a monotone solution $U$ satisfies that for any $V \in \mathbb{R}$, $y \in \mathbb{R}_+$, $K_0$ a point of local strict minimum of $K \to (U(K)-V)(K-y)$ \begin{multline*} rU(K_0)(K_0-y) \geq g(K_0)(K_0-y)\\ +\frac{1}{\epsilon} \left(\beta'(U(K_0)-s)K_0-\beta(b-U(K_0))\right)(U(K_0)-V) - \frac{1}{\epsilon}\beta(U(K_0) -s)(K_0-y). \end{multline*} Let us remark that if $b \leq V \leq s$, we obtain: \begin{equation*} rU(K_0)(K_0-y) \geq g(K_0)(K_0-y). \end{equation*} This remark leads us to the following definition. \begin{Def}\label{defes} A continuous function $U: \mathbb{R} \to \mathbb{R}$ is said to be a monotone solution of the MFG of entry and exit if \begin{itemize} \item $b \leq U \leq s$, \item for any $V \in [b,s]$, for any $y \in \mathbb{R}_+$, for any point $K_0$ of strict local minimum of $K \mto (U(K)-V)(K-y)$, \begin{equation*} rU(K_0)(K_0-y) \geq g(K_0)(K_0-y). \end{equation*} \end{itemize} \end{Def} The following result is easily obtained following the previous parts of this article. \begin{Theorem} Assume that $g$ is a Lipschitz and increasing function. There exists a unique monotone solution of the MFG of entry and exit in the sense of Definition~\ref{defes}. \end{Theorem} \begin{Rem} Obviously, we only intend to apply the concepts developed in the previous sections here, and extensions of this model could easily be considered, following modeling as in \cite{bertucci2020mean} or simply by adding terms which plays the same role as $F$ and $G$ in the previous sections. \end{Rem} \section{Conclusion and future perspectives}\label{sec:6} In this paper we have provided a notion of solution to study a class of first order systems of PDE arising from the MFG theory and referred to as master equations. This notion of monotone solution relies heavily on the so-called monotone structure of the master equation (Hypothesis~\ref{mon}). Although it was already well known in the MFG community that regularity and uniqueness for the solution of the master equation can be obtained in a monotone regime, we hope that our notion of solution can be helpful to reduce the assumptions made to obtain well-posedness of master equations. Moreover we have shown in this paper that we are able to characterize the value function of MFG involving a variety of novel actions for the players (stopping, jumping, entering) using monotone solutions. The natural extensions for this notion of solution are the case in which the master equation takes the form of an infinite dimensional PDE and the case of second order equations. The generalization of this work to infinite dimensional cases is standard and shall be the subject of another work. Concerning the case of second order master equations, defining a notion of monotone solutions for functions which are one time differentiable is straightforward following the techniques we here developed, however to treat functions which are merely continuous is much more involved and shall also be the subject of a future work. Finally, we do not claim that the notion of monotone solutions is appropriate to address the question of the characterization of a value function in a MFG in a non-monotone regime, in particular because in such a situation, if such a value function exists, it may be discontinuous. Personally, we do not believe this previous problem to be solvable outside under additional structural assumptions on the MFG. By structural assumptions, we mean assumptions which completely change the nature of the MFG such as being in the potential case, having specific couplings or particular information structure for instance. \appendix \section{Two maximum principle results}\label{app:A} \begin{Lemma}\label{max1} Let $\phi: (0,\infty)\times \mathbb{O}_d \to \mathbb{R}$ be a smooth solution of \begin{align*} \partial_t \phi+F(x,\phi)\cdot\nabla_x \phi-\sum_{i =1}^d \sigma(x^i)\partial_{ii} \phi+\lambda(\phi-\phi(Tx)) &\geq 0 \quad\text{in } (0,\infty)\times \mathbb{O}_d,\\[-3pt] \phi(0,x)&\geq 0 \quad\text{in } \mathbb{O}_d, \end{align*} where $\sigma$ is a non-negative function with $\sigma (0) = 0$ and $T: \mathbb{O}_d \to \mathbb{O}_d$. Under Hypothesis~\ref{stab}, $\phi$ is a non-negative function. \end{Lemma} \begin{proof} We argue by contradiction. Arguing as in the proof of lemma 3 in \cite{bertucci2019some}, we can assume without loss of generality that there exists $(t_0,x_0) \in (0,\infty)\times \mathbb{O}_d$ such that \begin{equation*} \begin{cases} \partial_t \phi (t_0,x_0) < 0,\\ \phi(t_0,x_0) = 0 \leq \phi(t_0,y)& \forall y \in \mathbb{O}_d,\\ \sum_{i =1}^d \sigma(x_0^i)\partial_{ii} \phi(t_0,x_0) \geq 0,\\ F(x_0,\phi)\cdot\nabla_x \phi (t_0,x_0) \geq 0. \end{cases} \end{equation*} Thus we conclude to a contradiction by evaluating the PDE satisfied by $\phi$ at $(t_0,x_0)$. Hence $\phi$ is non-negative. \end{proof} \begin{Lemma}\label{max2} Let $\phi: \mathbb{O}_d \to \mathbb{R}$ be a smooth solution of \begin{align*} r \phi+F(x,\phi)\cdot\nabla_x \phi-\sum_{i =1}^d \sigma(x^i)\partial_{ii} \phi+\lambda(\phi-\phi(Tx)) &\geq 0 \quad\text{in } (0,\infty)\times \mathbb{O}_d,\\[-3pt] \phi(0,x) &\geq 0 \quad\text{in } \mathbb{O}_d, \end{align*} where $\sigma$ is a non-negative function with $\sigma (0) = 0$ and $T: \mathbb{O}_d \to \mathbb{O}_d$. Under Hypotheses~\ref{stab} and \ref{grow}, $\phi$ is a non-negative function. \end{Lemma} \begin{proof} Consider $R$ sufficiently large and $x_0:= \argmin\{\phi(x) \sep x \in B^1_R\}$. Assumptions \ref{stab} and \ref{grow} precisely state that $-F(x,p)$ points inward $B^1_R$ for any $x \in \partial B^1_R$, $p \in \mathbb{R}^d$. Thus evaluating the PDE satisfied by $\phi$ at $x_0$ we deduce that~$\phi(x_0) \geq 0$, hence that~$\phi$ is a non-negative function. \end{proof} \section{Finite state MFG on the orthant}\label{app:B} In this section we consider a problem which arises from the discretization of a~continuous problem, mainly to highlight the links between a master equation on $\mathbb{O}_d$ and on the simplex. For other examples of master equations in finite state space, we refer to \cite{bayraktar2019finite,bertucci2020mean}. Namely we are interested in the case in which $G$ is given by \begin{equation}\label{discrete} G_i(x,p) = f_i(x)-\sum_{j \ne i}H((p_{j}- p_i)_-),\quad (x,p) \in (\mathbb{R}_+)^d \times \mathbb{R}^d, \end{equation} where $H: \mathbb{R} \to \mathbb{R}$ is a function with quadratic growth whose second derivative is bounded from above and from below by a non negative constant. We then define $F(x,p) =- D_pG(x,p)x$. The master equation in such a cases is the PDE of unknown~$U$ given by \begin{equation}\label{degmfg} \partial_t U+(F(x,U)\cdot \nabla_x)U = G(x, U) \quad\text{in } (0,T)\times \mathbb{R}^d, \end{equation} where for the sake of clarity we omit terms modeling common noise, such terms could be treated following the same technique as in the previous sections. Obviously such a master equation is associated to games in which the number of players is conserved,~\ie \begin{equation*} \sum_{i}F_i = 0. \end{equation*} Let us consider a reduction of \eqref{degmfg}. We define $V:(0,T)\times \mathbb{R}^{d-1} \to \mathbb{R}^{d-1}$ by: \begin{equation}\label{defV2} V_i(t,z) = U_i(t,\phi(z))-U_d(t,\phi(z)) \end{equation} for $1\leq i \leq d$, where $\phi: \mathbb{R}^{d-1}\to \mathbb{R}$ is defined by $\phi(z) = (z_1,z_2,\dots,z_{d-1},1-\sum_i z_i)$. The natural equation to consider for $V$ is \begin{equation}\label{changmfg} \partial_t V+(\tilde{F}(z,V)\cdot\nabla_z)V = \tilde{G}(z,V) \quad\text{in } (0, \infty) \times \Sigma^d, \end{equation} where $\Sigma^d:= \{ x \in \mathbb{O}_{d-1} \sep x_1+\cdots+ x_{d-1}= 1\}$ and $\tilde{G}$ and $\tilde{F}$ are defined for $i \in \{1,\dots,d-1\}$ by \begin{equation*} \begin{cases} \tilde{G}^i(z,V) = G^i(\phi(z),\psi(V))- G^d(\phi(z),\psi(V)),\\ \tilde{F}^i(z,V) = F^i(\phi(z),\psi(V)), \end{cases} \end{equation*} where $\psi(p) = (p_1,p_2,\dots,p_{d-1},0)$ for $p \in \mathbb{R}^{d-1}$. The following result then easily follows. \begin{Prop} Let us consider two functions \[ U: (0,\infty) \times \mathbb{R}^d \to \mathbb{R}^d\quad\text{and}\quad V: (0,\infty)\times \mathbb{R}^{d-1} \to \mathbb{R}^{d-1} \] which satisfy \eqref{defV2}. Then if $U$ solves \eqref{degmfg}, $V$ solves \eqref{changmfg}. Moreover, if $x \mto (f_i(x))_{1\leq i \leq d}$ is monotone and $H$ is convex, then both $(G,F)$ and $(\tilde{G},\tilde{F})$ are monotone. \end{Prop} \backmatter \bibliographystyle{jepplain+eid} \bibliography{bertucci} \end{document}