On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle
Annales de l'Institut Fourier, Volume 52 (2002) no. 5, p. 1443-1523

In the present paper, we associate the techniques of the Lewy-Pinchuk reflection principle with the Behnke-Sommer continuity principle. Extending a so-called reflection function to a parameterized congruence of Segre varieties, we are led to studying the envelope of holomorphy of a certain domain covered by a smooth Levi-flat “hat”. In our main theorem, we show that every 𝒞 -smooth CR diffeomorphism h:MM ' between two globally minimal real analytic hypersurfaces in n (n2) is real analytic at every point of M if M ' is holomorphically nondegenerate. More generally, we establish that the reflection function h ' associated to such a 𝒞 -smooth CR diffeomorphism between two globally minimal hypersurfaces in n (n1) always extends holomorphically to a neighborhood of the graph of h ¯ in M×M ¯ ' , without any nondegeneracy condition on M ' . This gives a new version of the Schwarz symmetry principle to several complex variables. Finally, we show that every 𝒞 - smooth CR mapping h:MM ' between two real analytic hypersurfaces containing no complex curves is real analytic at every point of M, without any rank condition on h.

Dans cet article, nous associons les techniques du principe de réflexion de Lewy-Pinchuk avec celles du principe de continuité de Behnke-Sommer. Après avoir prolongé holomorphiquement une fonction dite “de réflexion” à une congruence de sous-variétés de Segre, nous sommes conduits à l’étude de l’enveloppe d’holomorphie d’un domaine recouvert d’un “chapeau” Levi-plat lisse. D’après notre résultat principal, tout CR-difféomorphisme h:MM ' de classe 𝒞 entre deux hypersurfaces analytiques réelles globalement minimales de n (n2) est analytique réel en tout point de M si M ' est holomorphiquement non-dégénérée. Plus généralement, nous établissons que la fonction de réflexion h ' associée à un tel difféomorphisme CR de classe 𝒞 entre deux hypersurfaces analytiques réelles globalement minimales se prolonge toujours holomorphiquement à un voisinage du graphe de h ¯ dans M×M ¯ ' , sans aucune condition de non-dégénérescence sur M ' . Cet énoncé fournit une nouvelle version du principe de réflexion de Schwarz en plusieurs variables complexes. Enfin, nous démontrons que toute application h:MM ' de classe 𝒞 et CR entre deux hypersurfaces analytiques réelles ne contenant pas de courbes holomorphes est analytique réelle en tout point de M, sans aucune condition de rang sur h.

DOI : https://doi.org/10.5802/aif.1922
Classification:  32V25,  32V40,  32V10,  32D10,  32D20
Keywords: reflection principle, continuity principle, CR diffeomorphism, holomorphic nondegeneracy, global minimality in the sense of Trépreau-Tumanov, reflection function, envelopes of holomorphy
@article{AIF_2002__52_5_1443_0,
     author = {Merker, Jo\"el},
     title = {On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle},
     journal = {Annales de l'Institut Fourier},
     publisher = {Association des Annales de l'institut Fourier},
     volume = {52},
     number = {5},
     year = {2002},
     pages = {1443-1523},
     doi = {10.5802/aif.1922},
     zbl = {1031.32028},
     mrnumber = {1935554},
     language = {en},
     url = {http://www.numdam.org/item/AIF_2002__52_5_1443_0}
}
Merker, Joël. On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle. Annales de l'Institut Fourier, Volume 52 (2002) no. 5, pp. 1443-1523. doi : 10.5802/aif.1922. http://www.numdam.org/item/AIF_2002__52_5_1443_0/

[A] R.A. Ajrapetyan Extension of CR-functions from piecewise smooth CR manifolds, Mat. Sb., Tome 134 (1987) no. 176-1, pp. 108-118 | Zbl 0635.32012

[A] R.A. Ajrapetyan Extension of CR-functions from piecewise smooth CR manifolds, Math. USSR Sb. (English transl.), Tome 62 (1989) no. 1, pp. 111-120 | Article | MR 912414 | Zbl 0663.32015

[Ar] M. Artin On the solutions of analytic equations, Invent. Math., Tome 5 (1968), pp. 277-291 | Article | MR 232018 | Zbl 0172.05301

[BeFo] E. Bedford; J.E. Fornaess Local extension of CR functions from weakly pseudoconvex boundaries, Michigan Math. J., Tome 25 (1978) no. 3, pp. 259-262 | Article | MR 512898 | Zbl 0401.32007

[BePi] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild; E. Bedford; S. Pinchuk Analytic continuation of biholomorphic maps, Michigan Math. J., Tome 28 (1987) no. 3, pp. 405-408 | Zbl 0681.32010

[BER1] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Algebraicity of holomorphic mappings between real algebraic sets in n , Acta Math., Tome 177 (1996) no. 2, pp. 225-273 | Article | MR 1440933 | Zbl 0890.32005

[BER2] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Real submanifolds in complex space and their mappings, Princeton University Press,, Princeton, NJ (Princeton Mathematical Series) Tome 47 (1999), pp. 404 p. | Zbl 0944.32040

[BER3] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Convergence and finite determinacy of formal CR mappings, J. Amer. Math. Soc., Tome 13 (2000) no. 4, pp. 697-723 | Article | MR 1775734 | Zbl 0958.32033

[BER4] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Local geometric properties of real submanifolds in complex space, Bull. Amer. Math. Soc., Tome 37 (2000) no. 3, pp. 309-336 | Article | MR 1754643 | Zbl 0955.32027

[BHR] M.S. Baouendi; X. Huang; L.P. Rothschild Regularity of CR mappings between algebraic hypersurfaces, Invent. Math., Tome 125 (1996) no. 1, pp. 13-36 | Article | MR 1389959 | Zbl 0855.32009

[BJT] M.S. Baouendi; H. Jacobowitz; F. Treves On the analyticity of CR mappings, Ann. of Math., Tome 122 (1985) no. 2, pp. 365-400 | Article | MR 808223 | Zbl 0583.32021

[BR1] M.S. Baouendi; L.P. Rothschild Germs of CR maps between real analytic hypersurfaces, Invent. Math., Tome 93 (1988) no. 3, pp. 481-500 | Article | MR 952280 | Zbl 0653.32020

[BR2] M.S. Baouendi; L.P. Rothschild Geometric properties of mappings between hypersurfaces in complex space, J. Differential Geom., Tome 31 (1990) no. 2, pp. 473-499 | MR 1037411 | Zbl 0702.32014

[BR3] M.S. Baouendi; L.P. Rothschild Cauchy-Riemann functions on manifolds of higher codimension in complex space, Invent. Math., Tome 101 (1990) no. 1, pp. 45-56 | Article | MR 1055709 | Zbl 0712.32009

[BR4] M.S. Baouendi; L.P. Rothschild Mappings of real algebraic hypersurfaces, J. Amer. Math. Soc., Tome 8 (1995) no. 4, pp. 997-1015 | Article | MR 1317227 | Zbl 0869.14025

[BT1] M.S. Baouendi; F. Treves A property of the functions and distributions annihilated by a locally integrable system of complex vector fields, Ann. of Math., Tome 113 (1981) no. 2, pp. 387-421 | Article | MR 607899 | Zbl 0491.35036

[BT2] M.S. Baouendi; F. Treves About the holomorphic extension of CR functions on real hypersurfaces in complex space, Duke Math. J., Tome 51 (1984) no. 1, pp. 77-107 | MR 744289 | Zbl 0564.32011

[CMS] B. Coupet; F. Meylan; A. Sukhov Holomorphic maps of algebraic CR manifolds, Internat. Math. Res. Notices, Tome 1 (1999), pp. 1-29 | Article | MR 1666972 | Zbl 0926.32044

[CPS1] B. Coupet; S. Pinchuk; A. Sukhov On partial analyticity of CR mappings, Math. Z., Tome 235 (2000), pp. 541-557 | Article | MR 1800211 | Zbl 0972.32008

[CPS2] B. Coupet; S. Pinchuk; A. Sukhov Analyticité des applications CR, C.R. Acad. Sci. Paris, Sér. I Math., Tome 329 (1999) no. 6, pp. 489-494 | Article | MR 1715137 | Zbl 0949.32018

[D'A] J.P. D'Angelo Several Complex Variables and the Geometry of Real Hypersurfaces, CRC Press, Boca Raton, FL, Studies in Advanced Mathematics (1993) | MR 1224231 | Zbl 0854.32001

[Da1] S. Damour Sur l'algébricité des applications holomorphes, C. R. Acad. Sci. Paris, Sér. I Math., Tome 332 (2001) no. 6, pp. 491-496 | MR 1834056 | Zbl 1066.14509

[Da2] S. Damour On the analyticity of smooth CR mappings between real analytic CR manifolds, Michigan Math. J., Tome 49 (2001) no. 3, pp. 583-603 | Article | MR 1872758 | Zbl 0997.32033

[De] M. Derridj Le principe de réflexion en des points de faible pseudoconvexité pour des applications holomorphes propres, Invent. Math., Tome 79 (1985) no. 1, pp. 197-215 | Article | MR 774535 | Zbl 0538.32009

[DF1] K. Diederich; J.E. Fornaess Pseudoconvex domains with real analytic boundary (1978), pp. 371-384 | MR 477153 | Zbl 0378.32014

[DF2] K. Diederich; J.E. Fornaess Biholomorphic maps between certain real analytic domains in 2 , Math. Ann., Tome 245 (1979) no. 3, pp. 255-272 | Article | MR 553344 | Zbl 0418.32018

[DF3] K. Diederich; J.E. Fornaess; J.E. Fornaess, Ed. Biholomorphic mappings between two-dimensional Hartogs domains with real analytic boundaries, Recent Developments in Several Complex Variables, Princeton University Press, Princeton N.J. (Ann. Math. Studies) Tome 100 (1981), pp. 133-150 | Zbl 0504.32011

[DF4] K. Diederich; J.E. Fornaess Proper holomorphic mappings between real-analytic pseudoconvex domains in n ., Math. Ann., Tome 282 (1988) no. 4, pp. 681-700 | Article | MR 970228 | Zbl 0661.32025

[DFY] K. Diederich; J.E. Fornaess; Z. Ye Biholomorphisms in dimension 2, J. Geom. Anal., Tome 4 (1994) no. 4, pp. 539-552 | MR 1305994 | Zbl 0864.32014

[DP1] K. Diederich; S. Pinchuk Proper holomorphic maps in dimension 2 extend, Indiana Univ. Math. J., Tome 44 (1995) no. 4, pp. 1089-1126 | MR 1386762 | Zbl 0857.32015

[DP2] K. Diederich; S. Pinchuk Reflection principle in higher dimensions, ICM, Berlin, Doc. Math. (Proceedings of the International Congress of Mathematicians) Tome Extra Vol. II (1998), pp. 703-712 | Zbl 0914.32008

[DW] K. Diederich; S.M. Webster A reflection principle for degenerate real hypersurfaces, Duke Math. J., Tome 47 (1980) no. 4, pp. 835-843 | Article | MR 596117 | Zbl 0451.32008

[F] F. Forstneric Extending proper holomorphic mappings of positive codimension, Invent. Math., Tome 95 (1989) no. 1, pp. 31-62 | Article | MR 969413 | Zbl 0633.32017

[Ha] C.K. Han Analyticity of CR equivalences between some real analytic hypersurfaces in n with degenerate Levi-forms, Invent. Math., Tome 73 (1983) no. 1, pp. 51-69 | Article | MR 707348 | Zbl 0517.32007

[HaTr] N. Hanges; F. Treves Propagation of holomorphic extendability of CR functions, Math. Ann., Tome 263 (1983) no. 2, pp. 157-177 | Article | MR 698000 | Zbl 0494.32004

[HMM] X. Huang; J. Merker; F. Meylan Mappings between degenerate real analytic hypersurfaces in n , Analysis, geometry, number theory: the mathematics of Leon Ehrenpreis (Philadelphia, PA, 1998), Amer. Math. Soc., Providence, RI (Contemp. Math.) Tome 251 (2000), pp. 321-338 | Zbl 0967.32030

[Hu] X. Huang Schwarz reflection principle in complex spaces of dimension two, Comm. Partial Differential Equations, Tome 21 (1996) no. 11-12, pp. 1781-1828 | MR 1421212 | Zbl 0886.32010

[J] B. Jöricke Deformation of CR-manifolds, minimal points and CR-manifolds with the microlocal analytic extension property, J. Geom. Anal., Tome 6 (1996) no. 4, pp. 555-611 | MR 1601405 | Zbl 0917.32007

[L] H. Lewy On the boundary behaviour of holomorphic mappings, Contrib. Centro Linceo Inter. Sc. Mat. e Loro Appl., Accad. Naz. Lincei, Tome 35 (1977), pp. 1-8

[Ma] B. Malgrange Ideals of Differentiable Functions, Oxford University Press, London (Tata Institute of Fundamental Research Studies in Mathematics, Bombay) Tome No. 3 (1967), pp. 106 p. | Zbl 0177.17902

[MaMe] H. Maire; F. Meylan Extension of smooth CR mappings between non-essentially finite hypersurfaces in 3 , Ark. Math., Tome 35 (1997) no. 1, pp. 185-199 | Article | MR 1443041 | Zbl 0885.32017

[Me1] J. Merker Global minimality of generic manifolds and holomorphic extendibility of CR functions, Internat. Math. Res. Notices (1994) no. 8, pp. approx. 14 p. (electronic) 329-342 | MR 1289578 | Zbl 0815.32007

[Me2] J. Merker On removable singularities for CR functions in higher codimension, Internat. Math. Res. Notices (1997) no. 1, pp. approx. 36 p. (electronic) 21-56 | MR 1426732 | Zbl 0880.32009

[Me3] J. Merker On the Schwarz symmetry principle in three-dimensional complex euclidean space (1997) (Prépublication Ecole Normale Supérieure, 25, 62 p.)

[Me4] J. Merker Vector field construction of Segre sets (1998, augmented in 2000) (Preprint 1998, \tt arXiv.org/abs/math.CV/9901010)

[Me5] J. Merker On the partial algebraicity of holomorphic mappings between two real algebraic sets in the complex euclidean spaces of different dimensions, Bull. Soc. Math. France, Tome 129 (2001) no. 4, pp. 547-591 | Numdam | MR 1894150 | Zbl 0998.32019

[Me6] J. Merker Étude de la régularité analytique de l'application de symétrie CR formelle (June 2000) (Preprint, \tt arXiv.org/math/abs/0005290)

[Me7] J. Merker Convergence of formal invertible CR mappings between minimal holomorphically nondegenerate real analytic hypersurfaces, Int. J. Math. Math. Sci., Tome 26 (2001) no. 5, pp. 281-302 | Article | MR 1854009 | Zbl 1002.32029

[Me8] J. Merker Étude de la régularité analytique de l'application de symétrie CR formelle, C. R. Acad. Sci. Paris, Sér. I Math., Tome 333 (2001) no. 3, pp. 165-168 | MR 1851618 | Zbl 0994.32028

[Mey] F. Meylan The reflection principle in complex space, Indiana Univ. Math. J., Tome 44 (1995) no. 3, pp. 783-796 | MR 1375349 | Zbl 0857.32007

[Mi1] N. Mir An algebraic characterization of holomorphic nondegeneracy for real algebraic hypersurfaces and its application to CR mappings, Math. Z., Tome 231 (1999) no. 1, pp. 189-202 | Article | MR 1696763 | Zbl 0930.32020

[Mi2] N. Mir Germs of holomorphic mappings between real algebraic hypersurfaces, Ann. Inst. Fourier, Tome 48 (1998) no. 3, pp. 1025-1043 | Article | Numdam | MR 1656006 | Zbl 0914.32009

[Mi3] N. Mir On the convergence of formal mappings (To appear in Comm. Anal. Geom.) | MR 1894140 | Zbl 1008.32022

[Mi4] N. Mir Formal biholomorphic maps of real analytic hypersurfaces, Math. Res. Letters, Tome 7 (2000), p. 2-3, 343-359 | MR 1764327 | Zbl 0964.32013

[MM1] J. Merker; F. Meylan Extension de germes de difféomorphismes CR pour une classe d'hypersurfaces analytiques réelles non essentiellement finies dans 3 , Complex variables Theory Appl., Tome 40 (1999) no. 1, pp. 19-34 | MR 1742868 | Zbl 1025.32029

[MM2] J. Merker; F. Meylan On the Schwarz symmetry principle in a model case, Proc. Amer. Math. Soc., Tome 127 (1999) no. 4, pp. 1097-1102 | Article | MR 1476379 | Zbl 0919.32011

[MP1] J. Merker; E. Porten On removable singularities for integrable CR functions, Indiana Univ. Math. J., Tome 48 (1999) no. 3, pp. 805-856 | MR 1736977 | Zbl 0935.32010

[MP2] J. Merker; E. Porten On wedge extendability of CR-meromorphic functions (To appear in Math. Z) | MR 1938701 | Zbl 1026.32020

[N] T.S. Neelon On solutions of real analytic equations, Proc. Amer. Math. Soc., Tome 125 (1997) no. 3, pp. 2531-2535 | Article | MR 1396991 | Zbl 0890.32004

[P1] S. Pinchuk A boundary uniqueness theorem for holomorphic functions of several complex variables, Mat. Zametki, Tome 15 (1974), pp. 205-212 | MR 350065 | Zbl 0292.32002

[P2] S. Pinchuk On proper holomorphic mappings of strictly pseudoconvex domains (Russian), Sibirsk. Mat. Z., Tome 15 (1974), pp. 909-917 | Zbl 0303.32016

[P3] S. Pinchuk On the analytic continuation of holomorphic mappings (Russian), Mat. Sb. (N.S.), Tome 98(140)-3(11) (1975), p. 375-392, 416-435, 495-496 | MR 393562 | Zbl 0366.32010

[P4] S. Pinchuk Holomorphic mappings of real-analytic hypersurfaces (Russian), Mat. Sb. (N.S.), Tome 105(147) (1978) no. 4, p. 574-593, 640 | MR 496595 | Zbl 0438.32009

[Po] E. Porten Habilitationsschrift (In preparation)

[PV] S. Pinchuk; K. Verma Analytic sets and the boundary regularity of CR mappings, Proc. Amer. Math. Soc., Tome 129 (2001) no. 9, pp. 2623-2632 | Article | MR 1838785 | Zbl 0981.32024

[R] C. Rea Prolongement holomorphe des fonctions CR, conditions suffisantes, C. R. Acad. Sci. Paris, Sér I Math., Tome 297 (1983) no. 3, pp. 163-165 | MR 725396 | Zbl 0568.32011

[Sha] R. Shafikov Analytic continuation of germs of holomorphic mappings between real hypesurfaces in n , Michigan Math. J., Tome 47 (2000) no. 1, pp. 133-149 | Article | MR 1755261 | Zbl 0966.32007

[SS] R. Sharipov; A. Sukhov On CR mappings between algebraic Cauchy-Riemann manifolds and separate algebraicity for holomorphic functions, Trans. Amer. Math. Soc., Tome 348 (1996) no. 2, pp. 767-780 | Article | MR 1325920 | Zbl 0851.32017

[St1] N. Stanton Infinitesimal CR automorphisms of rigid hypersurfaces of the space of n complex variables, Amer. J. Math., Tome 117 (1995) no. 1, pp. 141-167 | MR 1314461 | Zbl 0826.32013

[St2] N. Stanton Infinitesimal CR automorphisms of real hypersurfaces, Amer. J. Math., Tome 118 (1996) no. 1, pp. 209-233 | Article | MR 1375306 | Zbl 0849.32012

[Su1] A. Sukhov On the mapping problem for quadric Cauchy-Riemann manifolds, Indiana Univ. Math. J., Tome 42 (1993) no. 1, pp. 27-36 | Article | MR 1218705 | Zbl 0848.32016

[Su2] A. Sukhov On CR mappings of real quadric manifolds, Michigan Math. J., Tome 41 (1994) no. 1, pp. 143-150 | Article | MR 1260615 | Zbl 0821.32015

[Tr1] J.-M. Trépreau Sur le prolongement holomorphe des fonctions CR définies sur une hypersurface réelle de classe 𝒞 2 dans n , Invent. Math., Tome 83 (1986), pp. 583-592 | Article | MR 827369 | Zbl 0586.32016

[Tr2] J.-M. Trépreau Sur la propagation des singularités dans les variétés CR, Bull. Soc. Math. Fr., Tome 118 (1990) no. 4, pp. 403-450 | Numdam | MR 1090408 | Zbl 0742.58053

[Tr3] J.-M. Trépreau Holomorphic extension of CR functions: a survey, Partial differential equations and mathematical physics (Copenhagen, 1995; Lund 1995), Birkhäuser Boston, Boston, MA (Progr. Nonlinear Differential Equations Appl.) Tome 21 (1996), pp. 333-355 | Zbl 0851.32021

[Tu1] A.E. Tumanov Extending CR functions on a manifold of finite type over a wedge (Russian), Mat. Sb. (N.S.), Tome 136(178) (1988) no. 1, pp. 128-139 | MR 945904 | Zbl 0692.58005

[Tu1] A.E. Tumanov Extending CR functions on a manifold of finite type over a wedge, Math. USSR Sb., Tome 64 (1989) no. 1, pp. 129-140 | Article | MR 945904 | Zbl 0692.58005

[Tu2] A.E. Tumanov Connections and propagation of analyticity for CR functions, Duke Math. J., Tome 73 (1994) no. 1, pp. 1-24 | Article | MR 1257276 | Zbl 0801.32005

[Tu3] A.E. Tumanov On the propagation of extendibility of CR functions, Complex analysis and geometry (Trento, 1993), Dekker, New York (Lecture Notes in Pure and Appl. Math.) Tome 173 (1996), pp. 479-498 | Zbl 0849.32013

[V] K. Verma Boundary regularity of correspondences in 2 , Math. Z., Tome 231 (1999) no. 2, pp. 253-299 | Article | MR 1703349 | Zbl 0939.32014

[W1] S.M. Webster On the mapping problem for algebraic real hypersurfaces, Invent. Math., Tome 43 (1977) no. 1, pp. 53-68 | Article | MR 463482 | Zbl 0348.32005

[W2] S.M. Webster On the reflection principle in several complex variables, Proc. Amer. Math. Soc., Tome 71 (1978) no. 1, pp. 26-28 | Article | MR 477138 | Zbl 0626.32019

[W3] S.M. Webster Holomorphic mappings of domains with generic corners, Proc. Amer. Math. Soc., Tome 86 (1982) no. 2, pp. 236-240 | Article | MR 667281 | Zbl 0505.32014