A robust, discrete-gradient descent procedure for optimisation with time-dependent PDE and norm constraints

Mannix, Paul M.; Skene, Calum S.; Auroux, Didier; Marcotte, Florence

The SMAI Journal of computational mathematics, Tome 10 (2024), pp. 1-28

Many physical questions in fluid dynamics can be recast in terms of norm constrained optimisation problems; which in-turn, can be further recast as unconstrained problems on spherical manifolds. Due to the nonlinearities of the governing PDEs, and the computational cost of performing optimal control on such systems, improving the numerical convergence of the optimisation procedure is crucial. Borrowing tools from the optimisation on manifolds community we outline a numerically consistent, discrete formulation of the direct-adjoint looping method accompanied by gradient descent and line-search algorithms with global convergence guarantees. We numerically demonstrate the robustness of this formulation on three example problems of relevance in fluid dynamics and provide an accompanying library SphereManOpt.

Classification : 65N35, 15A15
Keywords: optimal control, adjoint-based methods, optimisation on manifolds

[1] Absil, Pierre-Antoine; Mahony, Robert; Sepulchre, Rodolphe Optimization algorithms on matrix manifolds, Princeton University Press, 2009 | MR

[2] Ascher, Uri M.; Ruuth, Steven J.; Spiteri, Raymond J. Implicit-explicit Runge-Kutta methods for time-dependent partial differential equations, Appl. Numer. Math., Volume 25 (1997), pp. 151-167 | DOI | Zbl | MR

[3] Ascher, Uri M.; Ruuth, Steven J.; Wetton, Brian T. R. Implicit-explicit methods for time-dependent partial differential equations, SIAM J. Numer. Anal., Volume 32 (1995), pp. 797-823 | DOI | MR | Zbl

[4] Auroux, Didier; Blum, Jacques Data assimilation methods for an oceanographic problem, Multidisciplinary methods for analysis optimization and control of complex systems (Mathematics in Industry), Volume 6, Springer, 2005, pp. 180-194 | DOI | Zbl

[5] Berg, Jens; Nordström, Jan On the impact of boundary conditions on dual consistent finite difference discretizations, J. Comput. Phys., Volume 236 (2013), pp. 41-55 | Zbl | DOI | MR

[6] Boumal, Nicolas An introduction to optimization on smooth manifolds (2022) (https://www.nicolasboumal.net/book, to appear with Cambridge University Press) | MR

[7] Boumal, Nicolas; Mishra, Bamdev; Absil, Pierre-Antoine; Sepulchre, Rodolphe Manopt, a Matlab Toolbox for Optimization on Manifolds, J. Mach. Learn. Res., Volume 15 (2014) no. 42, pp. 1455-1459 | Zbl

[8] Burns, Keaton J.; Vasil, Geoffrey M.; Oishi, Jeffrey S.; Lecoanet, Daniel; Brown, Benjamin P. Dedalus: A flexible framework for numerical simulations with spectral methods, Phys. Rev. Res., Volume 2 (2020), 023068 | DOI

[9] Dauxois, Thierry; Peacock, Thomas; Bauer, Peter; Caulfield, Colm P.; Cenedese, Claudia; Gorlé, Catherine; Haller, George; Ivey, Greg N.; Linden, Paul F.; Meiburg, Eckart et al. Confronting grand challenges in environmental fluid mechanics, Phys. Rev. Fluids, Volume 6 (2021) no. 2, 020501

[10] De Pando, Miguel Fosas; Sipp, Denis; Schmid, Peter J. Efficient evaluation of the direct and adjoint linearized dynamics from compressible flow solvers, J. Comput. Phys., Volume 231 (2012) no. 23, pp. 7739-7755 | DOI | MR | Zbl

[11] Douglas, S. C.; Amari, S.; Kung, S. Y. Gradient adaptation under unit-norm constraints, Ninth IEEE Signal Processing Workshop on Statistical Signal and Array Processing (Cat. No.98TH8381), IEEE, 1998 | DOI

[12] Eggl, Maximilian F.; Schmid, Peter J. A gradient-based framework for maximizing mixing in binary fluids, J. Comput. Phys., Volume 368 (2018), pp. 131-153 | DOI | MR | Zbl

[13] Farrell, Patrick E.; Ham, David A.; Funke, Simon W.; Rognes, Marie E. Automated derivation of the adjoint of high-level transient finite element programs, SIAM J. Sci. Comput., Volume 35 (2013) no. 4, p. C369-C393 | MR | Zbl | DOI

[14] Fikl, Alexandru; Le Chenadec, Vincent; Sayadi, Taraneh Control and optimization of interfacial flows using adjoint-based techniques, Fluids, Volume 5 (2020) no. 3, p. 156 | DOI

[15] Fikl, Alexandru; Le Chenadec, Vincent; Sayadi, Taraneh; Schmid, Peter J. A comprehensive study of adjoint-based optimization of non-linear systems with application to Burgers’ equation, 46th AIAA Fluid Dynamics Conference (2016), p. 3805

[16] Foures, Dimitry P. G.; Caulfield, Colm P.; Schmid, Peter J. Localization of flow structures using-norm optimization, J. Fluid Mech., Volume 729 (2013), pp. 672-701 | MR | Zbl | DOI

[17] Foures, Dimitry P. G.; Caulfield, Colm P.; Schmid, Peter J. Optimal mixing in two-dimensional plane Poiseuille flow at finite Péclet number, J. Fluid Mech., Volume 748 (2014), pp. 241-277 | DOI | Zbl | MR

[18] Franco, Michael; Persson, Per-Olof; Pazner, Will; Zahr, Matthew J. An adjoint method using fully implicit runge-kutta schemes for optimization of flow problems, AIAA Scitech 2019 Forum, 2019 | DOI

[19] Griewank, Andreas; Walther, Andrea Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation, ACM Trans. Math. Softw., Volume 26 (2000) no. 1, pp. 19-45 | DOI | Zbl

[20] Hager, William W. Runge–Kutta methods in optimal control and the transformed adjoint system, Numer. Math., Volume 87 (2000), pp. 247-282 | MR | Zbl | DOI

[21] Hager, William W.; Zhang, Hongchao A survey of nonlinear conjugate gradient methods, Pac. J. Optim., Volume 2 (2006) no. 1, pp. 35-58 | MR | Zbl

[22] Hartmann, Ralf Adjoint consistency analysis of discontinuous Galerkin discretizations, SIAM J. Numer. Anal., Volume 45 (2007) no. 6, pp. 2671-2696 | DOI | MR | Zbl

[23] Hascoët, Laurent; Pascual, Valérie The Tapenade Automatic Differentiation tool: Principles, Model, and Specification, ACM Trans. Math. Softw., Volume 39 (2013) no. 3, 20, 43 pages | MR | Zbl | DOI

[24] Heffernan, Conor; Caulfield, Colm P. Robust and efficient identification of optimal mixing perturbations using proxy multiscale measures, Philos. Trans. R. Soc. Lond., Ser. A, Volume 380 (2022) no. 2225, 26, 20210026, 17 pages | MR | Zbl

[25] Hinze, Michael; Pinnau, René; Ulbrich, Michael; Ulbrich, Stefan Optimization with PDE constraints, 23, Springer, 2008

[26] Johnson, Steven G. The NLopt nonlinear-optimization package, http://github.com/stevengj/nlopt, 2014

[27] Juniper, Matthew P.; Sujith, R. I. Sensitivity and nonlinearity of thermoacoustic oscillations, Annu. Rev. Fluid Mech., Volume 50 (2018), pp. 661-689 | DOI | MR | Zbl

[28] Kerswell, Rich R. Nonlinear Nonmodal Stability Theory, Annu. Rev. Fluid Mech., Volume 50 (2018), pp. 319-345 | MR | Zbl | DOI

[29] Knobloch, E. Spatial Localization in Dissipative Systems, Ann. Rev. Cond. Matter Phys., Volume 6 (2015) no. 1, pp. 325-359 | DOI

[30] Kord, Ali; Capecelatro, Jesse Optimal perturbations for controlling the growth of a Rayleigh–Taylor instability, J. Fluid Mech., Volume 876 (2019), pp. 150-185 | Zbl | DOI | MR

[31] Kraft, Dieter et al. A software package for sequential quadratic programming, DFVLR Forschungsber, 28, DFVLR Obersfaffeuhofen, 1988, 33 pages

[32] Larmor, Joseph 17. How Could a Rotating Body such as the Sun Become a Magnet?, Reports of the British Association, Volume 87 (1919), pp. 159-160

[33] Lecoanet, Daniel; Kerswell, Rich R. Connection between nonlinear energy optimization and instantons, Phys. Rev. E, Volume 97 (2018), 012212 | DOI

[34] Leykekhman, Dmitriy Investigation of commutative properties of discontinuous Galerkin methods in PDE constrained optimal control problems, J. Sci. Comput., Volume 53 (2012) no. 3, pp. 483-511 | DOI | MR | Zbl

[35] Luenberger, David G. Introduction to linear and nonlinear programming, Addison-wesley Reading, 1973, xii+356 pages | MR

[36] Mannix, Paul M.; Skene, Calum S.; Auroux, Didier; Marcotte, Florence mannixp/SphereManOpt: Initial Release, https://zenodo.org/record/7271474, 2022 | DOI

[37] Marcotte, Florence; Caulfield, Colm P. Optimal mixing in two-dimensional stratified plane Poiseuille flow at finite Péclet and Richardson numbers, J. Fluid Mech., Volume 853 (2018), pp. 359-385 | DOI | Zbl | MR

[38] Mathew, George; Mezić, Igor; Petzold, Linda A multiscale measure for mixing, Physica D, Volume 211 (2005) no. 1-2, pp. 23-46 | Zbl | DOI | MR

[39] Maugars, Bruno; Bourasseau, Sébastien; Content, Cédric; Michel, Bertrand; Berthoul, Bérenger; Ramirez, Jorge Nunez; Raud, Pascal; Hascoët, Laurent Algorithmic Differentiation for an efficient CFD solver, ECCOMAS 2022-8th European Congress on Computational Methods in Applied Sciences and Engineering (2022)

[40] Mitusch, Sebastian K.; Funke, Simon W.; Dokken, Jørgen S. dolfin-adjoint 2018.1: automated adjoints for FEniCS and Firedrake, J. Open Source Softw., Volume 4 (2019) no. 38, p. 1292 | DOI

[41] Moré, Jorge J.; Thuente, David J. Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Softw., Volume 20 (1994) no. 3, pp. 286-307 | Zbl | DOI | MR

[42] Mu, Mu; Duan, W.; Wang, Q.; Zhang, R. An extension of conditional nonlinear optimal perturbation approach and its applications, Nonlinear Process. Geophys., Volume 17 (2010) no. 2, pp. 211-220 | DOI

[43] Müller, Jens-Dominik; Cusdin, P. On the performance of discrete adjoint CFD codes using automatic differentiation, Int. J. Numer. Methods Fluids, Volume 47 (2005) no. 8-9, pp. 939-945 | DOI | Zbl

[44] Naumann, Uwe The art of differentiating computer programs: an introduction to algorithmic differentiation, Society for Industrial and Applied Mathematics, 2011 | DOI | MR

[45] Nielsen, Eric J.; Diskin, Boris Discrete adjoint-based design for unsteady turbulent flows on dynamic overset unstructured grids, AIAA J., Volume 51 (2013) no. 6, pp. 1355-1373 | DOI

[46] Pringle, Chris C. T.; Kerswell, Rich R. Using nonlinear transient growth to construct the minimal seed for shear flow turbulence, Phys. Rev. Lett., Volume 105 (2010) | DOI

[47] Sandu, Adrian On the properties of Runge–Kutta discrete adjoints, Computational Science – ICCS 2006 (Lecture Notes in Computer Science), Volume 3994, 2006, pp. 550-557 | DOI | Zbl

[48] Sato, Hiroyuki Riemannian Optimization and Its Applications, Springer, 2021 | DOI | MR

[49] Sato, Hiroyuki Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses, SIAM J. Optim., Volume 32 (2022) no. 4, pp. 2690-2717 | Zbl | DOI | MR

[50] Sato, Hiroyuki Riemannian optimization on unit sphere with $p$-norm and its applications (2022) (https://arxiv.org/abs/2202.11597)

[51] Schmid, Peter J.; de Pando, Miguel Fosas; Peake, Nigel Stability analysis for n-periodic arrays of fluid systems, Phys. Rev. Fluids, Volume 2 (2017) no. 11, 113902 | DOI | Zbl

[52] Skene, Calum S.; Yeh, Chi-An; Schmid, Peter J.; Taira, Kunihiko Sparsifying the resolvent forcing mode via gradient-based optimisation, J. Fluid Mech., Volume 944 (2022), A52, 24 pages | DOI | MR | Zbl

[53] Terwisscha van Scheltinga, A. D.; Dijkstra, H. A. Conditional nonlinear optimal perturbations of the double-gyre ocean circulation, Nonlinear Process. Geophys., Volume 15 (2008) no. 5, pp. 727-734 | DOI

[54] Thiffeault, Jean-Luc Using multiscale norms to quantify mixing and transport, Nonlinearity, Volume 25 (2012) no. 2, p. R1-R44 | Zbl | DOI | MR

[55] Townsend, James; Koep, Niklas; Weichwald, Sebastian Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation, J. Mach. Learn. Res., Volume 17 (2016) no. 137, pp. 1-5 | MR | Zbl

[56] Utke, Jean; Hascoet, Laurent; Heimbach, Patrick; Hill, Chris; Hovland, Paul; Naumann, Uwe Toward adjoinable MPI, 2009 IEEE International Symposium on Parallel & Distributed Processing, IEEE (2009), pp. 1-8

[57] Vishnampet, Ramanathan; Bodony, Daniel J.; Freund, Jonathan B. A practical discrete-adjoint method for high-fidelity compressible turbulence simulations, J. Comput. Phys., Volume 285 (2015), pp. 173-192 | DOI | MR | Zbl

[58] Willis, Ashley P. Optimization of the magnetic dynamo, Phys. Rev. Lett., Volume 109 (2012), 251101 | DOI

[59] Wright, Stephen; Nocedal, Jorge Numerical optimization, Springer Series in Operations Research and Financial Engineering, 35, Springer, 1999 | MR

[60] Zahr, Matthew J.; Persson, Per-Olof An adjoint method for a high-order discretization of deforming domain conservation laws for optimization of flow problems, J. Comput. Phys., Volume 326 (2016), pp. 516-543 | DOI | MR | Zbl

[61] Zhang, Hong; Constantinescu, Emil M.; Smith, Barry F. PETSc TSAdjoint: A Discrete Adjoint ODE Solver for First-Order and Second-Order Sensitivity Analysis, SIAM J. Sci. Comput., Volume 44 (2022) no. 1, p. C1-C24 | DOI | MR | Zbl