The main idea of multi-frame super resolution (SR) algorithms is to recover a single high-resolution image from a sequence of low resolution ones of the same object. The success of the SR approaches is often related to a well registration and restoration steps. Therefore, we propose a new approach based on a partial differential equation (PDE)-constrained optimization fluid image registration and we use a fourth order PDE to treat both the registration and restoration steps that guarantee the success of SR algorithms. Since the registration step is usually a variational ill-posed model, a mathematical study is needed to check the existence of the solution to the regularized problem. Thus, we prove the existence and of the well posed fluid image registration and assure also the existence of the used second order PDE in the restoration step. The results show that the proposed method is competitive with the existing methods.
Keywords: Super resolution, bilevel PDE, fluid registration, image restoration, regularization
@article{RO_2022__56_4_3047_0,
author = {Laghrib, Amine and Hadri, Aissam and Hakim, Moad and Oummi, Hssaine},
editor = {Mahjoub, A. Ridha and Laghrib, A. and Metrane, A.},
title = {An improved {PDE-constrained} optimization fluid registration for image multi-frame super resolution},
journal = {RAIRO. Operations Research},
pages = {3047--3069},
year = {2022},
publisher = {EDP-Sciences},
volume = {56},
number = {4},
doi = {10.1051/ro/2022137},
mrnumber = {4474357},
zbl = {07585267},
language = {en},
url = {https://www.numdam.org/articles/10.1051/ro/2022137/}
}
TY - JOUR AU - Laghrib, Amine AU - Hadri, Aissam AU - Hakim, Moad AU - Oummi, Hssaine ED - Mahjoub, A. Ridha ED - Laghrib, A. ED - Metrane, A. TI - An improved PDE-constrained optimization fluid registration for image multi-frame super resolution JO - RAIRO. Operations Research PY - 2022 SP - 3047 EP - 3069 VL - 56 IS - 4 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/ro/2022137/ DO - 10.1051/ro/2022137 LA - en ID - RO_2022__56_4_3047_0 ER -
%0 Journal Article %A Laghrib, Amine %A Hadri, Aissam %A Hakim, Moad %A Oummi, Hssaine %E Mahjoub, A. Ridha %E Laghrib, A. %E Metrane, A. %T An improved PDE-constrained optimization fluid registration for image multi-frame super resolution %J RAIRO. Operations Research %D 2022 %P 3047-3069 %V 56 %N 4 %I EDP-Sciences %U https://www.numdam.org/articles/10.1051/ro/2022137/ %R 10.1051/ro/2022137 %G en %F RO_2022__56_4_3047_0
Laghrib, Amine; Hadri, Aissam; Hakim, Moad; Oummi, Hssaine. An improved PDE-constrained optimization fluid registration for image multi-frame super resolution. RAIRO. Operations Research, Tome 56 (2022) no. 4, pp. 3047-3069. doi: 10.1051/ro/2022137
[1] , , and , A fast approach of nonparametric elastic image registration problem. Math. Methods Appl. Sci. 42 (2019) 7059–7075. | MR | Zbl | DOI
[2] , , and , Markov-random-field-based super-resolution mapping for identification of urban trees in vhr images. ISPRS J. Photogramm. Remote Sens. 66 (2011) 762–775. | DOI
[3] and , Mathematical problems in image processing: partial differential equations and the calculus of variations, Vol. 147. Springer Science & Business Media (2006). | MR | Zbl
[4] , Un théorème de compacité. Acad. Sci. Paris 256 (1963) 5042–5044. | MR | Zbl
[5] , and , Adaptive regularization-based super resolution reconstruction technique for multi-focus low-resolution images. Signal Process. 103 (2014) 155–167. | DOI
[6] and , Super-resolution optical flow. Carnegie Mellon University, The Robotics Institute (1999).
[7] and , Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002) 1167–1183. | DOI
[8] and , A second-order model for image denoising. Set-Valued Var. Anal. 18 (2014) 277–306. | MR | Zbl | DOI
[9] , Functional Analysis, Sobolev Spaces and Partial Differential Equations. Springer (2011). | MR | Zbl | DOI
[10] , and , The staircasing effect in neighborhood filters and its solution. IEEE Trans. Image Process. 15 (2006) 1499–1505. | DOI
[11] and , Image recovery via total variation minimization and related problems. Numer. Math. 76 (1997) 167–188. | MR | Zbl | DOI
[12] and , Espaces fonctionnels. Utilisation dans la résolution des équations aux dérivées partielles. CNRS Editions (2007). | MR | Zbl
[13] , , , and , A new denoising model for multi-frame super-resolution image reconstruction. Signal Process. 132 (2017) 51–65. | DOI
[14] , and , An anisotropic pde for multi-frame super-resolution image reconstruction. In International Conference on Numerical Analysis and Optimization Days. Springer (2021) 29–41. | MR
[15] , , and , Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13 (2004) 1327–1344. | DOI
[16] , and , Optical flow based super-resolution: A probabilistic approach. Comput. Vis. Image Underst. 106 (2007) 106–115. | DOI
[17] , and , A robust iterative super-resolution mosaicking algorithm using an adaptive and directional huber-markov regularization. J. Vis. Commun. Image Represent. 40 (2016) 98–110. | DOI
[18] , , and , Mri inter-slice reconstruction using super-resolution. Magn. Reson. Imaging 20 (2002) 437–446. | DOI
[19] , , and , Multiplicative noise removal combining a total variation regularizer and a nonconvex regularizer. Int. J. Comput. Math. 91 (2014) 2243–2259. | MR | Zbl | DOI
[20] , , and , A nonlinear least square technique for simultaneous image registration and super-resolution. IEEE Trans. Image Process. 16 (2007) 2830–2841. | MR | DOI
[21] and , Scope of validity of psnr in image/video quality assessment. Electron. Lett. 44 (2008) 800–801. | DOI
[22] , and , Variational multiframe restoration of images degraded by noisy (stochastic) blur kernels. J. Comput. Appl. Math. 240 (2013) 123–134. | MR | Zbl | DOI
[23] , and , Convolutional neural networks for wavelet domain super resolution. Pattern Recognit. Lett. 90 (2017) 65–71. | DOI
[24] , and , A combined total variation and bilateral filter approach for image robust super resolution. EURASIP J. Image Video Process. 2015 (2015) 1–10. | DOI
[25] , , and , A multi-frame super-resolution using diffusion registration and a nonlocal variational image restoration. Comput. Math. Appl. 72 (2016) 2535–2548. | MR | Zbl | DOI
[26] , , and , A nonconvex fractional order variational model for multi-frame image super-resolution. Signal Process. Image Commun. 67 (2018) 1–11. | DOI
[27] , , , , and , Simultaneous deconvolution and denoising using a second order variational approach applied to image super resolution. Comput. Vis. Image Underst. 168 (2018) 50–63. | DOI
[28] , and , An edge preserving high-order pde for multiframe image super-resolution. J. Franklin Inst. 356 (2019) 5834–5857. | MR | Zbl | DOI
[29] , , and , A new multiframe super-resolution based on nonlinear registration and a spatially weighted regularization. Inf. Sci. 493 (2019) 34–56. | MR | Zbl | DOI
[30] , , and , A nonlinear fourth-order pde for multi-frame image super-resolution enhancement. Discrete Contin. Dyn. Syst.-B 25 (2020) 415. | Zbl | MR
[31] , and , A ridge and corner preserving model for surface restoration. SIAM J. Sci. Comput. 35 (2013) A675–A695. | MR | Zbl | DOI
[32] , , and , Template-based image reconstruction from sparse tomographic data. Appl. Math. Optim. (2019) 1–29. | MR | Zbl
[33] , and , Hessian-based norm regularization for image restoration with biomedical applications. IEEE Trans. Image Process. 21 (2012) 983–995. | MR | Zbl | DOI
[34] and , Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis. 66 (2006) 5–18. | Zbl | DOI
[35] , and , A noise-suppressing and edge-preserving multiframe super-resolution image reconstruction method. Signal Process. Image Commun. 34 (2015) 1–13. | DOI
[36] , Numerical Methods for Image Registration. Oxford University Press, USA (2007). | MR
[37] and , A combined first and second order variational approach for image reconstruction. J. Math. Imaging Vis. 48 (2014) 308–338. | MR | Zbl | DOI
[38] and , Regularized high-resolution reconstruction considering inaccurate motion information. Opt. Eng. 46 (2007) 117004. | DOI
[39] , , and , Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process. 18 (2008) 36–51. | MR | Zbl | DOI
[40] , and , Improved iterative back projection for video super-resolution. In 2014 22nd Signal Processing and Communications Applications Conference (SIU). IEEE (2014) 552–555. | DOI
[41] , , , , and , New applications of super-resolution in medical imaging. In Super-Resolution Imaging. CRC Press (2017) 401–430.
[42] and , Storage-efficient quasi-newton algorithms for image super-resolution. In 2009 16th International Conference on Digital Signal Processing. IEEE (2009) 1–6.
[43] , , and , Single image super-resolution based on space structure learning. Pattern Recognit. Lett. 34 (2013) 2094–2101. | DOI
[44] , and , Total generalized variation in diffusion tensor imaging. SIAM J. Imaging Sci. 6 (2013) 487–525. | MR | Zbl | DOI
[45] , , and , Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13 (2004) 600–612. | DOI
[46] , and , Super-resolution image synthesis using projections onto convex sets in the frequency domain. In Vol. 5674 of Computational Imaging III. International Society for Optics and Photonics (2005) 479–490.
[47] , , , and , Compressive sampling based single-image super-resolution reconstruction by dual-sparsity and non-local similarity regularizer. Pattern Recognit. Lett. 33 (2012) 1049–1059. | DOI
[48] and , Efficient diffeomorphic metric image registration via stationary velocity. J. Comput. Sci. 30 (2019) 90–97. | DOI
[49] , and , Multiframe super-resolution employing a spatially weighted total variation model. IEEE Trans. Circuits Syst. Video Technol. 22 (2012) 379–392. | DOI
[50] , , , , and , Image super-resolution: The techniques, applications, and future. Signal Process. 128 (2016) 389–408. | DOI
[51] , , and , Super-fusion: a super-resolution method based on fusion. In Vol. 2 of Object recognition supported by user interaction for service robots. IEEE (2002) 269–272. | DOI
Cité par Sources :
