Notwithstanding its efficiency and nice attributes, most research on the Hager–Zhang (HZ) iterative scheme are focused on unconstrained minimization problems. Inspired by this and recent extensions of the one-parameter HZ scheme to system of nonlinear monotone equations, two new HZ-type iterative methods are developed in this paper for solving system of monotone equations with convex constraint. This is achieved by developing two HZ-type search directions with new parameter choices combined with the popular projection method. The first parameter choice is obtained by minimizing the condition number of a modified HZ direction matrix, while the second choice is realized using singular value analysis and minimizing the spectral condition number of a nonsingular HZ search direction matrix. Interesting properties of the schemes include solving non-smooth problems and satisfying the inequality that is vital for global convergence. Using standard assumptions, global convergence of the schemes are proven and numerical experiments with recent methods in the literature, indicate that the methods proposed are promising. The effectiveness of the schemes are further demonstrated by their applications to sparse signal and image reconstruction problems, where they outperform some recent schemes in the literature.
Keywords: Nonlinear monotone systems, line search, inexact line search, projection operator
@article{RO_2022__56_1_239_0,
author = {Waziri, Mohammed Yusuf and Ahmed, Kabiru and Halilu, Abubakar Sani and Sabi{\textquoteright}u, Jamilu},
title = {Two new {Hager{\textendash}Zhang} iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing},
journal = {RAIRO. Operations Research},
pages = {239--273},
year = {2022},
publisher = {EDP-Sciences},
volume = {56},
number = {1},
doi = {10.1051/ro/2021190},
mrnumber = {4376285},
language = {en},
url = {https://www.numdam.org/articles/10.1051/ro/2021190/}
}
TY - JOUR AU - Waziri, Mohammed Yusuf AU - Ahmed, Kabiru AU - Halilu, Abubakar Sani AU - Sabi’u, Jamilu TI - Two new Hager–Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing JO - RAIRO. Operations Research PY - 2022 SP - 239 EP - 273 VL - 56 IS - 1 PB - EDP-Sciences UR - https://www.numdam.org/articles/10.1051/ro/2021190/ DO - 10.1051/ro/2021190 LA - en ID - RO_2022__56_1_239_0 ER -
%0 Journal Article %A Waziri, Mohammed Yusuf %A Ahmed, Kabiru %A Halilu, Abubakar Sani %A Sabi’u, Jamilu %T Two new Hager–Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing %J RAIRO. Operations Research %D 2022 %P 239-273 %V 56 %N 1 %I EDP-Sciences %U https://www.numdam.org/articles/10.1051/ro/2021190/ %R 10.1051/ro/2021190 %G en %F RO_2022__56_1_239_0
Waziri, Mohammed Yusuf; Ahmed, Kabiru; Halilu, Abubakar Sani; Sabi’u, Jamilu. Two new Hager–Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing. RAIRO. Operations Research, Tome 56 (2022) no. 1, pp. 239-273. doi: 10.1051/ro/2021190
[1] , and , A modified conjugate gradient method via a double direction approach for solving large-scale symmetric nonlinear equations. J. Numer. Math. Stoch. 10 (2018) 32–44. | MR
[2] , , , and , A modified Fletcher-Reeves conjugate gradient method for monotone nonlinear equations with some applications. Mathematics 7 (2019) 745. | DOI
[3] , Open problems in conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. 34 (2011) 319–330. | MR | Zbl
[4] , Accelerated adaptive Perry conjugate gradient algorithms based on the self-scaling BFGS update. J. Comput. Appl. Math. 325 (2017) 149–164. | MR | DOI
[5] , A note on memory-less SR1 and memory-less BFGS methods for large-scale unconstrained optimization. Numer. Algor. (2021). DOI: . | DOI | MR
[6] , and , An extended Dai-Liao conjugate gradient method with global convergence for nonconvex functions. Glasnik Matematic 52 (2017) 361–375. | MR | DOI
[7] and , A descent family of Dai-Liao conjugate gradient methods. Optim. Methods Softw. 29 (2013) 583–591. | MR | Zbl | DOI
[8] and , The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234 (2014) 625–630. | MR | Zbl | DOI
[9] and , Two optimal Dai-Liao conjugate gradient methods. Optimization 64 (2015) 2277–2287. | MR | DOI
[10] , and , Two new conjugate gradient methods based on modified secant equations. J. Comput. Appl. Math. 234 (2010) 1374–1386. | MR | Zbl | DOI
[11] and , A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2 (2009) 183–202. | MR | Zbl | DOI
[12] , A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19 (1965) 577–593. | MR | Zbl | DOI
[13] , A PRP type method for systems of monotone equations. Math. Comput Modelling 50 (2009) 15–20. | MR | Zbl | DOI
[14] and , New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43 (2001) 87–101. | MR | Zbl | DOI
[15] and , Nonlinear Conjugate Gradient Methods. Shanghai Scientific and Technical Publishers, Shanghai (2000).
[16] , and , A modified Perry’s conjugate gradient method-based derivative-free method for solving large-scale nonlinear monotone equation. Appl. Math. Comput. 270 (2015) 378–386. | MR | DOI
[17] , , and , Improved quasi-Newton method via SR1 update for solving symmetric systems of nonlinear equations. Malay. J. Fund. Appl. Sci. 15 (2019) 117–120. | DOI
[18] , and , A class of conjugate gradient methods for convex constrained monotone equations. Optimization 66 (2017) 2309–2328. | MR | DOI
[19] and , Benchmarking optimization software with performance profiles. Math. Program. 91 (2002) 201–213. | MR | Zbl | DOI
[20] , Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer Science and Business Media, LCC, Berlin (2010). | MR | Zbl | DOI
[21] , An optimal parameter for Dai-Liao family of conjugate gradient methods. J. Optim. Theory Appl. 169 (2016) 587–605. | MR | DOI
[22] , Model-based image reconstruction for MRI. IEEE Signal Process. Mag. 27 (2010) 81–89. | DOI
[23] and , An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12 (2003) 906–916. | MR | Zbl | DOI
[24] , and , Gradient projection for sparse reconstruction, application to compressed sensing and other inverse problems. In: IEEE J-STSP. IEEE Press, Piscataway, NJ (2007) 586–597.
[25] and , Multi-step quasi-Newton methods for optimization. J. Comput. Appl. Math. 50 (1994) 305–323. | MR | Zbl | DOI
[26] , and , Multi-step nonlinear conjugate gradient methods for unconstrained minimization. Comput. Optim. Appl. 40 (2008) 191–216. | MR | Zbl | DOI
[27] , and , A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23 (1986) 707–716. | MR | Zbl | DOI
[28] and , A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16 (2005) 170–192. | MR | Zbl | DOI
[29] and , A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2 (2006) 35–58. | MR | Zbl
[30] and , Algorithm 851: CG_Descent, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32 (2006) 113–137. | MR | DOI
[31] , and , A fixed-point continuation method for l1 regularized minimization with applications to compressed sensing. SIAM J. Optim. 19 (2008) 1107–1130. | MR | Zbl
[32] and , An improved derivative-free method via double direction approach for solving systems of nonlinear equations. J. Ram. Mathl. Soc. 33 (2018) 75–89. | MR
[33] , , and , Double direction and step length method for solving system of nonlinear equations. Eur. J. Mol. Clinl. Med. 7 (2020) 3899–3913.
[34] , , , and , On solving double direction methods for convex constrained monotone nonlinear equations with image restoration. Comput. Appl. Math. 40 (2021) 1–27. | MR | DOI
[35] , , , and , Motion control of the two joint planar robotic manipulators through accelerated Dai-Liao method for solving system of nonlinear equatiions. Eng. Comput. (2021). DOI: . | DOI
[36] , , and , Signal recovery with convex constrained nonlinear monotone equations through conjugate gradient hybrid approach. Math. Comp. Simul. 187 (2021) 520–539. | MR | DOI
[37] and , Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49 (1952) 409–436. | MR | Zbl | DOI
[38] , On a class of nonlinear integral equations arising in transport theory. SIAM J. Math. Anal. 9 (1978) 787–792. | MR | Zbl | DOI
[39] and , An efficient projected gradient method for convex constrained monotone equations with applications in compressive sensing. J. Appl. Math. Phys. 8 (2020) 983–998. | DOI
[40] and , A new modidified scaled conjugate gradient method for large-scale unconstrained optimization with non-convex objective function. Optim. Methods Softw. 34 (2019) 783–796. | MR | DOI
[41] and , Globally convergent three-term conjugate gradient projection methods for solving nonlinear monotone equations. Arab. J. Math. 7 (2018) 289–301. | MR | DOI
[42] , A spectral algorithm for large-scale systems of nonlinear monotone equations. Numer. Algor. 76 (2017) 1109–1130. | MR | DOI
[43] , and , Spectral residual method without gradient information for solving large-scale nonlinear systems of equations: theory and experiments. Technical Report RT-04-08 (2004). | MR | Zbl
[44] and , A derivative-free line search and global convergence of Broyden-like method for nonlinear equations. Optim. Methods Softw. 13 (2000) 583–599. | MR | Zbl
[45] and , A globally and superlinearly convergent Gauss–Newton-based BFGS method for symmetric nonlinear equations. SIAM J. Numer. Anal. 37 (2000) 152–172. | MR | Zbl | DOI
[46] and , A projection method for convex constrained monotone nonlinear equations with applications. Comput. Math. Appl. 70 (2015) 2442–2453. | MR | DOI
[47] and , A new Perry conjugate gradient method with the generalized conjugacy condition. In: Comput. Intel. Softw. Eng. (CiSE). 2010 International Conference on Issue (2010).
[48] and , A Perry descent conjugate gradient method with restricted spectrum. Optimization Online, Nonlinear Optimization (unconstrained optimization) (2011) 1–19.
[49] and , Globally convergent modified Perrys conjugate gradient method. Appl. Math. Comput. 218 (2012) 9197–9207. | MR | Zbl | DOI
[50] , and , On computing the regularization parameter for the Levenberg–Marquardt method via the spectral radius approach to solving systems of nonlinear equations. J. Numer. Math. Stoch. 9 (2017) 80–94. | MR | Zbl
[51] and , Numerical Optimization. Springer, New York (1999). | MR | Zbl | DOI
[52] and , Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970). | MR | Zbl
[53] , Inexact Newton methods for the nonlinear complementarity problem. Math. Program. 36 (1986) 54–71. | MR | Zbl | DOI
[54] and , A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations. Optimization 67 (2018) 1631–1648. | MR | Zbl | DOI
[55] , A class of conjugate gradient algorithms with a two step variable metric memory. Discussion paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, Chicago (1977).
[56] , A modified conjugate gradient algorithm. Oper. Res. Tech. Notes 26 (1978) 1073–1078. | MR | Zbl | DOI
[57] , The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 4 (1969) 94–112. | Zbl | DOI
[58] and , Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inf. Rech. Oper. 16 (1969) 35–43. | MR | Zbl | Numdam
[59] , Imaging via compressive sampling. IEEE Signal Process. Mag. 25 (2008) 14–20. | DOI
[60] , and , Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations. Appl. Numer. Math. 153 (2020) 217–233. | MR | Zbl | DOI
[61] , , and , Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving monotone nonlinear equations with convex constraint. Int. J. Comput. Meth. 18 (2021) 2050043. | MR | Zbl | DOI
[62] , On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15 (1978) 1247–1257. | MR | Zbl | DOI
[63] and , Newton-type methods with generalized distances for constrained optimization. Optimization 41 (1997) 257–278. | MR | Zbl | DOI
[64] and , A globally convergent inexact Newton method for systems of monotone equations. In: Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods. Springer (1998) 355–369. | MR | Zbl
[65] and , Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006). | Zbl | MR
[66] and , A scaled conjugate gradient method for solving monotone nonlinear equations with convex constraints. J. Appl. Math. (2013). DOI: . | DOI | MR | Zbl
[67] , Fundamentals of Matrix Computations, 2nd edition. John Wiley and Sons, New York (2004). | Zbl | MR
[68] and , Two descent Dai-Yuan conjugate gradient methods for systems of monotone nonlinear equations. J. Sci. Comput. 90 (2022) 1–53. | MR | Zbl | DOI
[69] , and , Jacobian free-diagonal Newton’s method for nonlinear systems with singular Jacobian. Malay. J. Math. Sci. 5 (2011) 241–255. | MR | Zbl
[70] , and , A family of Hager-Zhang conjugate gradient methods for system of monotone nonlinear equations. Appl. Math. Comput. 361 (2019) 645–660. | MR | Zbl | DOI
[71] , and , Descent Perry conjugate gradient methods for systems of monotone nonlinear equations. Numer. Algor. 85 (2020) 763–785. | MR | Zbl | DOI
[72] , and , A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arab. J. Math. 9 (2020) 443–457. | MR | Zbl | DOI
[73] , , and , Enhanced Dai-Liao conjugate gradient methods for systems of monotone nonlinear equations. SeMA J. 78 (2020) 15–51. | MR | Zbl | DOI
[74] , and , Derivative-free three-term spectral conjugate gradient method for symmetric nonlinear equations. Thai J. Math. 18 (2020) 1417–1431. | MR | Zbl
[75] , , and , Modified Dai-Yuan iterative scheme for nonlinear systems and its application. Numer. Alg. Control Optim. (2021). DOI: . | DOI | MR | Zbl
[76] , , and , Modified matrix-free methods for solving systems of nonlinear equations. Optimization 70 (2021) 2321–2340. | MR | Zbl | DOI
[77] , and , Double direction three-term spectral conjugate gradient method for solving symmetric nonlinear equations. Res. Control Opt. 6 (2022) 1–11.
[78] and , A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405 (2013) 310–319. | MR | Zbl | DOI
[79] , and , Non-smooth equations based method for -norm problems with applications to compressed sensing. Nonlinear Anal. TMA 74 (2011) 3570–3577. | MR | Zbl | DOI
[80] , , and , Image super-resolution via sparse representation. IEEE Trans. Image Process. 19 (2010) 2861–2873. | MR | Zbl | DOI
[81] , A derivative-free method for solving large-scale nonlinear systems of equations. J. Ind. Manag. Optim. 6 (2010) 149–160. | MR | Zbl | DOI
[82] , , , , and , Spectral gradient projection method for monotone nonlinear equations with convex constraints. Appl. Numer. Math. 59 (2009) 2416–2424. | MR | Zbl | DOI
[83] , A derivative-free projection method for solving convex constrained monotone equations. ScienceAsia 43 (2017) 195–200. | DOI
[84] , and , The Hager-Zhang conjugate gradient algorithm for large-scale nonlinear equations. Int. J. Comput. Math. 96 (2019) 1533–1547. | MR | Zbl | DOI
[85] and , Spectral gradient projection method for solving nonlinear monotone equations. J. Comput. Appl. Math. 196 (2006) 478–484. | MR | Zbl | DOI
[86] and , Monotonicity of fixed point and normal mappings associated with variational inequality and its application. SIAM J. Optim. 11 (2001) 962–973. | MR | Zbl | DOI
[87] and , Limited memory BFGS method for nonlinear monotone equations. J. Comput. Math. 25 (2007) 89–96. | MR
Cité par Sources :
