Newton’s method and secant methods: A longstanding relationship from vectors to matrices

Abstract

Nonlinear matrix equations arise in different scientific topics, such as applied statistics, control theory, and financial mathematics, among others. As in many other scientific areas, Newton’s method has played an important role when solving these matrix problems. Under standard assumptions, the specialized Newton methods that have been developed for specific problems exhibit local and q-quadratic convergence and require a suitable initial guess. They also require, as usual, a significant amount of computational work per iteration, that in this case involve several matrix factorizations per iterations. As expected, whenever a Newton method can be developed, a secant method can also be developed. Indeed, more recently, secant methods for solving specific nonlinear matrix problems have been developed opening a new line of research. As in previous scenarios, these specialized secant methods exhibit local and q-superlinear convergence, also require a suitable initial guess, and avoid the use of derivatives in the formulation of the schemes.

In this review we start by recalling the presence of Newton’s method and the secant methods, and also their classical relationship, in different and sometimes unexpected scenarios for vector problems. Then we present and describe the state of the art in the use of Newton's method and also the secant method in the space of matrices. A second objective is to present a unified approach for describing the features of these classical schemes, that in the space of matrices represent an interesting research area with special features to be explored.