Adding History to a Cycle
In what has been discussed to this point, each cycle is considered to operate independently of the preceding cycles.
Additional information can be gained, cheaply, by comparing successive cycles.
- As an analogy, one dimension of the gradient of a function can be estimated by two successive function evaluations.
- Two successive gradient calculations can provide an estimate of part of the second derivative. Since we are already performing the gradient calculations all that is required is the comparison.
If you are ignoring the off-diagonal elements, the result is called Preconditioned Conjugate Gradient.
When ignoring all elements, the result is called Conjugate Gradient.