Iterative methods search for a solution x of the linear system of equations A * x = b. A and b must be given. A is a matrix and b is a vector. Internally the method produces a solution in several iterations. The method stops when the maximum number of iterations is reached or when the new solutions is very similar to the solution of the last iteration (this case happens when the difference is less than a tolerance given by the user).

The result of an iterative method is only an approximation. The accuracy of the result depends on the number of iterations and the initial guess for x. Warm-starting can improve the result. This is done by providing a better initial guess for x: If several similar linear systems are solved, then the solution of one linear system is possible near the solution of the other linear systems. For many applications, like computer games, the same linear system of equations is computed each frame with only minor variations in A and b.

The advantage of an iterative solver is that is can provide an approximate solution very quickly. In tasks like animation, a quick estimate is often better than an exact solution which takes longer to compute.

Reference