Some equations have no formula. There is no quadratic formula for $e^x+x-3=0$, and the formula for cubic equations is so laborious that hardly anyone uses it. In these cases the way forward is to approximate: guess a value, see how far off we are, and correct the guess. The question is how to correct it well.

Newton’s example

In De analysi, of 1669, Newton chose the equation

$$y^3-2y-5=0.$$

By trial he saw that the root is near $2$, since $2^3-2\cdot2-5=-1$. So he wrote $y=2+p$, where $p$ is the missing correction, and substituted:

$$(2+p)^3-2(2+p)-5=p^3+6p^2+10p-1=0.$$

Since $p$ is small, $p^2$ and $p^3$ are smaller still. Newton threw them away and kept $10p-1=0$, so $p=0.1$. New guess: $y=2.1$. Repeating the same idea with $y=2.1+q$, he found $q\approx-0.0054$, and so on. In a few steps he reached $y\approx2.09455148$.

The geometric idea: follow the tangent

Throwing away $p^2$ and $p^3$ is the same as replacing the curve by its tangent line. Near the point of contact, the tangent almost coincides with the curve, and finding where a line crosses the axis is easy.

Starting from a guess $x_n$, the tangent to the graph of $f$ at that point is

$$y=f(x_n)+f'(x_n)(x-x_n).$$

It crosses the horizontal axis when $y=0$. Solving for $x$ gives the new guess:

$$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}.$$

In words: new guess = guess − f(guess) ÷ f′(guess).

Applied to Newton’s equation, with $f(y)=y^3-2y-5$ and $f'(y)=3y^2-2$, the formula gives:

Step $y$
0 2
1 2.1
2 2.0945681
3 2.0945514817
4 2.0945514815

At each step the number of correct decimal places roughly doubles.

Example: square roots

To compute $\sqrt2$ we solve $x^2-2=0$. With $f'(x)=2x$, the formula becomes

$$x_{n+1}=\frac12\left(x_n+\frac{2}{x_n}\right),$$

the average of the guess and $2$ divided by the guess. Starting at $x_0=2$: $1.5$, then $1.41667$, then $1.4142157$. The Babylonians used this recipe almost 4,000 years ago; Newton’s method shows it is one case of a much more general idea.

Approximations of √2
StepxResidual x² − 2

Choose a positive approximation and advance one step at a time. The residual x² − 2 shows how far the approximation still is from satisfying the equation.

In the same way, for $\sqrt7$ we solve $x^2-7=0$. Starting at $3$: $2.6667$, $2.64583$, $2.6457513$. Three steps and we already have seven correct places.

Example: the golden ratio

The Greeks already knew a famous problem: to divide a segment $AC$ at a point $B$ so that the whole segment is to the larger part as the larger part is to the smaller,

$$\frac{AC}{AB}=\frac{AB}{BC}.$$

Euclid called this cutting the segment in extreme and mean ratio and gave the ruler-and-compass construction in Proposition VI.30 of the Elements, in a version with areas that already appears in Proposition II.11. Setting $BC=1$ and $AB=x$, the condition becomes $\frac{x+1}{x}=x$, that is,

$$x^2-x-1=0.$$

The positive root is the golden ratio, $\varphi=\frac{1+\sqrt5}2\approx1.6180340$. With $f(x)=x^2-x-1$ and $f'(x)=2x-1$, Newton’s method gives

$$x_{n+1}=x_n-\frac{x_n^2-x_n-1}{2x_n-1}=\frac{x_n^2+1}{2x_n-1}.$$

Starting from $x_0=2$ and computing with fractions,

$$x_1=\frac53,\qquad x_2=\frac{34}{21},\qquad x_3=\frac{1597}{987}\approx1.6180344.$$

The numerators and denominators all belong to the Fibonacci sequence, $1,1,2,3,5,8,13,21,34,\dots$, in which each term is the sum of the two before it. Leonardo of Pisa, known as Fibonacci, published it in 1202 in the Liber abaci, in a problem about breeding rabbits. The ratio of two consecutive terms approaches $\varphi$: $\frac85=1.6$, $\frac{13}8=1.625$, $\frac{21}{13}\approx1.615$. Newton’s method leaps from ratio to ratio, and each step roughly doubles the number of correct digits: $x_3$ already has six correct decimal places.

Example: an equation with no formula

The equation $e^x+x-3=0$ mixes an exponential with a polynomial. Equations like this are called transcendental and appear in physics, chemistry and engineering. With $f(x)=e^x+x-3$ and $f'(x)=e^x+1$, starting at $x_0=0.85$:

$$x_1\approx0.7932,\qquad x_2\approx0.79206,\qquad x_3\approx0.7920600.$$

What if the equation is linear?

Test the method on $3x-6=0$. With $f'(x)=3$ and any guess $c$:

$$c-\frac{3c-6}{3}=c-(c-2)=2.$$

The exact answer appears in a single step, whatever the guess. It makes sense: the graph is already a line, and its tangent is the line itself.

When the method fails

The method is fast but not foolproof. If the tangent is horizontal ($f'(x_n)=0$), it never crosses the axis and the division is impossible. A bad guess can throw the sequence away from the root or make it jump around in cycles. In practice we start close to the root, for example by looking at the graph, and stop when two consecutive guesses agree to the desired precision.

Exercise: take one step for √2 starting from x = 1

$x_1=\frac12\left(1+\frac21\right)=1.5$. The residual goes from $1^2-2=-1$ to $1.5^2-2=0.25$: the guess jumped to the other side of the root, but much closer to it.

Exercise: set up the method for the cube root of 10

We solve $x^3-10=0$, with $f'(x)=3x^2$: $x_{n+1}=x_n-\dfrac{x_n^3-10}{3x_n^2}=\dfrac{2x_n^3+10}{3x_n^2}$. Starting at $2$: $x_1=\dfrac{26}{12}\approx2.1667$, and $\sqrt[3]{10}\approx2.1544$.

Sources: Newton biography, MacTutor · historical diagram of the Newton–Raphson method.

All Isaac Newton topics · ← Areas and integration: the inverse of fluxions · The laws of motion: forces and changes in velocity →

Image credits.