Chapter 12

Linearization and Approximation

12.1Linearization and Differentials

Take a complicated function, zoom in, replace it with something linear. Thus we are looking to replace graphs by planes. To start, look back on the tangent line formula from single variable calculus:

𝑦 = 𝑦 0 + 𝑓' (𝑥 0) (𝑥 - 𝑥 0)

Our extension to multiple dimensions is just....to add more variables! We need to adjust $𝑧$ not just for changes in $𝑥$ any more, but also for changes in any input.

Theorem 12.1 (Tangent Plane and Linearization). Suppose $𝑓$ is differentiable at $(𝑎, 𝑏)$ and let $𝑧 0 = 𝑓 (𝑎, 𝑏)$ . The plane

𝑧 = 𝑧 0 + 𝑓 𝑥 (𝑎, 𝑏) (𝑥 - 𝑎) + 𝑓 𝑦 (𝑎, 𝑏) (𝑦 - 𝑏)

is tangent to the graph of $𝑓$ at $(𝑎, 𝑏, 𝑧 0)$ . Because the tangent plane is the best linear approximation to a function at a point, it is often called the linearization:

𝐿 (𝑥, 𝑦) = 𝑓 (𝑎, 𝑏) + 𝑓 𝑥 (𝑎, 𝑏) (𝑥 - 𝑎) + 𝑓 𝑦 (𝑎, 𝑏) (𝑦 - 𝑏) .

More generally,

𝐿 (⃗ 𝑥) = 𝑓 (⃗ 𝑝) + \nabla 𝑓 (⃗ 𝑝) \cdot (⃗ 𝑥 - ⃗ 𝑝) .

Figure 12.1 The tangent plane is assembled from the two slopes of the previous chapter. Through the point, the

𝑥

-slice and the

𝑦

-slice each carry a tangent line, colored by the direction it varies along; the tangent plane is the one plane containing both, and the readout writes it with each slope in its line's color:

𝑧 = 𝑓 (𝑎, 𝑏) + 𝑓 𝑥 (𝑎, 𝑏) (𝑥 - 𝑎) + 𝑓 𝑦 (𝑎, 𝑏) (𝑦 - 𝑏)

. The point tours the domain by itself so the plane can be watched tilting; take hold of it to put it where you like. Toggle the normal to see

⃗ 𝑛 = ⟨ - 𝑓 𝑥, - 𝑓 𝑦, 1 ⟩

read straight off the same two numbers—the upward-facing one, which the last part of the book will lean on.

The same formula works in higher dimensions, by just adding more terms. Since this gives us an implicit plane, we can rearrange this to the "standard form" and find the normal vector to the graph.

Theorem 12.2 (Normal Vector to a Graph). At the point $(𝑎, 𝑏, 𝑓 (𝑎, 𝑏))$ , an upward-facing normal vector to the graph of $𝑓$ is

⃗ 𝑛 = ⟨ - 𝑓 𝑥 (𝑎, 𝑏), - 𝑓 𝑦 (𝑎, 𝑏), 1 ⟩ .

Its negative,

- ⃗ 𝑛 = ⟨ 𝑓 𝑥 (𝑎, 𝑏), 𝑓 𝑦 (𝑎, 𝑏), - 1 ⟩,

is a downward-facing normal vector.

Note that any scalar multiple of this vector is also a normal vector to the graph - this just provides one such vector. Depending on the application, sometimes we want the upward facing normal and sometimes we want the downward facing normal.

We will use normal vectors to surfaces a lot in the last portion of this course, on Vector Analysis. Here we will often need to be careful, and think about whether we want the normal that is pointed up or down in a given application.

Example 12.3 (A Tangent Plane). Find the tangent plane to $𝑧 = 𝑥 2 + 2 𝑦 2$ above the point $(𝑥, 𝑦) = (2, 3)$ .

Let $𝑓 (𝑥, 𝑦) = 𝑥 2 + 2 𝑦 2$ . At the given point,

𝑓 (2, 3) = 22, 𝑓 𝑥 (2, 3) = 4, 𝑓 𝑦 (2, 3) = 12 .

Therefore the tangent plane is

𝑧 = 22 + 4 (𝑥 - 2) + 12 (𝑦 - 3) .

An upward-facing normal is $⟨ - 4, - 12, 1 ⟩$ .

The second method we could use is to try and find a parametric equation for the plane. Again, we start by considering the one dimensional case. A parametric line is of the form $⃗ 𝑣 𝑡 + 𝑝$ for $⃗ 𝑣$ the direction and $𝑝$ a point. How do we find the tangent line to a function $𝑦 = 𝑓 (𝑥)$ ? The derivative measures the slope, so if we go over by 1 unit in the $𝑥$ direction we go up by $𝑓' (𝑥 0)$ units in the $𝑦$ -direction. Thus, the direction vector is

⃗ 𝑣 = ⟨ 1, 𝑓' (𝑥 0) ⟩ .

Now all we need is the position: but that's just $(𝑥 0, 𝑓 (𝑥 0))$ on the graph! Putting these together we get the tangent line

⃗ ℓ (𝑡) = (𝑥 0 𝑓 (𝑥 0)) + 𝑡 (1 𝑓' (𝑥 0)) .

What do we do for a function of two or more variables? We just...add more variables!

For

⃗ 𝑟 (𝑥, 𝑦) = ⟨ 𝑥, 𝑦, 𝑓 (𝑥, 𝑦) ⟩,

the two coordinate-direction tangent vectors are

⃗ 𝑟 𝑥 = ⟨ 1, 0, 𝑓 𝑥 ⟩, ⃗ 𝑟 𝑦 = ⟨ 0, 1, 𝑓 𝑦 ⟩ .

Theorem 12.4 (Parametric Tangent Plane). The tangent plane to the graph of $𝑓$ at $(𝑎, 𝑏, 𝑓 (𝑎, 𝑏))$ can be parameterized by

⃗ ℓ (𝑠, 𝑡) = ⎛ ⎜ ⎜ ⎜ ⎝ 𝑎 𝑏 𝑓 (𝑎, 𝑏) ⎞ ⎟ ⎟ ⎟ ⎠ + 𝑠 ⎛ ⎜ ⎜ ⎜ ⎝ 10 𝑓 𝑥 (𝑎, 𝑏) ⎞ ⎟ ⎟ ⎟ ⎠ + 𝑡 ⎛ ⎜ ⎜ ⎜ ⎝ 01 𝑓 𝑦 (𝑎, 𝑏) ⎞ ⎟ ⎟ ⎟ ⎠ .

Example 12.5 (A Parametric Tangent Plane). Find a parametric equation for the tangent plane to the saddle $𝑧 = 𝑥 2 - 𝑦 2$ at the point $(1, 1, 0)$ .

Here

𝑓 𝑥 (1, 1) = 2, 𝑓 𝑦 (1, 1) = - 2 .

Thus,

⃗ ℓ (𝑠, 𝑡) = ⎛ ⎜ ⎜ ⎜ ⎝ 110 ⎞ ⎟ ⎟ ⎟ ⎠ + 𝑠 ⎛ ⎜ ⎜ ⎜ ⎝ 102 ⎞ ⎟ ⎟ ⎟ ⎠ + 𝑡 ⎛ ⎜ ⎜ ⎜ ⎝ 01 - 2 ⎞ ⎟ ⎟ ⎟ ⎠ .

The differentiability of a function in multiple variables is defined in terms of the existence of a tangent plane: we say that a function $𝑓 (𝑥, 𝑦)$ is differentiable at a point $⃗ 𝑝$ if there exists some tangent plane that well-approximates it at that point.

Definition 12.6 (Multivariable Differentiability). A function $𝑓$ is differentiable at $⃗ 𝑝$ if

𝑓 (⃗ 𝑝 + ⃗ ℎ) = 𝑓 (⃗ 𝑝) + \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ ℎ + 𝐸 (⃗ ℎ),

where the error is small compared with the step:

l i m ⃗ ℎ \to ⃗ 0 | 𝐸 (⃗ ℎ) | ‖ ⃗ ℎ ‖ = 0 .

Functions we will see are mostly differentiable, but warning there are functions that are not. Luckily, there's an easy sufficient condition to check using partial derivatives:

Theorem 12.7 (A Sufficient Condition for Differentiability). If all first partial derivatives of a multivariable function exist and are continuous in a neighborhood of a point, then the function is differentiable there.

The existence of the partial derivatives at the point alone is not enough. They measure the function along coordinate lines, while differentiability requires one linear approximation to work for steps arriving from every direction.

The Fundamental Strategy of calculus is to take a complicated nonlinear object (like a function that you encounter in some real-world problem) and zoom in until it looks linear. Here, this zooming in process is realized by finding the tangent plane. Close to the point $(𝑎, 𝑏)$ the graph of the function $𝑧 = 𝑓 (𝑥, 𝑦)$ looks like

𝐿 (𝑥, 𝑦) = 𝑧 0 + 𝑓 𝑥 (𝑎, 𝑏) (𝑥 - 𝑎) + 𝑓 𝑦 (𝑎, 𝑏) (𝑦 - 𝑏)

where $𝑧 0 = 𝑓 (𝑎, 𝑏)$ . This is just rehashing our definition of the tangent plane of course: but one use for it is to be able to approximate the value of $𝑓 (𝑥, 𝑦)$ if you know the value of $𝑓$ at a nearby point $(𝑎, 𝑏)$ and also its partial derivatives there.

Example 12.8 (Numerical Approximation). Find an approximate value of $𝑥 2 + 3 𝑥 𝑦 - 𝑦 2$ at the point $(2.05, 2.96)$ .

Let

𝑓 (𝑥, 𝑦) = 𝑥 2 + 3 𝑥 𝑦 - 𝑦 2 .

The nearby point $(2, 3)$ is convenient, and there

𝑓 (2, 3) = 13, 𝑓 𝑥 (2, 3) = 13, 𝑓 𝑦 (2, 3) = 0 .

Since $𝑑 𝑥 = 0.05$ and $𝑑 𝑦 = - 0.04$ , the linearization gives

𝑓 (2.05, 2.96) \approx 13 + 13 (0.05) + 0 (- 0.04) = 13.65 .

The actual value is $13.6449$ , so the linear approximation is already quite close.

Using linearization to estimate changes in a value is fundamental to physics and engineering. In 1-dimension, we define a variable called $𝑑 𝑥$ that we think of as measuring small changes in the input variable, and $𝑑 𝑦 = 𝑦 - 𝑦 0$ which measures small changes in the output. These are related by

𝑑 𝑦 = 𝑓' (𝑥) 𝑑 𝑥

So, any change in the input is multiplied by the derivative to give a change in the output. We can do a similar thing in more variables.

Starting at $(𝑎, 𝑏)$ and stepping by $(𝑑 𝑥, 𝑑 𝑦)$ , the actual change in the output is

Δ 𝑧 = 𝑓 (𝑎 + 𝑑 𝑥, 𝑏 + 𝑑 𝑦) - 𝑓 (𝑎, 𝑏) .

The tangent plane predicts the linear change

Definition 12.9 (Differentials). The differential of $𝑧 = 𝑓 (𝑥, 𝑦)$ at $(𝑎, 𝑏)$ is

𝑑 𝑧 = 𝑓 𝑥 (𝑎, 𝑏) 𝑑 𝑥 + 𝑓 𝑦 (𝑎, 𝑏) 𝑑 𝑦 .

For a differentiable function and a small step, $Δ 𝑧 \approx 𝑑 𝑧$ .

Figure 12.2 Step away from the point by

(𝑑 𝑥, 𝑑 𝑦)

: the tangent plane predicts the rise

𝑑 𝑧 = 𝑓 𝑥 𝑑 𝑥 + 𝑓 𝑦 𝑑 𝑦

, and the surface delivers the true rise

Δ 𝑧

. The pane beside the surface cuts along the step, where the two rises sit on one ordinary graph—the line's rise is

𝑑 𝑧

, the curve's is

Δ 𝑧

, and the violet gap between them is the linearization's entire error. Drag the step shorter and watch the gap die faster than the step itself: halve the step and the miss roughly quarters, which is why

𝑑 𝑧

can be trusted for small changes.

This allows us to easily estimate how much $𝑧$ could change if we know how much $𝑥$ and $𝑦$ can change. This is of fundamental importance in error analysis, the foundation of all experimental science's ability to compare with theoretical predictions.

For worst-case error, the contributions from uncertain measurements may all push in the unfavorable direction, so we estimate

| Δ 𝑓 | \approx | 𝑓 𝑥 | | 𝑑 𝑥 | + | 𝑓 𝑦 | | 𝑑 𝑦 | .

Example 12.10 (Error in the Volume of a Cone). The volume of a cone is given by $𝑉 = 𝜋 𝑟 2 ℎ / 3$ . We have a cone which we measure the height to be 10cm and the radius to be 25cm, but our measuring device can have an error up to 1mm or $0.1 𝑐 𝑚$ . What is the estimated maximal error in volume our measurement could have?

The relevant partial derivatives are

𝑉 𝑟 = 2 𝜋 𝑟 ℎ 3, 𝑉 ℎ = 𝜋 𝑟 2 3 .

Thus,

| Δ 𝑉 | \approx ∣ 2 𝜋 (25) (10) 3 ∣ (0.1) + ∣ 𝜋 (25) 2 3 ∣ (0.1) = 37.5 𝜋 c m 3 .

This measurement can also be interpreted geometrically: this is the approximate volume of a thin-shelled cone of thickness $1 𝑚 𝑚$ with radius 25cm and height 10cm.

Figure 12.3 Why error analysis is linear. Each side of the plate is measured to within

𝑑

, so the area could be wrong by two strips and a corner. The differential

𝑑 𝐴 = ¯ 𝑦 𝑑 𝑥 + ¯ 𝑥 𝑑 𝑦

is exactly the two strips, and the corner

𝑑 𝑥 𝑑 𝑦

it ignores is the one piece that dies faster than the measurement improves— tighten the tolerance and watch it vanish from the budget first. For a worst case the contributions add in absolute value:

| Δ 𝐴 | \approx | ¯ 𝑦 | | 𝑑 𝑥 | + | ¯ 𝑥 | | 𝑑 𝑦 |

Example 12.11 (Error in the Volume of a Box). The dimensions of a rectangular box are measured to be 75cm, 60cm and 40cm. Each measurement is correct to within $0.05 𝑐 𝑚$ . What is the maximal error in volume measurement we might expect?

For $𝑉 = 𝑥 𝑦 𝑧$ ,

𝑑 𝑉 = 𝑦 𝑧 𝑑 𝑥 + 𝑥 𝑧 𝑑 𝑦 + 𝑥 𝑦 𝑑 𝑧 .

Therefore,

| Δ 𝑉 | \approx 0.05 ((60) (40) + (75) (40) + (75) (60)) = 495 c m 3 .

12.2Quadratic Approximation

We've already gotten a ton of use out of linear approximations to a multivariable function. But we can learn even more by proceeding to higher derivatives. Here we study the quadratic approximation that includes all the first and second derivative information. Like in the linear case, the best way to get started is to recall what happens in one variable for the second order term in a Taylor series:

𝑓 (𝑥) \approx 𝑓 (𝑎) + 𝑓' (𝑎) (𝑥 - 𝑎) + 12 𝑓'' (𝑎) (𝑥 - 𝑎) 2

The move to multiple variables for the linearization required us just tacking on analogous terms for all additional variables. Happily the same holds true here!

Definition 12.12 (Quadratic Approximation). If $𝑓 (𝑥, 𝑦)$ has continuous second partial derivatives near $(𝑎, 𝑏)$ , its quadratic approximation at $(𝑎, 𝑏)$ is

𝑓 (𝑥, 𝑦) \approx 𝑓 (𝑎, 𝑏) + 𝑓 𝑥 (𝑎, 𝑏) (𝑥 - 𝑎) + 𝑓 𝑦 (𝑎, 𝑏) (𝑦 - 𝑏) + 12 (𝑓 𝑥 𝑥 (𝑎, 𝑏) (𝑥 - 𝑎) 2 + 2 𝑓 𝑥 𝑦 (𝑎, 𝑏) (𝑥 - 𝑎) (𝑦 - 𝑏) + 𝑓 𝑦 𝑦 (𝑎, 𝑏) (𝑦 - 𝑏) 2) .

Because we know the order in which we take partials doesn't matter, the two mixed terms combine into $2 𝑓 𝑥 𝑦 (𝑎, 𝑏) (𝑥 - 𝑎) (𝑦 - 𝑏)$ .

If you have taken Linear Algebra (or have seen Matrix Multiplication before elsewhere) there is a nice way to remember this formula. For $⃗ ℎ = ⃗ 𝑥 - ⃗ 𝑝$ , writing it out in terms of row vectors, column vectors and matrices, where $\nabla 𝑓$ is the gradient (vector of first derivatives) and $𝐻 𝑓$ is the Hessian (matrix of second derivatives),

𝑓 (⃗ 𝑝 + ⃗ ℎ) \approx 𝑓 (⃗ 𝑝) + \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ ℎ + 12 ⃗ ℎ 𝑇 𝐻 𝑓 (⃗ 𝑝) ⃗ ℎ .

This formula looks intimidating in any of these forms when written out completely. But it is actually super easy to remember!

Start with the value $𝑓 (𝑎, 𝑏)$ at the point we know.
Add in first derivatives $𝑓 𝑥 (𝑎, 𝑏)$ and $𝑓 𝑦 (𝑎, 𝑏)$ multiplied by how far we've moved in that direction.
Add in $12$ each second derivative, multiplied by the differences $(𝑥 - 𝑎) (𝑥 - 𝑎)$ , or $(𝑥 - 𝑎) (𝑦 - 𝑏)$ or $(𝑦 - 𝑏) (𝑦 - 𝑏)$ depending on which second derivatives were taken.

Example 12.13 (A Quadratic Approximation). Find a quadratic approximation to $𝑓 (𝑥, 𝑦) = 𝑒 𝑥 𝑦 2$ at $(1, 2)$ .

At $(1, 2)$ ,

𝑓 = 4 𝑒, 𝑓 𝑥 = 4 𝑒, 𝑓 𝑦 = 4 𝑒,

and

𝑓 𝑥 𝑥 = 4 𝑒, 𝑓 𝑥 𝑦 = 4 𝑒, 𝑓 𝑦 𝑦 = 2 𝑒 .

Writing $ℎ = 𝑥 - 1$ and $𝑘 = 𝑦 - 2$ , the quadratic approximation is

𝑄 (𝑥, 𝑦) = 4 𝑒 + 4 𝑒 ℎ + 4 𝑒 𝑘 + 2 𝑒 ℎ 2 + 4 𝑒 ℎ 𝑘 + 𝑒 𝑘 2 .

Figure 12.4 The tangent plane matches the value and the slope; the quadratic approximation also matches the bend. Both are built entirely from data at the point—first the gradient, then the Hessian:

𝑓 (⃗ 𝑝 + ⃗ ℎ) \approx 𝑓 (⃗ 𝑝) + \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ ℎ + 12 ⃗ ℎ 𝖳 𝐻 𝑓 (⃗ 𝑝) ⃗ ℎ

. Switch tabs to swap the model standing over the point, and watch the readout: near the point the quadratic's worst miss runs a whole order smaller than the plane's. The slice alongside is the familiar picture from one variable—tangent line, then Taylor parabola.

The quadratic approximation remembers more of the local shape than the tangent plane, but we will postpone the classification into hills, bowls, and saddles until Chapter 14. First, the next chapter develops the geometry already hidden inside the linear term $\nabla 𝑓 (⃗ 𝑝) \cdot ⃗ ℎ$ .