Chapter 13

Gradient and Directional Derivatives

This set of notes dives deeper into the geometry of the first derivative. As a review, recall that the gradient packages the first partial derivatives:

\nabla 𝑓 = ⟨ 𝑓 𝑥 1, \dots, 𝑓 𝑥 𝑛 ⟩ .

Chapter 12 showed that this vector determines the differential

𝑑 𝑓 ⃗ 𝑝 (⃗ 𝑣) = \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ 𝑣 .

We now use that linear derivative to measure slopes in every direction and to understand what the gradient means geometrically.

13.1Directional Derivatives

We have seen that $𝜕 𝑥 𝑓$ and $𝜕 𝑦 𝑓$ measure the slope of a multivariable function in the $𝑥$ and $𝑦$ directions, respectively. But what is its rate of change in the direction of an arbitrary unit vector $⃗ 𝑢$ ?

Definition 13.1 (Directional Derivative). The derivative of $𝑓$ at $⃗ 𝑝$ in the direction of a unit vector $⃗ 𝑢$ is denoted $𝐷 ⃗ 𝑢 𝑓 (⃗ 𝑝)$ and is defined by the limit

𝐷 ⃗ 𝑢 𝑓 (⃗ 𝑝) = l i m ℎ \to 0 𝑓 (⃗ 𝑝 + ℎ ⃗ 𝑢) - 𝑓 (⃗ 𝑝) ℎ .

Computing this seems difficult. But we can use the Fundamental Strategy of Calculus to save the day! If $𝑓$ is differentiable at $⃗ 𝑝$ , its linear approximation says

𝑓 (⃗ 𝑝 + ℎ ⃗ 𝑢) - 𝑓 (⃗ 𝑝) = ℎ \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ 𝑢 + 𝐸 (ℎ ⃗ 𝑢),

where the error is small compared with $| ℎ |$ . Dividing by $ℎ$ and taking the limit gives

𝐷 ⃗ 𝑢 𝑓 (⃗ 𝑝) = 𝑑 𝑓 ⃗ 𝑝 (⃗ 𝑢) = \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ 𝑢 .

That is, the directional derivative is just a linear combination of the basic slopes we already know!

Theorem 13.2 (Directional Derivative). If $⃗ 𝑢 = ⟨ 𝑎, 𝑏 ⟩$ is a unit vector, then

𝐷 ⃗ 𝑢 𝑓 (𝑥, 𝑦) = 𝑎 𝑓 𝑥 (𝑥, 𝑦) + 𝑏 𝑓 𝑦 (𝑥, 𝑦) .

All of this carries over to three or higher dimensions: if $⃗ 𝑢 = ⟨ 𝑎, 𝑏, 𝑐 ⟩$ is a unit vector and $𝑓 (𝑥, 𝑦, 𝑧)$ is a three variable function then

𝐷 ⃗ 𝑢 𝑓 = 𝑎 𝑓 𝑥 + 𝑏 𝑓 𝑦 + 𝑐 𝑓 𝑧 .

Any time we have a collection of sums of products of terms we should think, "Is this a dot product?" And in this case it is! If we factor the above equations into a dot product we see the directional derivative is related directly to the gradient.

Theorem 13.3 (Directional Derivatives and the Gradient). For a unit vector $⃗ 𝑢$ ,

𝐷 ⃗ 𝑢 𝑓 (⃗ 𝑝) = \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ 𝑢 .

Figure 13.1 Chapter 11 held one variable fixed; nothing stops the glass from standing at any other angle. Aim the direction

⃗ 𝑢

—or let it sweep—and the sheet cuts its own slice through the point, drawn flat alongside with its tangent line. The slope of that line is the directional derivative, and it costs nothing new:

𝐷 ⃗ 𝑢 𝑓 = \nabla 𝑓 \cdot ⃗ 𝑢 = 𝑎 𝑓 𝑥 + 𝑏 𝑓 𝑦

, the two slopes already computed, recombined. When

⃗ 𝑢

lines up with an axis, this is Chapter 11's figure exactly.

The unit-vector requirement matters. For example, suppose

𝑓 (𝑥, 𝑦) = 𝑥 2 + 𝑦 2

and we want the directional derivative at $(1, 2)$ in the direction $⃗ 𝑣 = ⟨ 3, 4 ⟩$ . First normalize the supplied direction:

⃗ 𝑢 = ⃗ 𝑣 ‖ ⃗ 𝑣 ‖ = ⟨ 35, 45 ⟩ .

Since $\nabla 𝑓 (1, 2) = ⟨ 2, 4 ⟩$ ,

𝐷 ⃗ 𝑢 𝑓 (1, 2) = ⟨ 2, 4 ⟩ \cdot ⟨ 35, 45 ⟩ = 225 .

We reserve $𝐷 ⃗ 𝑢 𝑓$ for a unit direction. The differential can still be applied to any vector:

𝑑 𝑓 ⃗ 𝑝 (⃗ 𝑣) = \nabla 𝑓 (⃗ 𝑝) \cdot ⃗ 𝑣,

but if $‖ ⃗ 𝑣 ‖ \neq 1$ , this quantity includes both the direction and the length of the proposed step.

13.2Geometry of the Gradient

The gradient is not merely a list of derivatives. For a scalar field on the plane, it assigns a vector in the domain to every point of the domain.

Figure 13.2 The gradient, drawn where it lives. At every point of the domain,

\nabla 𝑓 = ⟨ 𝑓 𝑥, 𝑓 𝑦 ⟩

is a vector in the domain—the floor under the graph, where every arrow in this picture lies flat. The derivative of a scalar field is a field of arrows. Read the surface above, read the arrows below, and form your own suspicions about where they point and where they grow long. The next figures turn those suspicions into theorems.

Since we know the interpretation of dot products in terms of angles, we can use the directional derivative formula above to help us understand the direction the gradient points in.

If a unit vector $⃗ 𝑢$ makes angle $𝜃$ with the gradient, we see the directional derivative in direction $⃗ 𝑢$ is given by

𝐷 ⃗ 𝑢 𝑓 = \nabla 𝑓 \cdot ⃗ 𝑢 = ‖ \nabla 𝑓 ‖ ‖ ⃗ 𝑢 ‖ c o s 𝜃 = ‖ \nabla 𝑓 ‖ c o s 𝜃 .

This tells us a lot!

Theorem 13.4 (Direction and Magnitude of the Gradient). At a point where $\nabla 𝑓 \neq ⃗ 0$ :

The gradient points in the direction of maximal directional derivative.
Its magnitude is the directional derivative in that direction.
The opposite direction gives the steepest decrease.
In every direction orthogonal to the gradient, the directional derivative is zero: the function is not changing to first order.

In formulas,

⃗ 𝑢 s t e e p e s t i n c r e a s e = \nabla 𝑓 ‖ \nabla 𝑓 ‖, 𝐷 ⃗ 𝑢 𝑓 = ‖ \nabla 𝑓 ‖,

while

⃗ 𝑢 s t e e p e s t d e c r e a s e = - \nabla 𝑓 ‖ \nabla 𝑓 ‖, 𝐷 ⃗ 𝑢 𝑓 = - ‖ \nabla 𝑓 ‖ .

Figure 13.3 The rotating slice again, now with the gradient's whole line lit up on the floor. The pane beside it drops the slice and keeps only slopes: every direction through the point contributes one line to the faint fan, the current direction sweeps through it in orange, and the gradient's own line stays put in its own color at the top of the fan. The sweeping line climbs, reaches the gradient's exactly when the glass lands on the lit line—and never passes it:

𝐷 ⃗ 𝑢 𝑓 = ‖ \nabla 𝑓 ‖ c o s 𝜃

, so no direction beats

‖ \nabla 𝑓 ‖

, and downhill is steepest against it.

At a point where $\nabla 𝑓 = ⃗ 0$ , the gradient has no direction and all first directional derivatives are zero. Such a point is called a critical point; Chapter 14 studies what the function can do there.

One consequence of zero change in perpendicular directions is so useful that it gets its own theorem box.

Theorem 13.5 (The Gradient and Level Sets). At a regular point $⃗ 𝑝$ of a level set, where $\nabla 𝑓 (⃗ 𝑝) \neq ⃗ 0$ , the gradient vector is orthogonal to the level set and points in the direction of increase.

This is very helpful for understanding a function from its gradient, as it lets us convert between level-set understanding and gradient understanding!

When level sets are close to each other, that means the function is steeply increasing or decreasing, so the gradient is long. When level sets are far apart, that means the function is only slowly changing, so the gradient is short.

More precisely, suppose neighboring contours differ by the same value $| Δ 𝑐 |$ . Locally, their spacing in the normal direction is approximately

| Δ 𝑐 | ‖ \nabla 𝑓 ‖ .

Thus, for equally spaced contour values, crowded contours indicate a long gradient and widely separated contours indicate a short gradient. Without equal contour intervals, visual density alone does not determine the gradient's magnitude.

Figure 13.4 The gradient on the map. It crosses the level curve through its point at a right angle—uphill has no reason to lean along the coastline—and its length answers to the crowding: with contours drawn every

Δ 𝑐

, the next one lies about

| Δ 𝑐 | / ‖ \nabla 𝑓 ‖

away, so crowded contours mean a strong gradient and sparse ones a weak one. Drag the point across the island and watch both facts hold everywhere.

Figure 13.5 The two portraits of one function, laid on top of each other: every level set, and the whole gradient field crossing them. Each arrow stands perpendicular to the curve it sits on, and one linear scale draws them all, so their lengths can be compared: the arrows are long exactly where contours drawn at equal value increments crowd and short where they spread out. That trade is the thing to take away—crowded curves, strong gradient; sparse curves, weak one—and the graph alongside shows why: crowding is steepness. Swap the function and the pattern survives every swap.

Because the gradient is a normal vector to level sets we can use the gradient to derive the equation for a tangent plane to a surface!

If $⃗ 𝑝$ is a point on a plane and $⃗ 𝑛$ is a normal vector to the plane, the equation

⃗ 𝑛 \cdot (⃗ 𝑥 - ⃗ 𝑝) = 0

describes the plane because it says $⃗ 𝑥$ lies in the plane so long as the vector connecting it to $⃗ 𝑝$ is orthogonal to $⃗ 𝑛$ .

Now consider a regular point $⃗ 𝑝$ on the level surface

𝐹 (𝑥, 𝑦, 𝑧) = 𝑘 .

The gradient $\nabla 𝐹 (⃗ 𝑝)$ is the normal vector to the surface, so we can directly write down the tangent plane:

\nabla 𝐹 (⃗ 𝑝) \cdot (⃗ 𝑥 - ⃗ 𝑝) = 0 .

Chapter 12 found tangent planes to graphs $𝑧 = 𝑓 (𝑥, 𝑦)$ using linearization. Here we obtain tangent planes to implicit surfaces $𝐹 (𝑥, 𝑦, 𝑧) = 𝑘$ using the geometry of the gradient. These are two descriptions of the same local linear idea.

But knowing the normal vector also allows us to compute other geometric quantities of interest: such as the normal line: the parametric line which intersects a level set orthogonally.

This is also immediate: if we know a point $⃗ 𝑝$ and a direction vector $⃗ 𝑣$ , the associated line is $⃗ ℓ (𝑡) = ⃗ 𝑝 + 𝑡 ⃗ 𝑣$ . Here, the direction vector is $\nabla 𝐹 (⃗ 𝑝)$ , so the normal line is

⃗ ℓ (𝑡) = ⃗ 𝑝 + 𝑡 \nabla 𝐹 (⃗ 𝑝) .

Figure 13.6 One dimension up, the same fact writes equations for you. For a level surface

𝐹 (𝑥, 𝑦, 𝑧) = 𝑘

, the gradient

\nabla 𝐹

stands normal to the surface—watch it ride along, carrying the tangent plane

\nabla 𝐹 (⃗ 𝑝) \cdot (⃗ 𝑥 - ⃗ 𝑝) = 0

with it, while the dashed normal line

⃗ ℓ (𝑡) = ⃗ 𝑝 + 𝑡 \nabla 𝐹 (⃗ 𝑝)

runs straight through. Plane and line both fall out of a single gradient evaluation, with no new machinery at all.

Example 13.6 (A Tangent Plane and Normal Line). Compute the tangent plane and the normal line to $𝑥 = 𝑦 2 + 𝑧 2 + 1$ at $(3, 1, - 1)$ .

First, we rearrange so that the surface equation is written as a level set: $𝑥 - 𝑦 2 - 𝑧 2 = 1$ with all the variables on one side. Now we can compute the gradient:

\nabla 𝐹 = ⟨ 1, - 2 𝑦, - 2 𝑧 ⟩ \nabla 𝐹 (3, 1, - 1) = ⟨ 1, - 2, 2 ⟩

This vector and the original point $(3, 1, - 1)$ immediately determine the plane and line:

⟨ 1, - 2, 2 ⟩ \cdot ⟨ 𝑥 - 3, 𝑦 - 1, 𝑧 + 1 ⟩ = 0

𝑥 - 2 𝑦 + 2 𝑧 = - 1

⃗ ℓ (𝑡) = (3, 1, - 1) + 𝑡 ⟨ 1, - 2, 2 ⟩ = (3 + 𝑡, 1 - 2 𝑡, - 1 + 2 𝑡)

The gradient now gives us a complete first-order picture: it computes every directional derivative, points uphill, measures the steepest rate, and stands normal to level sets. In the next chapter, we investigate the points where this first-order picture goes silent because $\nabla 𝑓 = ⃗ 0$ .