Download 12 Gradients and optimization and more Exercises Vector Analysis in PDF only on Docsity!
12 Gradients and optimization
12.1 Vectors
Think of a vector as an arrow drawn from one point in the plane or three dimensional space to another. The arrow from (1, 1) to (2, 3) is shown in the figure. The only tricky thing about the definition is that we don’t care where the arrow is drawn, we only care about its magnitude (length) and direction. So for example the dashed arrow represents the same vector, started at the point (5/ 2 , 0) instead of (1, 1). In other words, the vector represents the move from the beginning to the end of the arrow, regardless of the absolute location of the beginning point.
The vector of unit length in the x-direction is called ˆi, the vector of unit length in the y-direction is called ˆj, and, if we’re in three dimensions, the vector of unit length in the z-direction is called kˆ. A vector that goes a units in the x-direction and b units in the y-direction is denoted aˆi + bˆj. It’s called that because you can add vectors and multiply them by real numbers (see definition below). For example, the vector in the picture should be written ˆi + 2ˆj.
Definition of adding vectors. First make one move, then make the other. You can do this by sliding one of the arrows (don’t rotate it!) so it starts where the other one ends, then following them both. If you add aˆi + bˆj to cˆi + dˆj you get (a + c)ˆi + (b + d)ˆj.
Definition of multiplying a vector by a real number. Don’t change the direction, just multiply the length. As a formula: multiply aˆi + bˆj by c you get acˆi + bcˆj. This easy formula hides an important fact: if you mutliply both the ˆi and ˆj coe cients by the same real number, the direction doesn’t change. That’s why the two vectors in the right-hand figure below are on top of each other.
The left-hand side of the figure below shows the vector ˆi + 2ˆj being added, tip to tail, to the vector ˆi ˆj. The result is the vector 2ˆi + ˆj show by the dotted arrow. In the right-hand figure, the vector ˆi ˆj is multiplied by the real number
p 6 which is a little under 21/2.
The length of a vector can be computed by the Pythagorean Theorem. The length of aˆi + bˆj is
p a 2 + b 2. For example, the vector ˆi + 2ˆj which appears in the previous figures has length
p
- The length of the vector v is denoted |v|. A unit vector is any vector whose length is 1. Often we want to know a unit vector in a given direction: what vector, having the same direction as v, has length 1? Answer: divide v by |v| (that is, multiply v by the reciprocal of its length). Self-check: what is the unit vector in the direction of our favorite example vector, v = ˆi + 2ˆj? The answer is posted in a link on Canvas (first student who actually wants to look at it, tell me and I’ll activate the link).
12.2 The gradient
Let z be a function of x and y. Think of this for now as the elevation at a point x units east and y units north of a central point. Pick a point (x 0 , y 0 ), let a = (@z/@x)(x 0 , y 0 ) and let b = @z/@y)(x 0 , y 0 ). Using these we can figure out the rate of elevation increase for a hiker traveling on the path (x(t), y(t)). By the multivariate chain rule, if the hiker is at the position (x 0 , y 0 ) at some time t 0 , then the rate of increase of the hiker’s elevation at time t 0 will be ax 0 (t) + by 0 (t) evaluated at t = t 0.
Here’s the important point. If we calculate a and b just once, we can figure out the rate of elevation gain of any hiker traveling with any speed in the x- and y-directions. The vector aˆi + gˆj is called the gradient of z at the point (x 0 , y 0 ) and is denoted rz(x 0 , y 0 ) or just |rz|. This definition is given in a box in the middle of page 833 in Section 14.5 of the textbook:
rz(x 0 , y 0 ) =
@z @x
(x 0 , y 0 ) ˆi +
@z @y
(x 0 , y 0 ) ˆj.
This leads to the idea of the directional derivative: what is the rate of elevation gain per unit traveled in any direction? The key here is “per unit traveled”. The unit vector w in the direction making an angle of ✓ with the positive x-direction is (cos ✓)ˆi + (sin ✓)ˆj. Therefore, a hiker traveling at unit speed in this direction gains elevation at the rate of a cos ✓ + b sin ✓. That’s the dot product rz · w. THIS IS THE MAIN REASON WE COVER VECTORS AND DOT PRODUCTS IN THIS COURSE.
Here are some conclusions you can draw from all of this. Let L = |rz(x 0 , y 0 )| be the length of the gradient vector of z at the point (x 0 , y 0 ). Now consider all directions the hiker could possibly be traveling: which one maximizes the rate of elevation gain? Let ↵ be the angel between the gradient vector and the hiker’s direction in the x-y plane. We have just seen that the rate of elevation gain per unit motion in the direction w is rz · w. The length of rz is L and the lngth of w is 1, so by formula (12.1), the dot product is L cos ↵. This cosine is at most 1 and is maximized when the angle is zero, in other words, when the hiker’s direction is parallel to the gradient vector. In that case the directional derivative is L. If the hiker is going in a direction making an angle ↵ with the gradient then the rate of elevation gain per unit distance traveled is L cos ↵. If ↵ is a right angle then this rate is zero. We can summarize these observations in a theorem, which constitutes more or less the “Properties of the directional derivative” stated in a box on page 834.
Gradient Theorem:
(i) The direction of greatest increase of a function z(x, y) at a point (x 0 , y 0 ) is the direction of its gradient vector rz(x 0 , y 0 ). The rate of increase per unit distance traveled in that direction is the length of the gradient vector which is given by
L =
s @z @x
(x 0 , y 0 ) 2 +
@z @y
(x 0 , y 0 ) 2.
(ii) In general the directional derivative in a direction making angle ↵ with respect to the gradient direction is equalt to L cos ↵. (iii) In particular, when ↵ is a right angle, we see that the rate of elevation increase in direction ↵ is zero.
This theorem is, more or less what’s in the box on page 834 entitled “Properties of the directional derivative”. Mull it over for a minute. By computing partial derivatives, we can stake out the‘ direction of maximum ascent, and it will have the property that the direction of zero elevation gain is at right angles to it (also, the direction of maximal descent is exactly opposite). Remember level curves? Along these, the elevation is constant. Therefore, traveling in these directions makes the rate of elevation gain zero. We see that the tangent to the level curve must be in the zero gain direction, that is, perpendicular to the gradient. This is shown in Figure 14.31 on page 835. A real life illustration is shown in the picture on page 831 of the textbook. A contour map shows contours of an actual mountainside in Yosemite National Park. These are perpendicular to the directions of steepest ascent and descent. You can see this because streams typically flow in the directions of steepest descent. The streams and the level contours are marked on the map and do, indeed, look perpendicular.
Some rules for computing
We won’t need a lot of rules for computing gradients because we’ll always be able to compute them by hand but it is good to look them over once. They’re collected in a box on page 836 of the textbook. Basically all the rules that work for derivatives work for gradients because in each component separately (the ˆi component, etc.) the gradient is a kind of all-encompassing partial derivative, and partial derivatives obey these laws.
find the critical points where the derivative of the readout is zero; the maximum will have to occur at one of these places; check them all. Computationally, the tricky part is to describe the curve in equations, then use those equations to compute the derivative along the curve.
The description of a curve can take one of three forms. It could be given by some function y = g(x). It could be given parametrically by ((x(t), y(t)). Finally, and most commonly, could be given implicitly, meaning it is the solution set to the equation H(x, y) = 0 for some function H. We treat these in the order: parametric, function, implicit, because each computation relies on the previous one.
Parametric case: the derivative along (x(t), (y(t)).
If the curve is paramterized as (x(t), y(t)), then the derivative of f along is just rf · v where v is the velocity vector x 0 (t)ˆi + y 0 (t)ˆj. In this case, finding the points where the derivative of f along vanishes boils down to solving
x 0 (t)
@f @x
@f @y
Self-check: what does it mean that the derivative of f along the curve (x(t), y(t)) is given by (12.3)? This formula computes the rate of change of what with respect to what?
Function case: the derivative along y = g(x).
If is paramtrized by y = g(x) then you can use the parametric description x = x, y = g(x) so that this equation becomes
@f @x
@f @y
Self-check: again, this is the rate of change of what with respect to what?
Implicit case: the derivative along H(x, y) = 0.
Finally, suppose that is given implicitly by H(x, y) = 0. Recall that we know how to find the slope dy/dx of the tangent line to the level curve H(x, y) = 0. By implicit di↵erentiation, we computed dy/dx = H (^) x /H (^) y. Therefore we can apply equation (12.4) with g 0 (t) = H (^) x /H (^) y. We get @f /@x (H (^) x /H (^) y )@f /@y = 0, which simplifies slightly to
H (^) y
@f @x
H (^) x
@f @y
IMPORTANT GEOMETRIC INTERPRETATION OF (12.5):
The gradient of H is H (^) x^ ˆi + H (^) y^ ˆj. The gradient of f is f (^) x^ ˆi + f (^) y^ ˆj. The test for these to be parallel is given by applying (12.2) to these two vectors. This results precisely in (12.5). In other words:
The critical points of f along a level curve of H are those points where the gradients of f and H are parallel.
Pictorial example: The figure shows a black constraint curve, H(x, y) = 0, along with contours for another function f (x, y). The maximum of f along the curve H(x, y) = 0 is the place where the level curves, when you move from higher to lower, just hit the black curve. At this point, the curves are tangent and the gradients are parallel. The single arrow represents the directions of both gradients.
Application
Let’s go back to the pizza and FroYo example from Unit 11.4, but without numbers. Let H(x, y) be the utility of a consumer who gets x ounces of pizza and y pints of FroYo. Let f (x, y) be the cost to me of producing x ounces of pizza and y pints of Froyo. For my ten dollar family bargain, I need to o↵er a pair that is on the curve H(x, y) = c because that’s what Burger Chef is o↵ering and I will lose customers if my pizza-FroYo combo is less desirable than theirs. But my function f is di↵erent from Burger Chef’s because my prodution line is di↵erent. Question: what bundle should I o↵er?
In mathematical terms, What value of (x, y) on the curve H(x, y) = c minimizes f (x, y)? We just saw the answer to that: it is either an endpoint of the curve or a place where rf is parallel to rH. Let’s interpret the parallel gradients in economic terms. Parallel gradients at a point occur when the tangent lines to the level curves are the same at that point. These tangents tell me the marginal rate of substitution. Remember the FroYo example. The tangent to H(x, y) = 30 at the point (60, 1 /2) tells me the marginal rate of substitution. Consumers at this point are indi↵erent between another ounce of pizza and another 1/120 point of FroYo. The tangent to the level curve of f at this point tells me the rate of substitution for costs: how many extra pints of FroYo can I make from the cost savings on each fewer ounce of pizza? If the two slopes are not the same, then I can slide along the customers indi↵erence curve one direction or the other, decreasing my costs while maintaining the same customers. The only way I can be at the minimim cost point on the consumers’ indi↵erence curve is to be at a point where the slopes are parallel.
Example: Using the numbers H(x, y) = xy from the original pizza and FroYo example, suppose my cost function is a simple linear function: it coses 10 cents to produce each ounce of pizza and $1 for each pint of FroYo. Thus f (x, y) = (0.1)x + y. The gradient of a linear function is constant: rf = (1/10)ˆi + ˆj. The gradient of H is yˆi + xˆj. These are parallel when y x/10 = 0. At what point on the curve H(x, y) = 30 does this occur? We solve
x = 10 y xy = 30
to get y =
p 3 and x = 10
p
- Look up the approximate value
p 3 = 1. 732... on your cheatsheet. In other words, the optimum combo meal for me to sell is (roughly) a 17 and a third ounce pizza and a pint and three quarters of FroYo.
