Lagrange multipliers: constrained optimization
The method of Lagrange multipliers offers a systematic approach for optimizing functions when constraints are present. Instead of searching only where the unconstrained gradient of a function f(x_1, \dots, x_n) vanishes, I address situations where the variables are tied by one or more constraint equations. If I want to extremize f subject to conditions g_k(x_1, \dots, x_n) = c_k, the optimal points occur where the gradient of f is a linear combination of the gradients of all g_k:
\nabla f = \sum_{k=1}^{m} \lambda_k \nabla g_k
where the \lambda_k are the Lagrange multipliers. Defining the auxiliary function:
\mathcal{L}^\prime(x_1, \dots, x_n, \lambda_1, \dots, \lambda_m) = f(x_1, \dots, x_n) - \sum_{k=1}^{m} \lambda_k (g_k(x_1, \dots, x_n) - c_k)
I solve the system:
\frac{\partial \mathcal{L}^\prime}{\partial x_i} = 0, \qquad \frac{\partial \mathcal{L}^\prime}{\partial \lambda_k} = 0
for all variables and multipliers. The latter set restores the constraint equations themselves.
This technique applies not just to functions, but also to functionals, which often appear as integrals involving unknown functions. Suppose I must extremize:
I[y] = \int_a^b F(x, y, y^\prime) \,\mathrm dx
subject to an additional integral constraint:
J[y] = \int_a^b G(x, y, y^\prime) \,\mathrm dx = \text{constant}
Introducing a Lagrange multiplier \lambda, I define a new functional:
H[y] = I[y] + \lambda J[y] = \int_a^b (F(x, y, y^\prime) + \lambda G(x, y, y^\prime))\,\mathrm dx
and then apply the Euler-Lagrange equation:
\frac{\partial \mathcal{L}_{new}}{\partial y} - \frac{\mathrm d}{\mathrm dx} \left( \frac{\partial \mathcal{L}_{new}}{\partial y^\prime} \right) = 0
with \mathcal{L}_{new} = F(x, y, y^\prime) + \lambda G(x, y, y^\prime). The solution that satisfies both the Euler-Lagrange equation and the original integral constraint gives the extremizing function y(x) and the corresponding multiplier \lambda.
For more insights into this topic, you can find the details here.