From Taylor's Theorem and Local Extrema to the Inverse Function Theorem
- Cristian Mesiano
- 3 days ago
- 5 min read
In this post, we'll explore two pillars of multivariable calculus:
The first is nothing but the extension in ℝ^n of the very well known Taylor series expansion.
The second gives the entire numerical simulation world a simple test to check whether a numerical approximation is feasible or not: the Inverse Function Theorem.
Taylor's Theorem
We spent quite some time in previous posts exploring the linearisation of functions in ℝ^n. However, as you might imagine, we can improve our approximation by accounting for the quadratic term, just as we do in ℝ with the Taylor series expansion. Without losing generality, we can introduce some auxiliary functions to reduce the problem to the one-dimensional case.
The derivative of f(γ(t)) can be computed via the chain rule as shown in the Foglietto nr 2.
It's beautiful how the Hessian emerges from a further application of the chain rule!
Ok now we got a second derivative formulation expressed as function of one-parameter, we just need to apply the Taylor series expansion formula to get an elegant representation for the multidimensional case!
The introduction of the multi-index notation makes the expression more compact and readable.
Local Extrema
The rules to determine local minima and maxima of functions in R^n now depend on the characteristics of the Hessian matrix. In Foglietto Nr. 3 above, we saw that it is symmetric with real entries, so we can apply an important theorem about its eigenvalues. Its proof provides an intuitive explanation of the conditions that link positive and negative definiteness with the type of local extrema.
The Inverse Function Theorem (IFT)
This theorem represents a milestone for every numerical simulation application. Whenever you try to approximate the root of a complex multidimensional function f(x) = y (where x = (x_1, ..., x_n)), for instance with a Newton method, the Inverse Function Theorem gives the green light to proceed. If the determinant of the Jacobian matrix at the true solution x_0 is non-zero determinant, the theorem guarantees that provided your initial guess is close enough your method will successfully find a good approximation of x_0 given y.
Diffeomorphism
Before delving into the IFT, we need to introduce the concept of a diffeomorphism, which relates the differentiability of a function to the differentiability of its inverse.
We can even relate the Jacobian chain rule to the concept of diffeomorphism:
If the function f is a diffeomorphism, then f^-1∘ f = f∘f^-1 = 𝐼. This is nothing but the definition of a bijective function.
Since f is a C^1 diffeomorphism the equivalences represented in Foglietto nr.2 allow us to state the following theorem: If f is a C^1 global diffeomorphism, then J_f(x) is invertible, meaning the determinant of J_f(x) is !=0 for all x in U.
Conversely, the Inverse function theorem states that if the determinant of the Jacobian at the point x_0 is different from zero, then in a neighbourhood of x0 the function f is a diffeomorphism C^1(U,V).
On the global/local asymmetry
On one hand, we have the following: a global diffeomorphism implies a Jacobian that is non-zero everywhere.
On the other hand the Inverse Function Theorem states: If the Jacobian of f is invertible at a point x0 (meaning det(J_f(x0)) != 0), then f is a local diffeomorphism in a neighborhood of x0.
By "zooming in" closely enough to that point, f behaves just like a global diffeomorphism. The asymmetry here is that, unfortunately, we have no idea how f behaves far away from x0.
So, one might try to "cheat":
If the Jacobian of f is non-zero everywhere (meaning we have a local diffeomorphism at every single point), does this guarantee a global diffeomorphism?
The answer is NOT. The transition from "local" to "global" is not automatic because the function can lose its injectivity on a large scale (it "loops" or wraps back around on itself).
Just picture any periodic function!
Inverse function Theorem: The Proof
Let's take an intuitive look at the idea behind the theorem: the local linearization performed by the Jacobian makes the inversion feasible through a linear map. Indeed, to move from the locally linear space induced by J_f(x_0) to the linear space of the domain, we just need operations like rescaling, rotations, and reflections!
Step 1: normalisation of the input and of the derivatives
The idea is to shift the input x_0 and the respective output y_0 to the origin. It is important to remark that the two auxiliary functions perform a merely shift on f: the slope remains unaltered.
The same reasoning can be applied to the Jacobian of the function g so that h(0) gives the identity matrix.
The proof of the theorem will be done on the function h(x) where h(x0) = 0 and J_h(x0) = I.
The function Z can be seen as an error function that measures the offset between the Jacobian observed and expected identity matrix I.
An example about Normalisation
Consider the function f(x, y) := {x^2 + y, x + y^2} and the generic point x0 for instance set to x0 = {1, 1}; We have y0 = f(x0[1], x0[2]);
Let's compute f_tilde(x0) = f(x, y) - y0 => f_tilde(1,1) = f(x, y) -(2,2)
Now g(x, y) := f_tilde(x + x0[1], y + x0[2])= f_tilde(x + 1, y + 1)
Then g(0,0) = f_tilde(0+ 1, 0 + 1)= f(1, 1)-(2,2)= (0,0)
As you can see we just operated a translation:
The effect of J on h is local and preserved along the shift (Notice that around f(x_0), the lattice is deformed back to its original shape!)
Notice that we have:
g(x) shifts the image of x0 to 0.
Since h(0) = 0 and J_h(0) = I
The function z(x) = h(x) - x => J_z(x) = J_h(x) - I =>J_z(0) = 0
if we move a bit far from 0 z(x) measures how much h deviates from the identity: it's a kind of error function for h that measures the offset between the Jacobian observed and expected identity matrix I.
Scheme of the proof
Ulisse Dini proved the IFT well before the publication of the Caccioppoli-Banach theorem. This proof scheme is not the original one proposed by Dini, but it is nonetheless less cumbersome and more intuitive.
Step 1: working in a ball
The function h, so the function z is continuous, so in a neighbourhood B of the origin, we can find for sure a closed set where J_z is not zero, and we conveniently limit the size of B so that it's smaller than 1.
In such ball, we can choose two points x, x̃ and link them through a line. The choose of vector form ease the math because it puts everything in a one dimension.
Step 2: mapping the ball into itself
The function a(t) gives us the chance to easily apply the Lagrange theorem.
The Jacobian of the function f composed with g expressed as inner product is explained in the former post available here. This formulation allows us to leverage the Schwarz inequality.
Step 3: The contraction map
We simply apply the Banach-Caccioppoli Theorem as shown in the Schema of Proof section,
Comments