For a function \(f: \mathbb{R} \to \mathbb{R}\) at \(a \in \mathbb{R}\), differentiability is given by:
\[ \lim_{x \to a} \frac{f(x)-f(a)}{x-a} = \lambda \]This can't happen in \(\mathbb{R}^n\) (cannot divide by vector). We rewrite this to show straight-line approximation near \(a\):
\[ \lim_{x \to a} \frac{f(x)-f(a)}{x-a} - \lambda = 0 \implies \lim_{x \to a} \frac{f(x)-f(a)-\lambda(x-a)}{x-a} = 0 \]Call \(\phi(x) = f(x) - f(a) - \lambda(x-a)\). We have \(\frac{\phi(x)}{x-a} \to 0\) as \(x \to a\). This is the equivalent formulation of the existence of the derivative \(f'(a) = \lambda\) at \(x=a\).
\[ f(x) \approx [f(a) + \lambda(x-a)] + \phi(x) \]Here, \(x-a \to 0\) as \(x \to a\), and \(\phi(x)\) goes to zero faster than \(x-a \to 0\).
Define a linear map \(\Lambda: \mathbb{R} \to \mathbb{R}\) such that \(\Lambda(x) = \lambda x\). It satisfies \(\Lambda(x+y) = \Lambda(x) + \Lambda(y)\) and \(\Lambda(kx) = k\Lambda(x)\). Then \(f(x) \approx f(a) + \Lambda(x-a)\).
A function \(f: \mathbb{R}^n \to \mathbb{R}^m\) is differentiable at \(a \in \mathbb{R}^n\) if there exists a linear transformation \(\Lambda: \mathbb{R}^n \to \mathbb{R}^m\) such that:
\[ \lim_{|x-a| \to 0} \frac{|f(x)-f(a)-\Lambda(x-a)|}{|x-a|} = 0 \]Let \(\Phi(x) = f(x) - f(a) - \Lambda(x-a)\). Then \(\frac{|\Phi(x)|}{|x-a|} \to 0\) as \(x \to a\).
Observe that this implies \(|\Phi(x)| = \frac{|\Phi(x)|}{|x-a|} |x-a| \to 0\) as \(x \to a\).
\(f(x) \approx f(a) + \Lambda(x-a)\) for a linear function near \(a\). Also, notice that since \(\lim_{x \to a} \frac{|\Phi(x)|}{|x-a|} = 0\), for \(x\) sufficiently close to \(a\), \(|\Phi(x)| < |x-a|\).
Rewriting the previous expression in terms of \(h = x-a\):
\[ \lim_{h \to 0} \frac{|f(a+h) - f(a) - \Lambda(h)|}{|h|} = 0 \]Hence, \(f(a+h) = f(a) + \Lambda(h) + \Phi(a+h)\). (Note: \(\Lambda\) is continuous at \(h=0\)).
Let \(f: \mathbb{R}^n \to \mathbb{R}^m\) be differentiable at \(a\). We have:
\[ \lim_{h \to 0} \frac{|f(a+h) - f(a) - \lambda(h)|}{|h|} = 0 \]where \(\lambda: \mathbb{R}^n \to \mathbb{R}^m\) is a linear map.
If \(f\) is differentiable at \(a\), then \(f\) must be continuous at \(a\).
Proof: We want to show \(f(a+h) \to f(a)\) as \(h \to 0\). We have \(f(a+h) - f(a) = \Phi(h) + \lambda(h)\).
\[ |f(a+h) - f(a)| = |\Phi(h) + \lambda(h)| \le |\Phi(h)| + |\lambda(h)| \]Since \(f\) is differentiable, \(\frac{|\Phi(h)|}{|h|} \to 0\) as \(h \to 0\), which implies \(|\Phi(h)| \to 0\). Since \(\lambda\) is a linear transformation, it is continuous, and \(\lambda(0) = 0\). Thus \(\lambda(h) \to 0\) as \(h \to 0\).
Therefore, \(|f(a+h) - f(a)| \to 0\), making \(f\) continuous at \(a\). ∎
The linear transformation \(\lambda\) is uniquely determined.
Proof: Suppose there is another linear map \(\mu: \mathbb{R}^n \to \mathbb{R}^m\) such that \(\lim_{h \to 0} \frac{|f(a+h) - f(a) - \mu(h)|}{|h|} = 0\).
\[ \frac{|\lambda(h) - \mu(h)|}{|h|} = \frac{|(f(a+h) - f(a) - \mu(h)) - (f(a+h) - f(a) - \lambda(h))|}{|h|} \] \[ \le \frac{|\Phi(h)| + |\Psi(h)|}{|h|} \to 0 \text{ as } |h| \to 0 \]For any \(x \in \mathbb{R}^n\) and \(t \in \mathbb{R}\), take \(h = tx\). As \(t \to 0\), \(h \to 0\).
\[ \frac{|\lambda(tx) - \mu(tx)|}{|tx|} = \frac{|t\lambda(x) - t\mu(x)|}{|t||x|} = \frac{|t| |\lambda(x) - \mu(x)|}{|t||x|} = \frac{|\lambda(x) - \mu(x)|}{|x|} \to 0 \text{ as } t \to 0 \]But \(\frac{|\lambda(x) - \mu(x)|}{|x|}\) is independent of \(t\). Thus it must be exactly \(0\). Therefore, \(\lambda(x) = \mu(x)\) for all \(x \in \mathbb{R}^n\). ∎
The unique linear map \(\lambda\) is called the derivative of \(f\) at \(a\) and is denoted by \(Df(a) = \lambda\).
- Suppose \(f(x)\) is constant \(\forall x \in \mathbb{R}^n\). Then \(f(a+h) = f(a)\). Take \(\lambda(h) = 0\), then \(\Phi(h) = 0 \, \forall h\). Thus \(Df(a) = 0 \, \forall a \in \mathbb{R}^n\).
- Let \(T: \mathbb{R}^n \to \mathbb{R}^m\) be a linear transformation. \(T(a+h) - T(a) = T(a) + T(h) - T(a) = T(h)\). Thus \(T(a+h) - T(a) - T(h) = 0 \, \forall h\). Taking \(\lambda(h) = T(h)\), we get \(DT(a) = T(h) \, \forall a \in \mathbb{R}^n\). The derivative of a linear transformation is the linear transformation itself.
\(Df(a): \mathbb{R}^n \to \mathbb{R}^m\). The matrix representation with respect to the standard basis is \([Df(a)]_{\text{std basis}} = f'(a)\), which is the Jacobian matrix of \(f\) at \(a\).
Suppose \(f: \mathbb{R}^n \to \mathbb{R}^m\) and \(g: \mathbb{R}^m \to \mathbb{R}^p\). If \(f\) is differentiable at \(a \in \mathbb{R}^n\) and \(g\) is differentiable at \(f(a) = b \in \mathbb{R}^m\), then \(g \circ f: \mathbb{R}^n \to \mathbb{R}^p\) is differentiable at \(a\), and:
\[ D(g \circ f)(a) = Dg(f(a)) \circ Df(a) \]and \((g \circ f)'(a) = g'(f(a)) \cdot f'(a)\).
Proof: Let \(Df(a) = \lambda\) and \(Dg(f(a)) = \mu\). We evaluate:
\[ (g \circ f)(a+h) - (g \circ f)(a) - \mu \circ \lambda(h) \]Let \(\Phi(h) = f(a+h) - f(a) - \lambda(h) \implies f(a+h) - f(a) = \Phi(h) + \lambda(h)\).
Let \(\Psi(k) = g(b+k) - g(b) - \mu(k)\).
Letting \(k = f(a+h) - f(a)\), we have:
Observe how for \(f(a+h) - f(a) \neq 0\):
\[ \frac{|g(f(a+h)) - g(f(a)) - \mu(f(a+h) - f(a))|}{|f(a+h) - f(a)|} \cdot \frac{|f(a+h) - f(a)|}{|h|} \]Now, \(\frac{g(f(a) + (f(a+h)-f(a))) - g(f(a)) - \mu(f(a+h)-f(a))}{|f(a+h)-f(a)|} = \frac{g(b+k) - g(b) - \mu(k)}{|k|} \to 0\) as \(k \to 0\) (Since \(g\) is diff.).
Now \(\frac{|f(a+h) - f(a)|}{|h|} \le \frac{|f(a+h) - f(a) - \lambda(h)|}{|h|} + \frac{|\lambda(h)|}{|h|} \le M\) near \(h=0\).
Now \(\frac{|\Psi(k)|}{|k|} = \frac{|g(f(a+h)) - g(f(a)) - \mu(f(a+h) - f(a))|}{|h|} \cdot \frac{|h|}{|k|} \to 0\) as \(|h| \to 0\). ∎
(a) \(S: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}^n\), \(S(x,y) = x+y\) is a differentiable map and \(DS(a,b) = S\) since \(S\) is linear.
(b) Let \(p: \mathbb{R} \times \mathbb{R} \to \mathbb{R}\) by \(p(x,y) = xy\). Then \(p\) is differentiable.
Proof for (b): \(p(a+h, b+k) - p(a,b) = (a+h)(b+k) - ab = ab + ak + hb + hk - ab = ak + hb + hk\).
\[ \frac{|p(a+h, b+k) - p(a,b) - (ak+hb)|}{\sqrt{h^2+k^2}} = \frac{|hk|}{\sqrt{h^2+k^2}} \le |k| \to 0 \text{ as } (h,k) \to (0,0) \]Observe that the map \(\mathbb{R}^2 \to \mathbb{R}\) is \(\binom{h}{k} \mapsto ak+bh\), which is a linear map. Hence, \(Dp(a,b)(h,k) = bh + ak\). ∎