Maths Notes

Just a bunch of mathematical definitions and theorems, unrelated to statistics, which are often used.

Definitions and Properties

Jacobian matrix

For a function $f:\mathbb{R}^n\rightarrow\mathbb{R}^m$, the Jacobian matrix is defined as $$J_f(x)=\left[\frac{\partial f}{\partial x_1}\cdots\frac{\partial f}{\partial x_n}\right]= \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \cdots & \frac{\partial f_1}{\partial x_n}\\ \vdots & \ddots & \vdots\\ \frac{\partial f_m}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_n} \end{bmatrix}.$$

Binomial coefficient

Suppose, $n,k\in\mathbb{N}_0$ are integers with $0\le k\le n$. Then, the binomial coefficient is defined as $$\binom nk=\frac{n!}{k!(n-k)!}$$ and has the following properties:

• $\binom nn=\binom n0=1$.
• $\binom nk=\binom{n-1}{k-1}+\binom{n-1}k$, for $1\le k\le n-1$.
• (Binomial theorem) $(x+y)^n=\sum_{k=0}^n\binom nk x^{n-k}y^k$ holds for any $x,y$ that are elements of a commutative ring.
• $2^n=\sum_{k=0}^n\binom nk$ (from binomial theorem with $x=y=1$).
• There are $\binom nk$ possible results, when picking $k$ elements from a set of $n$ unique elements. If elements can be picked multiple times, there are $\binom{n+k-1}k$ possible outcomes.

Theorems

Inverse function theorem

(For proving the existence of a (local) inverse function) Let $U\subseteq \mathbb{R}^n$ be an open subset, $f:U\rightarrow\mathbb{R}^n$ a continuously differentiable function and $a$ an element of $U$, with $f(a)=:b$. Then, if the Jacobian matrix of $f$ is invertible at $a$, there exist open subsets $a\in U_a\subseteq U$ and $b\in V_b\subseteq\mathbb{R}^n$ with a continuously differentiable bijection $g=f^{-1}:V_b\rightarrow U_a$. In this case, the following equalities hold: $$J_{f^{-1}}(b)=J_g(b)=(J_f(a))^{-1}=(J_f(g(b)))^{-1},$$ where $J_f(x)$ denotes the Jacobian matrix of $f$, evaluated at $x$.
In the special case of $n=1$, this reduces to $$(f^{-1})'(b)=\frac{1}{f'(a)}.$$

Multinomial Theorem

(Generalization of binomial theorem) Suppose that $x_1,\dots,x_m$ are elements of a commutative ring with $0\lt m\in\mathbb N$ and define $n\in\mathbb N_0$.
Then, $$(x_1+\dots+x_m)^n=\sum_{k_1+\dots+k_m=n}\binom n{k_1,\dots,k_m}\prod_{t=1}^m x_t^{k_t}$$ holds, where $$\binom n{k_1,\dots,k_m}=\frac{n!}{k_1!\cdots k_m!}$$ is called a multinomial coefficient.
In combinatorics, it is equivalent to the number of possible outcomes, when putting a total of $n$ unique objects into $m$ different boxes, where $k_1,\dots,k_m$ is the amount of objects placed in each box.