What is the domain and codomain of a norm mapping on a vector space $V$?
It is a function that maps from $V$ to the set of real numbers $\mathbb{R}$.
The property $\|\lambda x\| = |\lambda| \|x\|$ in the definition of a norm is known as _____.
Homogeneity
State the Triangle Inequality axiom for a norm $\| \cdot \|$.
$\|x + y\| \le \|x\| + \|y\|$ for all $x, y \in V$.
According to the norm axioms, what is the necessary and sufficient condition for $\|x\| = 0$?
$x = 0$
What is the lower bound for the value of any norm $\|x\|$?
It must be greater than or equal to zero ($\|x\| \ge 0$).
How is the $L_1$ norm of a vector $x \in \mathbb{R}^n$ calculated?
It is the sum of the absolute values of its components, $\sum_{i=1}^n |x_i|$.
What is the common name for the $L_1$ norm?
The Manhattan Norm.
What is the mathematical definition of the $L_2$ norm for a vector $x \in \mathbb{R}^n$?
$\sqrt{\sum_{i=1}^n x_i^2}$ or $\sqrt{x^\top x}$.
How is the $L_\infty$ norm of a vector $x$ defined?
It is the maximum absolute value among all components of the vector, $\max_i |x_i|$.
In a 2D coordinate system, what geometric shape is formed by the set of points where $\|x\|_1 = 1$?
A diamond (or a rotated square).
In a 2D coordinate system, what geometric shape is formed by the set of points where $\|x\|_2 = 1$?
A circle.
In a 2D coordinate system, what geometric shape is formed by the set of points where $\|x\|_\infty = 1$?
A square.
How is the dot product of two vectors $x, y \in \mathbb{R}^n$ calculated?
$x^\top y = \sum_{i=1}^n x_i y_i$.
Term: Bilinear Mapping
A mapping $f: V \times V \to \mathbb{R}$ that is linear in both its first and second arguments.
What formula represents the linearity of a mapping $f$ in its first argument?
$f(\lambda x + \psi y, z) = \lambda f(x, z) + \psi f(y, z)$.
What formula represents the linearity of a mapping $f$ in its second argument?
$f(x, \lambda y + \psi z) = \lambda f(x, y) + \psi f(x, z)$.
When is a bilinear mapping $f$ considered symmetric?
If $f(x, y) = f(y, x)$ for all $x, y \in V$.
What two conditions must a bilinear mapping satisfy to be considered positive definite?
$f(x, x) > 0$ for all non-zero $x$, and $f(0, 0) = 0$.
Concept: Inner Product
A symmetric, positive definite bilinear mapping $f: V \times V \to \mathbb{R}$.
In machine learning, symmetric positive definite bilinear mappings are essential for defining _____ in Support Vector Machines.
Kernels
Given an ordered basis $(b_1, \dots, b_n)$, how are the elements of the matrix representation $A$ of an inner product defined?
$A_{ij} := \langle b_i, b_j \rangle$.
How is an inner product $\langle x, y \rangle$ expressed using coordinate vectors $\hat{x}, \hat{y}$ and a representation matrix $A$?
$\langle x, y \rangle = \hat{x}^\top A \hat{y}$.
If an inner product is symmetric, what property must its representation matrix $A$ possess?
The matrix $A$ must be symmetric ($A = A^\top$).
What is the defining property of a symmetric positive definite matrix $A$ regarding the quadratic form $x^\top A x$?
$x^\top A x > 0$ for all $x \ne 0$.
What is the term for a symmetric matrix $A$ where $x^\top A x \ge 0$ for all $x$?
Symmetric positive semidefinite.
If $A$ is not necessarily symmetric or positive definite, what can be said about the matrix $\hat{A} = A A^\top$?
$\hat{A}$ must be positive semidefinite.
What is the null space of any symmetric positive definite matrix $A$?
The zero vector set, $\{0\}$.
Why is a symmetric positive definite matrix $A$ always invertible?
Because $Ax \ne 0$ for all $x \ne 0$, implying it has full rank and a null space of $\{0\}$.
What can be concluded about the diagonal elements $a_{ii}$ of a symmetric positive definite matrix $A$?
They are all strictly positive ($a_{ii} > 0$).
How is the norm of a vector $x$ induced by an inner product $\langle \cdot \rangle$?
$\|x\| = \sqrt{\langle x, x \rangle}$.
State the Cauchy-Schwarz inequality for an inner product space.
$|\langle x, y \rangle| \le \|x\| \|y\|$.
How is the distance $d(x, y)$ between two vectors defined in an inner product space?
$d(x, y) := \|x - y\| = \sqrt{\langle x - y, x - y \rangle}$.
What is the mathematical term for a mapping $d$ that defines distance between elements in a vector space?
A metric.
The metric property $d(x, y) = 0$ if and only if $x = y$ is part of which axiom?
Positive definiteness.
State the triangle inequality for a metric $d(x, z)$.
$d(x, z) \le d(x, y) + d(y, z)$.
State the Law of Cosines for vectors $u$ and $v$.
$\|u - v\|^2 = \|u\|^2 + \|v\|^2 - 2\|u\| \|v\| \cos \theta$.
How can the inner product $\langle u, v \rangle$ be expressed in terms of norms and the angle $\theta$ between the vectors?
$\langle u, v \rangle = \|u\| \cdot \|v\| \cos \theta$.
What is the formula to find the cosine of the angle $\theta$ between two non-zero vectors $x$ and $y$?
$\cos \theta = \frac{\langle x, y \rangle}{\|x\| \|y\|}$.
Under what condition are two vectors $x$ and $y$ defined as orthogonal?
They are orthogonal if and only if their inner product is zero ($\langle x, y \rangle = 0$).
What is the unique range of the angle $\theta$ between two vectors as defined in linear algebra?
$\theta \in [0, \pi]$.
In the exercise provided, why is the matrix $A_2 = \begin{pmatrix} 9 & 6 \\ 6 & 3 \end{pmatrix}$ not positive definite?
Its determinant is negative ($27 - 36 = -9$), failing the positive definiteness test.
Given $\langle x, y \rangle = x_1 y_1 - (x_1 y_2 + x_2 y_1) + 2x_2 y_2$, what is the representation matrix $A$ relative to the standard basis?
$A = \begin{pmatrix} 1 & -1 \\ -1 & 2 \end{pmatrix}$.
How is the distance metric $d(x, y)$ related to the symmetry of vectors?
It must satisfy $d(x, y) = d(y, x)$ for all $x, y$.
What does the Cauchy-Schwarz inequality guarantee about the ratio $\frac{\langle x, y \rangle}{\|x\| \|y\|}$?
The value of the ratio always lies within the interval $[-1, 1]$.
Why does $\|x\| > 0$ for all $x \ne 0$ in a symmetric positive definite matrix context?
Because the induced norm $\|x\| = \sqrt{x^\top Ax}$ and $x^\top Ax$ is strictly positive for $x \ne 0$.
In the inner product calculation $\langle x, y \rangle = \hat{x}^\top A \hat{y}$, what do $\hat{x}$ and $\hat{y}$ represent?
The coordinate vectors of $x$ and $y$ with respect to a specific ordered basis $B$.
In the context of the dot product, what is the length of the vector $x = [1, 1]^\top$?
$\sqrt{2}$.
What is the inner product of the basis vectors $b_i$ and $b_j$ in the standard basis $\{e_1, \dots, e_n\}$ using the dot product?
$1$ if $i = j$, and $0$ otherwise.
If $A$ is a positive definite matrix, how do we know $Ax = 0$ implies $x = 0$?
If $x \ne 0$, then $x^\top Ax > 0$, which contradicts $x^\top(Ax) = x^\top(0) = 0$.
A bilinear mapping $f(x, y)$ is linear in its second argument. Expand $f(x, 2y - 3z)$.
$2f(x, y) - 3f(x, z)$.