A self-contained guide to basic linear algebra ideas that commonly come up in robotics.

Basics

A matrix $A \in \mathbb{R}^{m \times n}$ represents a linear map $A: \mathbb{R}^n \rightarrow \mathbb{R}^m$.

For example, a robot Jacobian $J \in \mathbb{R}^{m \times n}$ maps joint velocity $\dot{q} \in \mathbb{R}^n$ to task-space velocity $\dot{x} \in \mathbb{R}^m$, $$\dot{x} = J \dot{q}.$$

Transpose

The transpose of matrix $A \in \mathbb{R}^{m \times n}$ satisfies:

• $A^\top \in \mathbb{R}^{n \times m}$
• $(A^\top)_{ij} = A_{ji}$

• 💡 Fact: $(AB)^\top = B^\top A^\top$.

Proof: The $(i, j)$-entry of $(AB)^\top$ is: $(AB)_{ij}^\top = (AB)_{ji}$. By matrix multiplication, $$(AB)_{ji} = \sum_k A_{jk} B_{ki}.$$ Also, $$(B^\top A^\top)_{ij} = \sum_k (B^\top)_{ik} (A^\top )_{kj} = \sum_k B_{ki} A_{jk} = (AB)_{ji}.$$

Column space

Suppose $A$ is a matrix with $a_1, a_2, \ldots, a_n$ as its column vectors. Then $$Ax = x_1 a_1 + \cdots + x_n a_n.$$So every possible output $Ax$ is a linear combination of the columns of $A$.

The column space is $\text{Col}(A) = \{A x: x \in \mathbb{R}^n\}$.

Intuitively, the column space tells us which output directions the matrix can generate.

• 🔑 For a Jacobian, $\text{Col}(J)$ is the set of task-space velocities that the robot can instantaneously produce.

Rank

The rank of $A$ is the dimension of its column space: $$\text{rank}(A) = \dim \text{Col}(A).$$ i.e. it is the number of linearly independent columns.

There is a theorem that says $\text{column rank}(A) = \text{row rank}(A)$.

• 🔑 Intuitively, rank measures "how many independent directions" a matrix contains.

Theorem. The rank cannot exceed the number of rows or columns, i.e. $\text{rank}(A) \leq \min(m, n)$ for $A \in \mathbb{R}^{m \times n}$.

This makes sense: we cannot have more than $m$ linearly independent vectors in $\mathbb{R}^m$, and the columns of $A$ live in $\mathbb{R}^m$.

Null space

The null space of $\text{Null}(A) = \{x : Ax = 0\}$. It is also referred to as the kernel, i.e. $\ker(A)$.

These are the input directions that the matrix "kills."

• 💡 For a robot Jacobian, $J\dot{q} = 0$ means the joints can move with velocity $\dot{q}$ while producing no instantaneous end-effector motion (redundancy!)

The dimension of the null space is called the nullity: $\text{nullity}(A) = \dim \text{Null}(A)$.

Rank-nullity theorem

For $A \in \mathbb{R}^{m \times n}$, $\text{rank}(A) + \text{nullity}(A) = n$.

Proof: Fill in later.

Invertibility and full rank

For a square matrix $A \in \mathbb{R}^{n \times n}$, the following statements are equivalent: $$A \text{ invertible} \iff \text{rank}(A)= n \iff \text{Null}(A) = \{0\}.$$

• Invertibility implies trivial null space: $$Ax = 0 \implies A^{-1}A x = A^{-1} 0 \implies x = 0.$$Thus $\text{Null}(A) = \{0\}$.
• Trivial null space implies full rank: $$\text{rank}(A) + \underbrace{\text{nullity}(A)}_0 =n \implies \text{rank}(A) = n.$$
• Full rank implies invertibility: For every $n \times n$ matrix, rank $n$ means its $n$ columns are linearly independent. They therefore form a basis of $\mathbb{R}^n$. Thus, for every $b$, $Ax = b$ has exactly one solution, i.e. $A^{-1} A x = A^{-1} b$ exists and is unique.

For a square matrix, singular means not invertible.

Key identities

• 🔑 $x^\top A^\top Ax = ||Ax||^2$
  • Proof: Using $(Ax)^\top = x^\top A^\top$, we get $$x^\top A^\top Ax = (Ax)^\top (Ax).$$For any vector $y$, $y^\top y = ||y||^2$. Hence $x^\top A^\top Ax = ||Ax||^2$.
• 🔑 $\text{Null}(A^\top A) = \text{Null}(A)$
  • Proof: We prove both inclusions.
    • Direction 1: Suppose $x \in \text{Null}(A)$. Then $Ax = 0$. Therefore $$A^\top Ax = A^\top 0 = 0 \implies x \in \text{Null}(A^\top A) \implies \text{Null}(A) \subseteq \text{Null}(A^\top A).$$
    • Direction 2: Suppose $x \in \text{Null}(A^\top A)$. Then $A^\top Ax = 0$. Multiply on the left by $x^\top$: $$x^\top A^\top Ax = (Ax)^\top Ax = ||Ax||^2 = 0 \implies Ax = 0\implies x \in \text{Null}(A) \implies \text{Null}(A^\top A) = \text{Null}(A).$$This uses the fact that a vector has norm zero only if it is the zero vector.
  • Hence $\text{Null}(A^\top A) = \text{Null}(A)$.
• 🔑 $\text{rank}(A^\top A) = \text{rank}(A)$
  • Proof: Both $A: \mathbb{R}^n \to \mathbb{R}^m$ and $A^\top A : \mathbb{R}^n \to \mathbb{R}^n$ have the same domain dimension $n$. We just proved that they have the same null space, so they have the same nullity. Therefore: $$\text{rank}(A) = n - \text{nullity}(A) = n - \text{nullity}(A^\top A) = \text{rank}(A^\top A).$$
• 🔑 $\text{rank}(AA^\top) = \text{rank}(A)$
  • Proof: Applying the above fact, $$\text{rank}(AA^\top ) = \text{rank}(A^\top) = \text{rank}(A).$$This uses the fact that $\text{rank}(A^\top) = \text{rank}(A)$, which follows directly from the theorem that $\text{row rank}(A) = \text{column rank}(A)$.

Relevant definitions used in this section

The squared norm of a vector is the sum of the squares of its entries. If $y = \begin{bmatrix} y_1 & y_2 & \cdots & y_n \end{bmatrix}$, then its Euclidean norm is $||y|| = \sqrt{y_1^2 + y_2^2 + \cdots + y_n^2}$. So its squared norm is $||y||^2 = y_1^2 + \cdots + y_n^2$.

General rank inequality

$$\text{rank}(AB) \leq \min (\text{rank}(A), \text{rank}(B))$$

Symmetric matrices

Positive semidefinite matrices

Jacobian facts

Now that we've established some basic linear algebra facts, we can apply them to the robot Jacobian!

• 🔖 $J^\top J$ is always singular for a redundant Jacobian.
  • Proof:
• 💡 $JJ^\top$ can be invertible for a redundant robot.