6.6 KiB
- Why would that be interesting?
- The structure of a Lie group
- Interesting properties of Lie groups
- Applications
- References
— title: "Learning some Lie theory for fun and profit " date: 2020-11-10 toc: false —
[fn::{-} The phrase "for fun and profit" seems to be a pretty old expression: according to the answers to this StackExchange question, it might date back to Horace's Ars Poetica ("prodesse et delectare"). I like the idea that books (and ideas!) should be both instructive and enjoyable…]
While exploring quaternions and the theory behind them, I noticed an interesting pattern: in the exposition of cite:sola2017_quater_kinem_error_state_kalman_filter, quaternions and rotations matrices had exactly the same properties, and the derivation of these properties was rigorously identical (bar some minor notation changes).
This is expected because in this specific case, these are just two representations of the same underlying object: rotations. However, from a purely mathematical and abstract point of view, it cannot be a coincidence that you can imbue two different types of objects with exactly the same properties.
Indeed, this is not a coincidence: the important structure that is common to the set of rotation matrices and to the set of quaternions is that of a Lie group.
Why would that be interesting?
From a mathematical point of view, seeing a common structure like this should raise alarm signals in our heads. Is there a deeper concept at play here? If we can find that two objects are two examples of the same abstract structure, maybe we'll also be able to identify that structure elsewhere, maybe where it's less obvious. And then, if we prove interesting theorems on the abstract structure, we'll essentially get the same theorems on every example of this structure, and for free! (i.e. without any additional work!)1
1{-} When you push that idea to its extremes, you get category theory, which is just the study of (abstract) structure. This in a fun rabbit hole to get into, and if you're interested, I recommend the amazing math3ma blog, or cite:riehlCategoryTheoryContext2017 for a complete and approachable treatment. cite:fongSevenSketchesCompositionality2018 gives an interesting perspective on why category theory is interesting in the real world.
We can think of it as a kind of factorization: instead of doing the same thing over and over, we can basically do it once and recall the general result whenever it is needed, as one would define a function and call it later in a piece of software.
In this case, Lie theory provides a general framework for manipulating objects that we want to combine and on which we'd like to compute derivatives. Differentiability is an essentially linear property, in the sense that it works best in vector spaces. Indeed, think of what you do to with a derivative: you want to add it to other stuff to represent increase rates or uncertainties.
Once you can differentiate, a whole new world opens2: optimization becomes easier (because you can use gradient descent), you can have random variables, and so on.
In the case of quaternions, we can define explicitly a differentiation operator, and prove that it has all the nice properties that we come to expect from derivatives. Wouldn't it be nice if we could have all of this automatically? Lie theory gives us the general framework in which we can imbue non-"linear" objects with differentiability.
The structure of a Lie group
Continuing on the example of rotations, what common properties can we identify?
- Quaternions and rotation matrices can be multiplied together (to compose rotations), have an identity element, along with nice properties.
- Quaternions and rotation matrices can be differentiated, and we can map them with usual vectors.
These two group of properties actually correspond to common mathematical structures: a group and a differentiable manifold.
You're probably already familiar with groups, but let's recall the basic properties:
- It's a set $G$ equipped with a binary operation $\cdot$.
- The group is closed under the operation: for any element $x,y$ in G, $x \cdot y$ is always in $G$.
- The operation is associative: $x \cdot (y \cdot z) = (x \cdot y) \cdot z$.
- There is a special element $e$ of $G$ (called the identity element), such that $x \cdot e = e \cdot x$ for all $x \in G$.
- For every element $x$ of $G$, there is a unique element of $G$ denoted $x^{-1}$ such that $x \cdot x^{-1} = x^{-1} \cdot x = e$.
A differentiable manifold is a more complex beast. Although the definition is more complex, we can loosely imagine it as a surface (in higher dimension) on which we can compute derivatives at every point. This means that there is a tangent hyperplane at each point, which is a nice vector space where our derivatives will live.
And that's all! What we have defined so far is a Lie group3, i.e. a group that is also a differentiable manifold. To take the example of rotation matrices:
- We can combine them (i.e. by matrix multiplication): they form a group.
- if we have a function $R : \mathbb{R} \rightarrow \mathrm{GL}_3(\mathbb{R})$ defining a trajectory (e.g. the successive attitudes of a object in space), we can find derivatives of this trajectory! They would represent instantaneous orientation change, or angular velocities.
Interesting properties of Lie groups
Applications
References
This is why a lot of programming languages now try to make differentiability a first-class concept. The ability to differentiate arbitrary programs is a huge bonus for all kinds of operations common in scientific computing. Pioneering advances were made in deep learning libraries, such as TensorFlow and PyTorch; but recent advances are even more exciting. JAX is basically a differentiable Numpy, and Julia has always made differentiable programming a priority, via projects such as JuliaDiff and Zygote.
{-} Lie theory is named after Sophus Lie, a Norwegian mathematician. As such, "Lie" is pronounced lee. Lie was inspired by Galois' work on algebraic equations, and wanted to establish a similar general theory for differential equations.