Final update to the post

bib/bibliography.bib

@@ -444,3 +444,13 @@
   series =	 {Undergraduate Texts in Mathematics},
 }
 
+@book{lafontaine2015_introd_differ_manif,
+  author =	 {Jacques Lafontaine},
+  title =	 {An Introduction to Differential Manifolds},
+  year =	 2015,
+  publisher =	 {Springer International Publishing},
+  url =		 {https://doi.org/10.1007/978-3-319-20735-3},
+  DATE_ADDED =	 {Sat Nov 14 17:05:06 2020},
+  doi =		 {10.1007/978-3-319-20735-3},
+  isbn =	 9783319207346,
+}

posts/lie-theory.org

@@ -27,12 +27,18 @@ Indeed, this is not a coincidence: the important structure that is
 common to the set of rotation matrices and to the set of quaternions
 is that of a /Lie group/.
 
+In this post, I want to explain why I find Lie theory interesting,
+both in its theoretical aspects (for fun) and in its potential for
+real-world application (for profit). I will also give a minimal set of
+references that I used to get started.
+
 * Why would that be interesting?
 
-From a mathematical point of view, seeing a common structure like this
-should raise alarm signals in our heads. Is there a deeper concept at
-play here? If we can find that two objects are two examples of the
-same abstract structure, maybe we'll also be able to identify that
+From a mathematical point of view, seeing a common structure in
+different objects, such as quaternions and rotation matrices, should
+raise alarm signals in our heads. Is there a deeper concept at play
+here? If we can find that two objects are two examples of the same
+abstract structure, maybe we'll also be able to identify that
 structure elsewhere, maybe where it's less obvious. And then, if we
 prove interesting theorems on the abstract structure, we'll
 essentially get the same theorems on every example of this structure,
@@ -43,9 +49,7 @@ and /for free!/ (i.e. without any additional work!)[fn:structure]
 in a fun rabbit hole to get into, and if you're interested, I
 recommend the amazing [[https://www.math3ma.com/][math3ma]] blog, or
 cite:riehlCategoryTheoryContext2017 for a complete and approachable
-treatment. cite:fongSevenSketchesCompositionality2018 gives an
-interesting perspective on why category theory is interesting in the
-real world.
+treatment.
 
 
 We can think of it as a kind of factorization: instead of doing the
@@ -58,7 +62,8 @@ objects that we want to /combine/ and on which we'd like to compute
 /derivatives/. Differentiability is an essentially linear property, in
 the sense that it works best in vector spaces. Indeed, think of what
 you do to with a derivative: you want to /add/ it to other stuff to
-represent increase rates or uncertainties.
+represent increase rates or uncertainties. (And of course, the
+differential operator itself is linear.)
 
 Once you can differentiate, a whole new world
 opens[fn:differentiability]: optimization becomes easier (because you
@@ -89,7 +94,7 @@ identify?
    compose rotations), have an identity element, along with nice
    properties.
 2. Quaternions and rotation matrices can be differentiated, and we can
-   map them with usual vectors.
+   map them to and from usual vectors in $\mathbb{R}^m$.
 
 These two group of properties actually correspond to common
 mathematical structures: a /group/ and a /differentiable manifold/.
@@ -106,22 +111,23 @@ basic properties:
 - For every element $x$ of $G$, there is a unique element of $G$
   denoted $x^{-1}$ such that $x \cdot x^{-1} = x^{-1} \cdot x = e$.
 
-A [[https://en.wikipedia.org/wiki/Differentiable_manifold][differentiable manifold]] is a more complex beast. Although the
-definition is more complex, we can loosely imagine it as a surface (in
-higher dimension) on which we can compute derivatives at every
-point. This means that there is a tangent hyperplane at each point,
-which is a nice vector space where our derivatives will live.
+A [[https://en.wikipedia.org/wiki/Differentiable_manifold][differentiable manifold]] is a more complex
+beast.[fn:differential_geometry] Although the definition is more
+complex, we can loosely imagine it as a surface (in higher dimension)
+on which we can compute derivatives at every point. This means that
+there is a tangent hyperplane at each point, which is a nice vector
+space where our derivatives will live.
 
 You can think of the manifold as a tablecloth that has a weird shape,
 all kinds of curvatures, but no edges or spikes. The idea here is that
 we can define an /atlas/, i.e. a local approximation of the manifold
 as a plane. The name is telling: they're called atlases because they
-play the exact same role as maps. The Earth is not flat, it is a
-sphere with all kinds of deformations (mountains, canyons, oceans),
-but we can have maps that represent a small area with a very good
-precision. Similarly, atlases are the vector spaces that provide the
-best linear approximation of a small region around a point on the
-manifold.
+play the exact same role as geographical maps. The Earth is not flat,
+it is a sphere with all kinds of deformations (mountains, canyons,
+oceans), but we can have planar maps that represent a small area with
+a very good precision. Similarly, atlases are the vector spaces that
+provide the best linear approximation of a small region around a point
+on the manifold.
 
 So we know what a group and a differential manifold are. As it turns
 out, that's all we need to know! What we have defined so far is a /Lie
@@ -138,6 +144,12 @@ To take the example of rotation matrices:
   of this trajectory! They would represent instantaneous orientation
   change, or angular velocities.
 
+[fn:differential_geometry] {-} For a more complete introduction to
+differential geometry and differentiable manifolds, see
+cite:lafontaine2015_introd_differ_manif. It introduces manifolds,
+differential topology, Lie groups, and more advanced topics, all with
+little prerequisites (basics of differential calculus).
+
 [fn:lie] {-} Lie theory is named after [[https://en.wikipedia.org/wiki/Sophus_Lie][Sophus Lie]], a Norwegian
 mathematician. As such, "Lie" is pronounced /lee/. Lie was inspired by
 [[https://en.wikipedia.org/wiki/%C3%89variste_Galois][Galois']] work on algebraic equations, and wanted to establish a similar
@@ -150,36 +162,84 @@ material that you can find online.[fn:princeton_companion] I
 especially recommend the tutorial by
 cite:sola2018_micro_lie_theor_state_estim_robot: just enough maths to
 understand what is going on, but without losing track of the
-applications. There is also a [[https://www.youtube.com/watch?v=QR1p0Rabuww][video tutorial]] for the [[https://www.iros2020.org/][IROS2020]]
-conference[fn::More specifically for the workshop on [[https://sites.google.com/view/iros2020-geometric-methods/][Bringing
-geometric methods to robot learning, optimization and control]].]. For a
-more complete treatment, cite:stillwell2008_naive_lie_theor is
-great.[fn::{-} John Stillwell is one of the best textbook writers. All
-his books are extremely clear and a pleasure to read. You generally
-read a book because you're interested in learning the topic; you begin
-learning a topic just because Stillwell wrote a book on it.]
+applications. There is also a [[https://www.youtube.com/watch?v=QR1p0Rabuww][video tutorial]] made for the [[https://www.iros2020.org/][IROS2020]]
+conference[fn:workshop]. For a more complete treatment,
+cite:stillwell2008_naive_lie_theor is great[fn:stillwell].
+
+Because of the group structure, the manifold is similar at every
+point: in particular, all the tangent spaces look alike. This is why
+the /Lie algebra/, the tangent space at the identity, is so
+important. All tangent spaces are vector spaces isomorphic to the Lie
+algebra, therefore studying the Lie algebra is sufficient to derive
+all the interesting aspects of the Lie group.
+
+Lie algebras are always vector spaces. Even though their definition
+may be quite complex (e.g. skew-symmetric matrices in the case of the
+group of rotation matrices[fn:skewsymmetric]), we can always find an
+isomorphism of vector spaces between the Lie algebra and
+$\mathbb{R}^m$ (in the case of finite-dimensional Lie groups). This is
+really nice for many applications: for instance, the usual probability
+distributions on $\mathbb{R}^m$ translate directly to the Lie algebra.
+
+The final aspect I'll mention is the existence of /exponential maps/,
+allowing transferring elements of the Lie algebra to the Lie
+group. The operator $\exp$ will wrap an element of the Lie algebra
+(i.e. a tangent vector) to its corresponding element of the Lie group
+by wrapping along a geodesic of the manifold. There is also a
+logarithmic map providing the inverse operation.
+
+[fn::{-} The Lie group (in blue) with its associated Lie algebra
+(red). We can see how each element of the Lie algebra is wrapped on
+the manifold via the exponential map. Figure from
+cite:sola2018_micro_lie_theor_state_estim_robot.]
+#+ATTR_HTML: :width 500px
+[[../images/lie_exponential.svg]] 
+
+If all this piqued your interest, you can read a very short (only 14
+pages!) overview of Lie theory in
+cite:sola2018_micro_lie_theor_state_estim_robot. They also expand on
+applications to estimation and robotics (as the title suggests), so
+they focus on deriving Jacobians and other essential tools for any Lie
+group. They also give very detailed examples of common Lie groups
+(complex numbers, rotation matrices, quaternions, translations).
 
 [fn:princeton_companion] {-} There is also a chapter on Lie theory in
 the amazing /Princeton Companion to Mathematics/
 [[citep:gowersPrincetonCompanionMathematics2010][::, §II.48]].
 
+[fn:workshop] More specifically for the workshop on [[https://sites.google.com/view/iros2020-geometric-methods/][Bringing geometric
+methods to robot learning, optimization and control]].
 
-The Lie group is a set of elements. At every element, there is a
-tangent vector space. The tangent vector space at the identity is
-called the Lie algebra, and as we will see, it plays a special role.
+[fn:stillwell] I really like John Stillwell as a textbook author. All
+his books are extremely clear and a pleasure to read.
 
-TODO
+[fn:skewsymmetric] {-} [[https://en.wikipedia.org/wiki/Skew-symmetric_matrix][Skew-symmetric matrices]] are matrices $A$ such
+that $A^\top = -A$:
+\[ [\boldsymbol\omega]_\times = \begin{bmatrix}
+0 & -\omega_x & \omega_y \\
+\omega_x & 0 & -\omega_z \\
+-\omega_y & \omega_z & 0
+\end{bmatrix}. \]
 
-* Applications
+* Conclusion
 
 Lie theory is useful because it gives strong theoretical guarantees
 whenever we need to linearize something. If you have a system evolving
 on a complex geometric structure (for example, the space of rotations,
 which is definitely not linear), but you need to use a linear
 operation (if you need uncertainties, or you have differential
-equations), you have to approximate somehow.
+equations), you have to approximate somehow. Using the Lie structure
+of the underlying space, you immediately get a principled way of
+defining derivatives, random variables, and so on.
 
-Using the Lie structure of the underlying space, you immediately get a
-principled way of defining derivatives, random variables, and so on.
+Therefore, for estimation problems, Lie theory provides a strong
+backdrop to define state spaces, in which all the usual manipulations
+are possible. It has thus seen a spike of interest in the robotics
+literature, with applications to estimation, optimal control, general
+optimization, and many other fields.
+
+I hope that this quick introduction has motivated you to learn more
+about Lie theory, as it is a fascinating topic with a lot of
+potential!
 
 * References