112 lines
5.9 KiB
Org Mode
112 lines
5.9 KiB
Org Mode
---
|
|
title: "Operations Research and Optimisation: where to start?"
|
|
date: 2020-05-26
|
|
---
|
|
|
|
[[https://en.wikipedia.org/wiki/Operations_research][Operations research]] (OR) is a vast area comprising a lot of theory,
|
|
different branches of mathematics, and too many applications to
|
|
count. In this post, I will try to explain why I find it so
|
|
fascinating, but also why it can be a little disconcerting to explore
|
|
at first. Then I will try to ease the newcomer's path in this rich
|
|
area, by suggesting a very rough "map" of the field and a few
|
|
references to get started.
|
|
|
|
Keep in mind that although I studied it during my graduate studies,
|
|
this is not my primary area of expertise (I'm a data scientist by
|
|
trade), and I definitely don't pretend to know everything in OR. This
|
|
is a field too vast for any single person to understand in its
|
|
entirety, and I talk mostly from a "amateur mathematician and computer
|
|
scientist" standpoint.
|
|
|
|
* Why is it hard to approach?
|
|
|
|
Operations research can be difficult to approach, since there are many
|
|
references and subfields. Compared to machine learning for instance,
|
|
OR has a slightly longer history (going back to the 17th century, for
|
|
example with [[https://en.wikipedia.org/wiki/Gaspard_Monge][Monge]] and the [[https://en.wikipedia.org/wiki/Transportation_theory_(mathematics)][optimal transport
|
|
problem]])[fn:optimaltransport]. This means that good textbooks and such
|
|
have existed for a long time, but also that there will be plenty of
|
|
material to choose from.
|
|
|
|
[fn:optimaltransport] {-} For a very nice introduction (in French) to
|
|
optimal transport, see these blog posts by [[https://twitter.com/gabrielpeyre][Gabriel Peyré]], on the CNRS
|
|
maths blog: [[https://images.math.cnrs.fr/Le-transport-optimal-numerique-et-ses-applications-Partie-1.html][Part 1]] and [[https://images.math.cnrs.fr/Le-transport-optimal-numerique-et-ses-applications-Partie-2.html][Part 2]]. See also the resources on
|
|
[[https://optimaltransport.github.io/][optimaltransport.github.io]] (in English).
|
|
|
|
|
|
Moreover, OR is very close to applications. Sometimes methods may vary
|
|
a lot in their presentation depending on whether they're applied to
|
|
train tracks, sudoku, or travelling salesmen. In practice, the
|
|
terminology and notations are not the same everywhere. This is
|
|
disconcerting if you are used to "pure" mathematics, where notations
|
|
evolved over a long time and is pretty much standardised for many
|
|
areas. In contrast, if you're used to the statistics literature with
|
|
its [[https://lingpipe-blog.com/2009/10/13/whats-wrong-with-probability-notation/][strange notations]], you will find that OR is actually very well
|
|
formalized.
|
|
|
|
There are many subfields of operations research, including all kinds
|
|
of optimization (constrained and unconstrained), game theory, dynamic
|
|
programming, stochastic processes, etc.
|
|
|
|
* Where to start
|
|
|
|
** Introduction and modelling
|
|
|
|
For an overall introduction, I recommend cite:wentzel1988_operat. It
|
|
is an old book, published by Mir Publications, a Soviet publisher
|
|
which published many excellent scientific textbooks[fn:mir]. It is out
|
|
of print, but it is available [[https://archive.org/details/WentzelOperationsResearchMir1983][on Archive.org]]. The book is quite old,
|
|
but everything presented is still extremely relevant today. It
|
|
requires absolutely no background, and covers everything: a general
|
|
introduction to the field, linear programming, dynamic programming,
|
|
Markov processes and queues, Monte Carlo methods, and game
|
|
theory. Even if you already know some of these topics, the
|
|
presentations is so clear that it is a pleasure to read! (In
|
|
particular, it is one of the best presentations of dynamic programming
|
|
that I have ever read. The explanation of the simplex algorithm is
|
|
also excellent.)
|
|
|
|
[fn:mir] {-} Mir also published [[https://mirtitles.org/2011/06/03/physics-for-everyone/][/Physics for Everyone/]] by Lev Landau
|
|
and Alexander Kitaigorodsky, a three-volume introduction to physics
|
|
that is really accessible. Together with Feynman's famous [[https://www.feynmanlectures.caltech.edu/][lectures]], I
|
|
read them (in French) when I was a kid, and it was the best
|
|
introduction I could possibly have to the subject.
|
|
|
|
|
|
If you are interested in optimization, the first thing you have to
|
|
learn is modelling, i.e. transforming your problem (described in
|
|
natural language, often from a particular industrial application) into
|
|
a mathematical programme. The mathematical programme is the structure
|
|
on which you will be able to apply an algorithm to find an optimal
|
|
solution. Even if (like me) you are initially more interested by the
|
|
algorithmic side of things, learning to create models will shed a lot
|
|
of light on the overall process, and will give you more insight in
|
|
general on the reasoning behind algorithms.
|
|
|
|
The best book I have read on the subject is
|
|
cite:williams2013_model. It contains a lot of concrete, step-by-step
|
|
examples on concrete applications, in a multitude of domains, and
|
|
remains very easy to read and to follow. It covers nearly every type
|
|
of problem, so it is very useful as a reference. When you encounter a
|
|
concrete problem in real life afterwards, you will know how to
|
|
construct an appropriate model, and in the process you will often
|
|
identify a common type of problem. The book then gives plenty of
|
|
advice on how to best approach each type of problem. Finally, it is
|
|
also a great resource to build a "mental map" of the field, avoiding
|
|
to get lost in the jungle of linear, stochastic, mixed integer,
|
|
quadratic, and other network problems.
|
|
|
|
Another interesting resource is the freely available [[https://docs.mosek.com/modeling-cookbook/index.html][MOSEK Modeling
|
|
Cookbook]], covering many types of problems, with more mathematical
|
|
details than in cite:williams2013_model. It is built for people
|
|
wanting to use the commercial [[https://www.mosek.com/][MOSEK]] solver, so it could be useful if
|
|
you plan to use a solver package like this one (more details on
|
|
solvers [[solvers][below]]).
|
|
|
|
** Theory and algorithms
|
|
|
|
** Online courses
|
|
|
|
* Solvers and computational resources <<solvers>>
|
|
|
|
* References
|