Add images
This commit is contained in:
parent
0a8a606f26
commit
5b815f5a3c
15 changed files with 113 additions and 43 deletions
|
|
@ -32,18 +32,30 @@
|
|||
<h3 id="introduction-and-modelling">Introduction and modelling</h3>
|
||||
<p>For an overall introduction, I recommend <span class="citation" data-cites="wentzel1988_operat">Wentzel (<a href="#ref-wentzel1988_operat">1988</a>)</span>. It is an old book, published by Mir Publications, a Soviet publisher which published many excellent scientific textbooks<span><label for="sn-2" class="margin-toggle">⊕</label><input type="checkbox" id="sn-2" class="margin-toggle" /><span class="marginnote"> Mir also published <a href="https://mirtitles.org/2011/06/03/physics-for-everyone/"><em>Physics for Everyone</em></a> by Lev Landau and Alexander Kitaigorodsky, a three-volume introduction to physics that is really accessible. Together with Feynman’s famous <a href="https://www.feynmanlectures.caltech.edu/">lectures</a>, I read them (in French) when I was a kid, and it was the best introduction I could possibly have to the subject.<br />
|
||||
<br />
|
||||
</span></span>. It is out of print, but it is available <a href="https://archive.org/details/WentzelOperationsResearchMir1983">on Archive.org</a>. The book is quite old, but everything presented is still extremely relevant today. It requires absolutely no background, and covers everything: a general introduction to the field, linear programming, dynamic programming, Markov processes and queues, Monte Carlo methods, and game theory. Even if you already know some of these topics, the presentations is so clear that it is a pleasure to read! (In particular, it is one of the best presentations of dynamic programming that I have ever read. The explanation of the simplex algorithm is also excellent.)</p>
|
||||
</span></span>. It is out of print, but it is available <a href="https://archive.org/details/WentzelOperationsResearchMir1983">on Archive.org</a><span><label for="sn-3" class="margin-toggle">⊕</label><input type="checkbox" id="sn-3" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/wentzel.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span>. The book is quite old, but everything presented is still extremely relevant today. It requires absolutely no background, and covers everything: a general introduction to the field, linear programming, dynamic programming, Markov processes and queues, Monte Carlo methods, and game theory. Even if you already know some of these topics, the presentations is so clear that it is a pleasure to read! (In particular, it is one of the best presentations of dynamic programming that I have ever read. The explanation of the simplex algorithm is also excellent.)</p>
|
||||
<p>If you are interested in optimization, the first thing you have to learn is modelling, i.e. transforming your problem (described in natural language, often from a particular industrial application) into a mathematical programme. The mathematical programme is the structure on which you will be able to apply an algorithm to find an optimal solution. Even if (like me) you are initially more interested in the algorithmic side of things, learning to create models will shed a lot of light on the overall process, and will give you more insight in general on the reasoning behind algorithms.</p>
|
||||
<p>The best book I have read on the subject is <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span>. It contains a lot of concrete, step-by-step examples on concrete applications, in a multitude of domains, and remains very easy to read and to follow. It covers nearly every type of problem, so it is very useful as a reference. When you encounter a concrete problem in real life afterwards, you will know how to construct an appropriate model, and in the process you will often identify a common type of problem. The book then gives plenty of advice on how to approach each type of problem. Finally, it is also a great resource to build a “mental map” of the field, avoiding getting lost in the jungle of linear, stochastic, mixed integer, quadratic, and other network problems.</p>
|
||||
<p>The best book I have read on the subject is <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span><span><label for="sn-4" class="margin-toggle">⊕</label><input type="checkbox" id="sn-4" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/williams.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span>. It contains a lot of concrete, step-by-step examples on concrete applications, in a multitude of domains, and remains very easy to read and to follow. It covers nearly every type of problem, so it is very useful as a reference. When you encounter a concrete problem in real life afterwards, you will know how to construct an appropriate model, and in the process you will often identify a common type of problem. The book then gives plenty of advice on how to approach each type of problem. Finally, it is also a great resource to build a “mental map” of the field, avoiding getting lost in the jungle of linear, stochastic, mixed integer, quadratic, and other network problems.</p>
|
||||
<p>Another interesting resource is the freely available <a href="https://docs.mosek.com/modeling-cookbook/index.html">MOSEK Modeling Cookbook</a>, covering many types of problems, with more mathematical details than in <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span>. It is built for people wanting to use the commercial <a href="https://www.mosek.com/">MOSEK</a> solver, so it could be useful if you plan to use a solver package like this one (more details on solvers <a href="#solvers">below</a>).</p>
|
||||
<h3 id="theory-and-algorithms">Theory and algorithms</h3>
|
||||
<p>The basic algorithm for optimization is the <a href="https://en.wikipedia.org/wiki/Simplex_algorithm">simplex algorithm</a>, developed by Dantzig in the 1940s to solve <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a> problems. It is the one of the main building blocks for mathematical optimization, and is used and referenced extensively in all kinds of approaches. As such, it is really important to understand it in detail. There are many books on the subject, but I especially liked <span class="citation" data-cites="chvatal1983_linear">Chvátal (<a href="#ref-chvatal1983_linear">1983</a>)</span> (out of print, but you can find cheap used versions on Amazon). It covers everything there is to know on the simplex algorithms (step-by-step explanations with simple examples, correctness and complexity analysis, computational and implementation considerations) and to many applications. I think it is overall the best introduction. <span class="citation" data-cites="vanderbei2014_linear">Vanderbei (<a href="#ref-vanderbei2014_linear">2014</a>)</span> follows a very similar outline, but contains more recent computational considerations<span><label for="sn-3" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-3" class="margin-toggle" /><span class="sidenote">For all the details about practical implementations of the simplex algorithm, <span class="citation" data-cites="maros2003_comput">Maros (<a href="#ref-maros2003_comput">2003</a>)</span> is dedicated to the computational aspects and contains everything you will need.<br />
|
||||
<p>The basic algorithm for optimization is the <a href="https://en.wikipedia.org/wiki/Simplex_algorithm">simplex algorithm</a>, developed by Dantzig in the 1940s to solve <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a> problems. It is the one of the main building blocks for mathematical optimization, and is used and referenced extensively in all kinds of approaches. As such, it is really important to understand it in detail. There are many books on the subject, but I especially liked <span class="citation" data-cites="chvatal1983_linear">Chvátal (<a href="#ref-chvatal1983_linear">1983</a>)</span> (out of print, but you can find cheap used versions on Amazon). It covers everything there is to know on the simplex algorithms (step-by-step explanations with simple examples, correctness and complexity analysis, computational and implementation considerations) and to many applications. I think it is overall the best introduction. <span class="citation" data-cites="vanderbei2014_linear">Vanderbei (<a href="#ref-vanderbei2014_linear">2014</a>)</span> follows a very similar outline, but contains more recent computational considerations<span><label for="sn-5" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-5" class="margin-toggle" /><span class="sidenote">For all the details about practical implementations of the simplex algorithm, <span class="citation" data-cites="maros2003_comput">Maros (<a href="#ref-maros2003_comput">2003</a>)</span> is dedicated to the computational aspects and contains everything you will need.<br />
|
||||
<br />
|
||||
</span></span>. (The author also has <a href="http://vanderbei.princeton.edu/307/lectures.html">lecture slides</a>.)</p>
|
||||
<p>For more books on linear programming, the two books <span class="citation" data-cites="dantzig1997_linear">Dantzig (<a href="#ref-dantzig1997_linear">1997</a>)</span>, <span class="citation" data-cites="dantzig2003_linear">Dantzig (<a href="#ref-dantzig2003_linear">2003</a>)</span> are very complete, if somewhat more mathematically advanced. <span class="citation" data-cites="bertsimas1997_introd">Bertsimas and Tsitsiklis (<a href="#ref-bertsimas1997_introd">1997</a>)</span> is also a great reference, if you can find it.</p>
|
||||
<p>For all the other subfields, <a href="https://or.stackexchange.com/a/870">this great StackExchange answer</a> contains a lot of useful references, including most of the above. Of particular note are <span class="citation" data-cites="peyreComputationalOptimalTransport2019">Peyré and Cuturi (<a href="#ref-peyreComputationalOptimalTransport2019">2019</a>)</span> for optimal transport, <span class="citation" data-cites="boyd2004_convex">Boyd (<a href="#ref-boyd2004_convex">2004</a>)</span> for convex optimization (<a href="https://web.stanford.edu/~boyd/cvxbook/">freely available online</a>), and <span class="citation" data-cites="nocedal2006_numer">Nocedal (<a href="#ref-nocedal2006_numer">2006</a>)</span> for numerical optimization. <span class="citation" data-cites="kochenderfer2019_algor">Kochenderfer (<a href="#ref-kochenderfer2019_algor">2019</a>)</span> is not in the list (because it is very recent) but is also excellent, with examples in Julia covering nearly every kind of optimization algorithms.</p>
|
||||
<p>For all the other subfields, <a href="https://or.stackexchange.com/a/870">this great StackExchange answer</a> contains a lot of useful references, including most of the above. Of particular note are <span class="citation" data-cites="peyreComputationalOptimalTransport2019">Peyré and Cuturi (<a href="#ref-peyreComputationalOptimalTransport2019">2019</a>)</span> for optimal transport, <span class="citation" data-cites="boyd2004_convex">Boyd (<a href="#ref-boyd2004_convex">2004</a>)</span> for convex optimization (<a href="https://web.stanford.edu/~boyd/cvxbook/">freely available online</a>), and <span class="citation" data-cites="nocedal2006_numer">Nocedal (<a href="#ref-nocedal2006_numer">2006</a>)</span> for numerical optimization. <span class="citation" data-cites="kochenderfer2019_algor">Kochenderfer (<a href="#ref-kochenderfer2019_algor">2019</a>)</span><span><label for="sn-6" class="margin-toggle">⊕</label><input type="checkbox" id="sn-6" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/kochenderfer.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span> is not in the list (because it is very recent) but is also excellent, with examples in Julia covering nearly every kind of optimization algorithms.</p>
|
||||
<h3 id="online-courses">Online courses</h3>
|
||||
<p>If you would like to watch video lectures, there are a few good opportunities freely available online, in particular on <a href="https://ocw.mit.edu/index.htm">MIT OpenCourseWare</a>. The list of courses at MIT is available <a href="https://orc.mit.edu/academics/course-offerings">on their webpage</a>. I haven’t actually looked in details at the courses content<span><label for="sn-4" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-4" class="margin-toggle" /><span class="sidenote">I am more comfortable reading books than watching lecture videos online. Although I liked attending classes during my studies, I do not have the same feeling in front of a video. When I read, I can re-read three times the same sentence, pause to look up something, or skim a few paragraphs. I find that the inability to do that with a video diminishes greatly my ability to concentrate.<br />
|
||||
<p>If you would like to watch video lectures, there are a few good opportunities freely available online, in particular on <a href="https://ocw.mit.edu/index.htm">MIT OpenCourseWare</a>. The list of courses at MIT is available <a href="https://orc.mit.edu/academics/course-offerings">on their webpage</a>. I haven’t actually looked in details at the courses content<span><label for="sn-7" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-7" class="margin-toggle" /><span class="sidenote">I am more comfortable reading books than watching lecture videos online. Although I liked attending classes during my studies, I do not have the same feeling in front of a video. When I read, I can re-read three times the same sentence, pause to look up something, or skim a few paragraphs. I find that the inability to do that with a video diminishes greatly my ability to concentrate.<br />
|
||||
<br />
|
||||
</span></span>, so I cannot vouch for them directly, but MIT courses are generally of excellent quality. Most courses are also taught by Bertsimas and Bertsekas, who are very famous and wrote many excellent books.</p>
|
||||
<p>Of particular notes are:</p>
|
||||
|
|
@ -54,12 +66,16 @@
|
|||
<li><a href="https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-972-algebraic-techniques-and-semidefinite-optimization-spring-2006/">Algebraic Techniques and Semidefinite Optimization</a>,</li>
|
||||
<li><a href="https://ocw.mit.edu/courses/sloan-school-of-management/15-083j-integer-programming-and-combinatorial-optimization-fall-2009/">Integer Programming and Combinatorial Optimization</a>.</li>
|
||||
</ul>
|
||||
<p>Another interesting course I found online is <a href="https://www.ams.jhu.edu/~wcook12/dl/index.html">Deep Learning in Discrete Optimization</a>, at Johns Hopkins<span><label for="sn-5" class="margin-toggle">⊕</label><input type="checkbox" id="sn-5" class="margin-toggle" /><span class="marginnote"> It is taught by William Cook, who is the author of <a href="https://press.princeton.edu/books/paperback/9780691163529/in-pursuit-of-the-traveling-salesman"><em>In Pursuit of the Traveling Salesman</em></a>, a nice introduction to the TSP problem in a readable form.<br />
|
||||
<p>Another interesting course I found online is <a href="https://www.ams.jhu.edu/~wcook12/dl/index.html">Deep Learning in Discrete Optimization</a>, at Johns Hopkins<span><label for="sn-8" class="margin-toggle">⊕</label><input type="checkbox" id="sn-8" class="margin-toggle" /><span class="marginnote"> It is taught by William Cook, who is the author of <a href="https://press.princeton.edu/books/paperback/9780691163529/in-pursuit-of-the-traveling-salesman"><em>In Pursuit of the Traveling Salesman</em></a>, a nice introduction to the TSP problem in a readable form.<br />
|
||||
<br />
|
||||
</span></span>. It contains an interesting overview of deep learning and integer programming, with a focus on connections, and applications to recent research areas in ML (reinforcement learning, attention, etc.).</p>
|
||||
<h2 id="solvers-and-computational-resources">Solvers and computational resources <span id="solvers"></span></h2>
|
||||
<p>When you start reading about modelling and algorithms, I recommend you try solving a few problems yourself, either by hand for small instances, or using an existing solver. It will allow you to follow the examples in books, while also practising your modelling skills. You will also get an intuition of what is difficult to model and to solve.</p>
|
||||
<p>There are many solvers available, both free and commercial, with various capabilities. I recommend you use the fantastic <a href="https://github.com/JuliaOpt/JuMP.jl">JuMP</a> library for Julia, which exposes a domain-specific language for modelling, along with interfaces to nearly all major solver packages. (Even if you don’t know Julia, this is a great and easy way to start!) If you’d rather use Python, you can use Google’s <a href="https://developers.google.com/optimization/introduction/python">OR-Tools</a> or <a href="https://github.com/coin-or/pulp">PuLP</a> for linear programming.</p>
|
||||
<p>There are many solvers available, both free and commercial, with various capabilities. I recommend you use the fantastic <a href="https://github.com/JuliaOpt/JuMP.jl">JuMP</a><span><label for="sn-9" class="margin-toggle">⊕</label><input type="checkbox" id="sn-9" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/jump.svg" width="250" /><br />
|
||||
<br />
|
||||
</span></span> library for Julia, which exposes a domain-specific language for modelling, along with interfaces to nearly all major solver packages. (Even if you don’t know Julia, this is a great and easy way to start!) If you’d rather use Python, you can use Google’s <a href="https://developers.google.com/optimization/introduction/python">OR-Tools</a> or <a href="https://github.com/coin-or/pulp">PuLP</a> for linear programming.</p>
|
||||
<p>Regarding solvers, there is a <a href="http://www.juliaopt.org/JuMP.jl/stable/installation/#Getting-Solvers-1">list of solvers</a> on JuMP’s documentation, with their capabilities and their license. Free solvers include <a href="https://www.gnu.org/software/glpk/">GLPK</a> (linear programming), <a href="https://github.com/coin-or/Ipopt">Ipopt</a> (non-linear programming), and <a href="https://scip.zib.de/">SCIP</a> (mixed-integer linear programming).</p>
|
||||
<p>Commercial solvers often have better performance, and some of them propose a free academic license: <a href="https://www.mosek.com/">MOSEK</a>, <a href="https://www.gurobi.com/">Gurobi</a>, and <a href="https://www.ibm.com/analytics/cplex-optimizer">IBM CPLEX</a> in particular all offer free academic licenses and work very well with JuMP.</p>
|
||||
<p>Another awesome resource is the <a href="https://neos-server.org/neos/">NEOS Server</a>. It offers free computing resources for numerical optimization, including all major free and commercial solvers! You can submit jobs on it in a standard format, or interface your favourite programming language with it. The fact that such an amazing resource exists for free, for everyone is extraordinary. They also have an accompanying book, the <a href="https://neos-guide.org/">NEOS Guide</a>, containing many case studies and description of problem types. The <a href="https://neos-guide.org/content/optimization-taxonomy">taxonomy</a> may be particularly useful.</p>
|
||||
|
|
|
|||
1
_site/images/or_references/jump.svg
Normal file
1
_site/images/or_references/jump.svg
Normal file
|
|
@ -0,0 +1 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 377.42 122.03"><defs><style>.cls-1{fill:#231f20;}.cls-2,.cls-3,.cls-5,.cls-7,.cls-9{fill:none;}.cls-2,.cls-3{stroke:#231f20;stroke-linecap:round;stroke-linejoin:round;}.cls-2{stroke-width:6.63px;}.cls-3{stroke-width:6.63px;fill-rule:evenodd;}.cls-4{fill:#aa7dc0;}.cls-5{stroke:#945bb0;}.cls-5,.cls-7,.cls-9{stroke-miterlimit:10;stroke-width:3px;}.cls-6{fill:#6bab5b;}.cls-7{stroke:#3b972e;}.cls-8{fill:#d66661;}.cls-9{stroke:#c93d39;}</style></defs><title>Thin-Border-Logo-Text</title><g id="Page_1" data-name="Page 1"><g id="Layer_1-2" data-name="Layer 1"><polygon class="cls-1" points="313.71 117.21 326.3 117.21 326.3 50.53 311.9 50.53 291.01 79.2 270.12 50.53 255.62 50.53 255.62 117.21 268.21 117.21 268.21 68.72 289.48 97.78 291.96 97.78 313.71 68.81 313.71 117.21 313.71 117.21"/><path class="cls-1" d="M172.45,50.54H185.2V93.37c0,3.86.29,7.71-.65,11.5a19.26,19.26,0,0,1-7.72,11.33,19.75,19.75,0,0,1-9.07,3.42,20.85,20.85,0,0,1-22.67-16.11l12.38-3.16a8.45,8.45,0,0,0,5.17,6.09,8,8,0,0,0,3.67.45,7.09,7.09,0,0,0,3.2-1.16c2.77-1.93,2.94-5.34,2.94-8.34V50.54Zm26.46,0h12.52V74.35c0,4.61.31,7.84.95,9.63a8.46,8.46,0,0,0,3.06,4.19,8.76,8.76,0,0,0,5.2,1.51,9.15,9.15,0,0,0,5.24-1.47,8.68,8.68,0,0,0,3.2-4.34q.76-2.13.76-9.12V50.57h12.43V71.49c0,8.61-.69,14.52-2,17.69a19.21,19.21,0,0,1-7.34,8.9q-4.86,3.11-12.34,3.11-8.13,0-13.13-3.62a19.27,19.27,0,0,1-7.06-10.12c-.94-3-1.45-8.44-1.45-16.32V50.57Zm138.87,0h13.46q11,0,15.76,2a16,16,0,0,1,7.63,6.54,20.21,20.21,0,0,1,2.79,10.8,18.23,18.23,0,0,1-3.65,11.61A18.82,18.82,0,0,1,363.86,88q-3.66,1-13.37,1v28.26H337.78V50.54Zm12.71,26.08h4a22.42,22.42,0,0,0,6.63-.68,5.82,5.82,0,0,0,2.92-2.25,6.53,6.53,0,0,0,1.07-3.79,6.09,6.09,0,0,0-3-5.62C360.69,63.41,358,63,354.06,63h-3.57V76.62Z"/><line class="cls-2" x1="19.64" y1="67.28" x2="120.16" y2="28.42"/><polyline class="cls-3" points="3.32 118.72 89.75 32.29 118.72 3.32"/><path class="cls-4" d="M106.29,120.18a17.17,17.17,0,1,1,12.16-5A17.06,17.06,0,0,1,106.29,120.18Z"/><path class="cls-5" d="M106.29,120.18a17.17,17.17,0,1,1,12.16-5A17.06,17.06,0,0,1,106.29,120.18Z"/><path class="cls-6" d="M106.29,79.33a17.19,17.19,0,1,1,12.16-5A17.07,17.07,0,0,1,106.29,79.33Z"/><path class="cls-7" d="M106.29,79.33a17.19,17.19,0,1,1,12.16-5A17.07,17.07,0,0,1,106.29,79.33Z"/><path class="cls-8" d="M65.28,120.18a17.19,17.19,0,1,1,12.15-5A17,17,0,0,1,65.28,120.18Z"/><path class="cls-9" d="M65.28,120.18a17.19,17.19,0,1,1,12.15-5A17,17,0,0,1,65.28,120.18Z"/></g></g></svg>
|
||||
|
After Width: | Height: | Size: 2.5 KiB |
BIN
_site/images/or_references/kochenderfer.jpg
Normal file
BIN
_site/images/or_references/kochenderfer.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
BIN
_site/images/or_references/kochenderfer_small.jpg
Normal file
BIN
_site/images/or_references/kochenderfer_small.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
BIN
_site/images/or_references/wentzel.jpg
Normal file
BIN
_site/images/or_references/wentzel.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 108 KiB |
BIN
_site/images/or_references/wentzel_small.jpg
Normal file
BIN
_site/images/or_references/wentzel_small.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 108 KiB |
BIN
_site/images/or_references/williams.jpg
Normal file
BIN
_site/images/or_references/williams.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 52 KiB |
BIN
_site/images/or_references/williams_small.jpg
Normal file
BIN
_site/images/or_references/williams_small.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 52 KiB |
|
|
@ -61,18 +61,30 @@
|
|||
<h3 id="introduction-and-modelling">Introduction and modelling</h3>
|
||||
<p>For an overall introduction, I recommend <span class="citation" data-cites="wentzel1988_operat">Wentzel (<a href="#ref-wentzel1988_operat">1988</a>)</span>. It is an old book, published by Mir Publications, a Soviet publisher which published many excellent scientific textbooks<span><label for="sn-2" class="margin-toggle">⊕</label><input type="checkbox" id="sn-2" class="margin-toggle" /><span class="marginnote"> Mir also published <a href="https://mirtitles.org/2011/06/03/physics-for-everyone/"><em>Physics for Everyone</em></a> by Lev Landau and Alexander Kitaigorodsky, a three-volume introduction to physics that is really accessible. Together with Feynman’s famous <a href="https://www.feynmanlectures.caltech.edu/">lectures</a>, I read them (in French) when I was a kid, and it was the best introduction I could possibly have to the subject.<br />
|
||||
<br />
|
||||
</span></span>. It is out of print, but it is available <a href="https://archive.org/details/WentzelOperationsResearchMir1983">on Archive.org</a>. The book is quite old, but everything presented is still extremely relevant today. It requires absolutely no background, and covers everything: a general introduction to the field, linear programming, dynamic programming, Markov processes and queues, Monte Carlo methods, and game theory. Even if you already know some of these topics, the presentations is so clear that it is a pleasure to read! (In particular, it is one of the best presentations of dynamic programming that I have ever read. The explanation of the simplex algorithm is also excellent.)</p>
|
||||
</span></span>. It is out of print, but it is available <a href="https://archive.org/details/WentzelOperationsResearchMir1983">on Archive.org</a><span><label for="sn-3" class="margin-toggle">⊕</label><input type="checkbox" id="sn-3" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="../images/or_references/wentzel.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span>. The book is quite old, but everything presented is still extremely relevant today. It requires absolutely no background, and covers everything: a general introduction to the field, linear programming, dynamic programming, Markov processes and queues, Monte Carlo methods, and game theory. Even if you already know some of these topics, the presentations is so clear that it is a pleasure to read! (In particular, it is one of the best presentations of dynamic programming that I have ever read. The explanation of the simplex algorithm is also excellent.)</p>
|
||||
<p>If you are interested in optimization, the first thing you have to learn is modelling, i.e. transforming your problem (described in natural language, often from a particular industrial application) into a mathematical programme. The mathematical programme is the structure on which you will be able to apply an algorithm to find an optimal solution. Even if (like me) you are initially more interested in the algorithmic side of things, learning to create models will shed a lot of light on the overall process, and will give you more insight in general on the reasoning behind algorithms.</p>
|
||||
<p>The best book I have read on the subject is <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span>. It contains a lot of concrete, step-by-step examples on concrete applications, in a multitude of domains, and remains very easy to read and to follow. It covers nearly every type of problem, so it is very useful as a reference. When you encounter a concrete problem in real life afterwards, you will know how to construct an appropriate model, and in the process you will often identify a common type of problem. The book then gives plenty of advice on how to approach each type of problem. Finally, it is also a great resource to build a “mental map” of the field, avoiding getting lost in the jungle of linear, stochastic, mixed integer, quadratic, and other network problems.</p>
|
||||
<p>The best book I have read on the subject is <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span><span><label for="sn-4" class="margin-toggle">⊕</label><input type="checkbox" id="sn-4" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="../images/or_references/williams.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span>. It contains a lot of concrete, step-by-step examples on concrete applications, in a multitude of domains, and remains very easy to read and to follow. It covers nearly every type of problem, so it is very useful as a reference. When you encounter a concrete problem in real life afterwards, you will know how to construct an appropriate model, and in the process you will often identify a common type of problem. The book then gives plenty of advice on how to approach each type of problem. Finally, it is also a great resource to build a “mental map” of the field, avoiding getting lost in the jungle of linear, stochastic, mixed integer, quadratic, and other network problems.</p>
|
||||
<p>Another interesting resource is the freely available <a href="https://docs.mosek.com/modeling-cookbook/index.html">MOSEK Modeling Cookbook</a>, covering many types of problems, with more mathematical details than in <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span>. It is built for people wanting to use the commercial <a href="https://www.mosek.com/">MOSEK</a> solver, so it could be useful if you plan to use a solver package like this one (more details on solvers <a href="#solvers">below</a>).</p>
|
||||
<h3 id="theory-and-algorithms">Theory and algorithms</h3>
|
||||
<p>The basic algorithm for optimization is the <a href="https://en.wikipedia.org/wiki/Simplex_algorithm">simplex algorithm</a>, developed by Dantzig in the 1940s to solve <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a> problems. It is the one of the main building blocks for mathematical optimization, and is used and referenced extensively in all kinds of approaches. As such, it is really important to understand it in detail. There are many books on the subject, but I especially liked <span class="citation" data-cites="chvatal1983_linear">Chvátal (<a href="#ref-chvatal1983_linear">1983</a>)</span> (out of print, but you can find cheap used versions on Amazon). It covers everything there is to know on the simplex algorithms (step-by-step explanations with simple examples, correctness and complexity analysis, computational and implementation considerations) and to many applications. I think it is overall the best introduction. <span class="citation" data-cites="vanderbei2014_linear">Vanderbei (<a href="#ref-vanderbei2014_linear">2014</a>)</span> follows a very similar outline, but contains more recent computational considerations<span><label for="sn-3" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-3" class="margin-toggle" /><span class="sidenote">For all the details about practical implementations of the simplex algorithm, <span class="citation" data-cites="maros2003_comput">Maros (<a href="#ref-maros2003_comput">2003</a>)</span> is dedicated to the computational aspects and contains everything you will need.<br />
|
||||
<p>The basic algorithm for optimization is the <a href="https://en.wikipedia.org/wiki/Simplex_algorithm">simplex algorithm</a>, developed by Dantzig in the 1940s to solve <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a> problems. It is the one of the main building blocks for mathematical optimization, and is used and referenced extensively in all kinds of approaches. As such, it is really important to understand it in detail. There are many books on the subject, but I especially liked <span class="citation" data-cites="chvatal1983_linear">Chvátal (<a href="#ref-chvatal1983_linear">1983</a>)</span> (out of print, but you can find cheap used versions on Amazon). It covers everything there is to know on the simplex algorithms (step-by-step explanations with simple examples, correctness and complexity analysis, computational and implementation considerations) and to many applications. I think it is overall the best introduction. <span class="citation" data-cites="vanderbei2014_linear">Vanderbei (<a href="#ref-vanderbei2014_linear">2014</a>)</span> follows a very similar outline, but contains more recent computational considerations<span><label for="sn-5" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-5" class="margin-toggle" /><span class="sidenote">For all the details about practical implementations of the simplex algorithm, <span class="citation" data-cites="maros2003_comput">Maros (<a href="#ref-maros2003_comput">2003</a>)</span> is dedicated to the computational aspects and contains everything you will need.<br />
|
||||
<br />
|
||||
</span></span>. (The author also has <a href="http://vanderbei.princeton.edu/307/lectures.html">lecture slides</a>.)</p>
|
||||
<p>For more books on linear programming, the two books <span class="citation" data-cites="dantzig1997_linear">Dantzig (<a href="#ref-dantzig1997_linear">1997</a>)</span>, <span class="citation" data-cites="dantzig2003_linear">Dantzig (<a href="#ref-dantzig2003_linear">2003</a>)</span> are very complete, if somewhat more mathematically advanced. <span class="citation" data-cites="bertsimas1997_introd">Bertsimas and Tsitsiklis (<a href="#ref-bertsimas1997_introd">1997</a>)</span> is also a great reference, if you can find it.</p>
|
||||
<p>For all the other subfields, <a href="https://or.stackexchange.com/a/870">this great StackExchange answer</a> contains a lot of useful references, including most of the above. Of particular note are <span class="citation" data-cites="peyreComputationalOptimalTransport2019">Peyré and Cuturi (<a href="#ref-peyreComputationalOptimalTransport2019">2019</a>)</span> for optimal transport, <span class="citation" data-cites="boyd2004_convex">Boyd (<a href="#ref-boyd2004_convex">2004</a>)</span> for convex optimization (<a href="https://web.stanford.edu/~boyd/cvxbook/">freely available online</a>), and <span class="citation" data-cites="nocedal2006_numer">Nocedal (<a href="#ref-nocedal2006_numer">2006</a>)</span> for numerical optimization. <span class="citation" data-cites="kochenderfer2019_algor">Kochenderfer (<a href="#ref-kochenderfer2019_algor">2019</a>)</span> is not in the list (because it is very recent) but is also excellent, with examples in Julia covering nearly every kind of optimization algorithms.</p>
|
||||
<p>For all the other subfields, <a href="https://or.stackexchange.com/a/870">this great StackExchange answer</a> contains a lot of useful references, including most of the above. Of particular note are <span class="citation" data-cites="peyreComputationalOptimalTransport2019">Peyré and Cuturi (<a href="#ref-peyreComputationalOptimalTransport2019">2019</a>)</span> for optimal transport, <span class="citation" data-cites="boyd2004_convex">Boyd (<a href="#ref-boyd2004_convex">2004</a>)</span> for convex optimization (<a href="https://web.stanford.edu/~boyd/cvxbook/">freely available online</a>), and <span class="citation" data-cites="nocedal2006_numer">Nocedal (<a href="#ref-nocedal2006_numer">2006</a>)</span> for numerical optimization. <span class="citation" data-cites="kochenderfer2019_algor">Kochenderfer (<a href="#ref-kochenderfer2019_algor">2019</a>)</span><span><label for="sn-6" class="margin-toggle">⊕</label><input type="checkbox" id="sn-6" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="../images/or_references/kochenderfer.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span> is not in the list (because it is very recent) but is also excellent, with examples in Julia covering nearly every kind of optimization algorithms.</p>
|
||||
<h3 id="online-courses">Online courses</h3>
|
||||
<p>If you would like to watch video lectures, there are a few good opportunities freely available online, in particular on <a href="https://ocw.mit.edu/index.htm">MIT OpenCourseWare</a>. The list of courses at MIT is available <a href="https://orc.mit.edu/academics/course-offerings">on their webpage</a>. I haven’t actually looked in details at the courses content<span><label for="sn-4" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-4" class="margin-toggle" /><span class="sidenote">I am more comfortable reading books than watching lecture videos online. Although I liked attending classes during my studies, I do not have the same feeling in front of a video. When I read, I can re-read three times the same sentence, pause to look up something, or skim a few paragraphs. I find that the inability to do that with a video diminishes greatly my ability to concentrate.<br />
|
||||
<p>If you would like to watch video lectures, there are a few good opportunities freely available online, in particular on <a href="https://ocw.mit.edu/index.htm">MIT OpenCourseWare</a>. The list of courses at MIT is available <a href="https://orc.mit.edu/academics/course-offerings">on their webpage</a>. I haven’t actually looked in details at the courses content<span><label for="sn-7" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-7" class="margin-toggle" /><span class="sidenote">I am more comfortable reading books than watching lecture videos online. Although I liked attending classes during my studies, I do not have the same feeling in front of a video. When I read, I can re-read three times the same sentence, pause to look up something, or skim a few paragraphs. I find that the inability to do that with a video diminishes greatly my ability to concentrate.<br />
|
||||
<br />
|
||||
</span></span>, so I cannot vouch for them directly, but MIT courses are generally of excellent quality. Most courses are also taught by Bertsimas and Bertsekas, who are very famous and wrote many excellent books.</p>
|
||||
<p>Of particular notes are:</p>
|
||||
|
|
@ -83,12 +95,16 @@
|
|||
<li><a href="https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-972-algebraic-techniques-and-semidefinite-optimization-spring-2006/">Algebraic Techniques and Semidefinite Optimization</a>,</li>
|
||||
<li><a href="https://ocw.mit.edu/courses/sloan-school-of-management/15-083j-integer-programming-and-combinatorial-optimization-fall-2009/">Integer Programming and Combinatorial Optimization</a>.</li>
|
||||
</ul>
|
||||
<p>Another interesting course I found online is <a href="https://www.ams.jhu.edu/~wcook12/dl/index.html">Deep Learning in Discrete Optimization</a>, at Johns Hopkins<span><label for="sn-5" class="margin-toggle">⊕</label><input type="checkbox" id="sn-5" class="margin-toggle" /><span class="marginnote"> It is taught by William Cook, who is the author of <a href="https://press.princeton.edu/books/paperback/9780691163529/in-pursuit-of-the-traveling-salesman"><em>In Pursuit of the Traveling Salesman</em></a>, a nice introduction to the TSP problem in a readable form.<br />
|
||||
<p>Another interesting course I found online is <a href="https://www.ams.jhu.edu/~wcook12/dl/index.html">Deep Learning in Discrete Optimization</a>, at Johns Hopkins<span><label for="sn-8" class="margin-toggle">⊕</label><input type="checkbox" id="sn-8" class="margin-toggle" /><span class="marginnote"> It is taught by William Cook, who is the author of <a href="https://press.princeton.edu/books/paperback/9780691163529/in-pursuit-of-the-traveling-salesman"><em>In Pursuit of the Traveling Salesman</em></a>, a nice introduction to the TSP problem in a readable form.<br />
|
||||
<br />
|
||||
</span></span>. It contains an interesting overview of deep learning and integer programming, with a focus on connections, and applications to recent research areas in ML (reinforcement learning, attention, etc.).</p>
|
||||
<h2 id="solvers-and-computational-resources">Solvers and computational resources <span id="solvers"></span></h2>
|
||||
<p>When you start reading about modelling and algorithms, I recommend you try solving a few problems yourself, either by hand for small instances, or using an existing solver. It will allow you to follow the examples in books, while also practising your modelling skills. You will also get an intuition of what is difficult to model and to solve.</p>
|
||||
<p>There are many solvers available, both free and commercial, with various capabilities. I recommend you use the fantastic <a href="https://github.com/JuliaOpt/JuMP.jl">JuMP</a> library for Julia, which exposes a domain-specific language for modelling, along with interfaces to nearly all major solver packages. (Even if you don’t know Julia, this is a great and easy way to start!) If you’d rather use Python, you can use Google’s <a href="https://developers.google.com/optimization/introduction/python">OR-Tools</a> or <a href="https://github.com/coin-or/pulp">PuLP</a> for linear programming.</p>
|
||||
<p>There are many solvers available, both free and commercial, with various capabilities. I recommend you use the fantastic <a href="https://github.com/JuliaOpt/JuMP.jl">JuMP</a><span><label for="sn-9" class="margin-toggle">⊕</label><input type="checkbox" id="sn-9" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="../images/or_references/jump.svg" width="250" /><br />
|
||||
<br />
|
||||
</span></span> library for Julia, which exposes a domain-specific language for modelling, along with interfaces to nearly all major solver packages. (Even if you don’t know Julia, this is a great and easy way to start!) If you’d rather use Python, you can use Google’s <a href="https://developers.google.com/optimization/introduction/python">OR-Tools</a> or <a href="https://github.com/coin-or/pulp">PuLP</a> for linear programming.</p>
|
||||
<p>Regarding solvers, there is a <a href="http://www.juliaopt.org/JuMP.jl/stable/installation/#Getting-Solvers-1">list of solvers</a> on JuMP’s documentation, with their capabilities and their license. Free solvers include <a href="https://www.gnu.org/software/glpk/">GLPK</a> (linear programming), <a href="https://github.com/coin-or/Ipopt">Ipopt</a> (non-linear programming), and <a href="https://scip.zib.de/">SCIP</a> (mixed-integer linear programming).</p>
|
||||
<p>Commercial solvers often have better performance, and some of them propose a free academic license: <a href="https://www.mosek.com/">MOSEK</a>, <a href="https://www.gurobi.com/">Gurobi</a>, and <a href="https://www.ibm.com/analytics/cplex-optimizer">IBM CPLEX</a> in particular all offer free academic licenses and work very well with JuMP.</p>
|
||||
<p>Another awesome resource is the <a href="https://neos-server.org/neos/">NEOS Server</a>. It offers free computing resources for numerical optimization, including all major free and commercial solvers! You can submit jobs on it in a standard format, or interface your favourite programming language with it. The fact that such an amazing resource exists for free, for everyone is extraordinary. They also have an accompanying book, the <a href="https://neos-guide.org/">NEOS Guide</a>, containing many case studies and description of problem types. The <a href="https://neos-guide.org/content/optimization-taxonomy">taxonomy</a> may be particularly useful.</p>
|
||||
|
|
|
|||
|
|
@ -28,18 +28,30 @@
|
|||
<h3 id="introduction-and-modelling">Introduction and modelling</h3>
|
||||
<p>For an overall introduction, I recommend <span class="citation" data-cites="wentzel1988_operat">Wentzel (<a href="#ref-wentzel1988_operat">1988</a>)</span>. It is an old book, published by Mir Publications, a Soviet publisher which published many excellent scientific textbooks<span><label for="sn-2" class="margin-toggle">⊕</label><input type="checkbox" id="sn-2" class="margin-toggle" /><span class="marginnote"> Mir also published <a href="https://mirtitles.org/2011/06/03/physics-for-everyone/"><em>Physics for Everyone</em></a> by Lev Landau and Alexander Kitaigorodsky, a three-volume introduction to physics that is really accessible. Together with Feynman’s famous <a href="https://www.feynmanlectures.caltech.edu/">lectures</a>, I read them (in French) when I was a kid, and it was the best introduction I could possibly have to the subject.<br />
|
||||
<br />
|
||||
</span></span>. It is out of print, but it is available <a href="https://archive.org/details/WentzelOperationsResearchMir1983">on Archive.org</a>. The book is quite old, but everything presented is still extremely relevant today. It requires absolutely no background, and covers everything: a general introduction to the field, linear programming, dynamic programming, Markov processes and queues, Monte Carlo methods, and game theory. Even if you already know some of these topics, the presentations is so clear that it is a pleasure to read! (In particular, it is one of the best presentations of dynamic programming that I have ever read. The explanation of the simplex algorithm is also excellent.)</p>
|
||||
</span></span>. It is out of print, but it is available <a href="https://archive.org/details/WentzelOperationsResearchMir1983">on Archive.org</a><span><label for="sn-3" class="margin-toggle">⊕</label><input type="checkbox" id="sn-3" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/wentzel.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span>. The book is quite old, but everything presented is still extremely relevant today. It requires absolutely no background, and covers everything: a general introduction to the field, linear programming, dynamic programming, Markov processes and queues, Monte Carlo methods, and game theory. Even if you already know some of these topics, the presentations is so clear that it is a pleasure to read! (In particular, it is one of the best presentations of dynamic programming that I have ever read. The explanation of the simplex algorithm is also excellent.)</p>
|
||||
<p>If you are interested in optimization, the first thing you have to learn is modelling, i.e. transforming your problem (described in natural language, often from a particular industrial application) into a mathematical programme. The mathematical programme is the structure on which you will be able to apply an algorithm to find an optimal solution. Even if (like me) you are initially more interested in the algorithmic side of things, learning to create models will shed a lot of light on the overall process, and will give you more insight in general on the reasoning behind algorithms.</p>
|
||||
<p>The best book I have read on the subject is <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span>. It contains a lot of concrete, step-by-step examples on concrete applications, in a multitude of domains, and remains very easy to read and to follow. It covers nearly every type of problem, so it is very useful as a reference. When you encounter a concrete problem in real life afterwards, you will know how to construct an appropriate model, and in the process you will often identify a common type of problem. The book then gives plenty of advice on how to approach each type of problem. Finally, it is also a great resource to build a “mental map” of the field, avoiding getting lost in the jungle of linear, stochastic, mixed integer, quadratic, and other network problems.</p>
|
||||
<p>The best book I have read on the subject is <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span><span><label for="sn-4" class="margin-toggle">⊕</label><input type="checkbox" id="sn-4" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/williams.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span>. It contains a lot of concrete, step-by-step examples on concrete applications, in a multitude of domains, and remains very easy to read and to follow. It covers nearly every type of problem, so it is very useful as a reference. When you encounter a concrete problem in real life afterwards, you will know how to construct an appropriate model, and in the process you will often identify a common type of problem. The book then gives plenty of advice on how to approach each type of problem. Finally, it is also a great resource to build a “mental map” of the field, avoiding getting lost in the jungle of linear, stochastic, mixed integer, quadratic, and other network problems.</p>
|
||||
<p>Another interesting resource is the freely available <a href="https://docs.mosek.com/modeling-cookbook/index.html">MOSEK Modeling Cookbook</a>, covering many types of problems, with more mathematical details than in <span class="citation" data-cites="williams2013_model">Williams (<a href="#ref-williams2013_model">2013</a>)</span>. It is built for people wanting to use the commercial <a href="https://www.mosek.com/">MOSEK</a> solver, so it could be useful if you plan to use a solver package like this one (more details on solvers <a href="#solvers">below</a>).</p>
|
||||
<h3 id="theory-and-algorithms">Theory and algorithms</h3>
|
||||
<p>The basic algorithm for optimization is the <a href="https://en.wikipedia.org/wiki/Simplex_algorithm">simplex algorithm</a>, developed by Dantzig in the 1940s to solve <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a> problems. It is the one of the main building blocks for mathematical optimization, and is used and referenced extensively in all kinds of approaches. As such, it is really important to understand it in detail. There are many books on the subject, but I especially liked <span class="citation" data-cites="chvatal1983_linear">Chvátal (<a href="#ref-chvatal1983_linear">1983</a>)</span> (out of print, but you can find cheap used versions on Amazon). It covers everything there is to know on the simplex algorithms (step-by-step explanations with simple examples, correctness and complexity analysis, computational and implementation considerations) and to many applications. I think it is overall the best introduction. <span class="citation" data-cites="vanderbei2014_linear">Vanderbei (<a href="#ref-vanderbei2014_linear">2014</a>)</span> follows a very similar outline, but contains more recent computational considerations<span><label for="sn-3" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-3" class="margin-toggle" /><span class="sidenote">For all the details about practical implementations of the simplex algorithm, <span class="citation" data-cites="maros2003_comput">Maros (<a href="#ref-maros2003_comput">2003</a>)</span> is dedicated to the computational aspects and contains everything you will need.<br />
|
||||
<p>The basic algorithm for optimization is the <a href="https://en.wikipedia.org/wiki/Simplex_algorithm">simplex algorithm</a>, developed by Dantzig in the 1940s to solve <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a> problems. It is the one of the main building blocks for mathematical optimization, and is used and referenced extensively in all kinds of approaches. As such, it is really important to understand it in detail. There are many books on the subject, but I especially liked <span class="citation" data-cites="chvatal1983_linear">Chvátal (<a href="#ref-chvatal1983_linear">1983</a>)</span> (out of print, but you can find cheap used versions on Amazon). It covers everything there is to know on the simplex algorithms (step-by-step explanations with simple examples, correctness and complexity analysis, computational and implementation considerations) and to many applications. I think it is overall the best introduction. <span class="citation" data-cites="vanderbei2014_linear">Vanderbei (<a href="#ref-vanderbei2014_linear">2014</a>)</span> follows a very similar outline, but contains more recent computational considerations<span><label for="sn-5" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-5" class="margin-toggle" /><span class="sidenote">For all the details about practical implementations of the simplex algorithm, <span class="citation" data-cites="maros2003_comput">Maros (<a href="#ref-maros2003_comput">2003</a>)</span> is dedicated to the computational aspects and contains everything you will need.<br />
|
||||
<br />
|
||||
</span></span>. (The author also has <a href="http://vanderbei.princeton.edu/307/lectures.html">lecture slides</a>.)</p>
|
||||
<p>For more books on linear programming, the two books <span class="citation" data-cites="dantzig1997_linear">Dantzig (<a href="#ref-dantzig1997_linear">1997</a>)</span>, <span class="citation" data-cites="dantzig2003_linear">Dantzig (<a href="#ref-dantzig2003_linear">2003</a>)</span> are very complete, if somewhat more mathematically advanced. <span class="citation" data-cites="bertsimas1997_introd">Bertsimas and Tsitsiklis (<a href="#ref-bertsimas1997_introd">1997</a>)</span> is also a great reference, if you can find it.</p>
|
||||
<p>For all the other subfields, <a href="https://or.stackexchange.com/a/870">this great StackExchange answer</a> contains a lot of useful references, including most of the above. Of particular note are <span class="citation" data-cites="peyreComputationalOptimalTransport2019">Peyré and Cuturi (<a href="#ref-peyreComputationalOptimalTransport2019">2019</a>)</span> for optimal transport, <span class="citation" data-cites="boyd2004_convex">Boyd (<a href="#ref-boyd2004_convex">2004</a>)</span> for convex optimization (<a href="https://web.stanford.edu/~boyd/cvxbook/">freely available online</a>), and <span class="citation" data-cites="nocedal2006_numer">Nocedal (<a href="#ref-nocedal2006_numer">2006</a>)</span> for numerical optimization. <span class="citation" data-cites="kochenderfer2019_algor">Kochenderfer (<a href="#ref-kochenderfer2019_algor">2019</a>)</span> is not in the list (because it is very recent) but is also excellent, with examples in Julia covering nearly every kind of optimization algorithms.</p>
|
||||
<p>For all the other subfields, <a href="https://or.stackexchange.com/a/870">this great StackExchange answer</a> contains a lot of useful references, including most of the above. Of particular note are <span class="citation" data-cites="peyreComputationalOptimalTransport2019">Peyré and Cuturi (<a href="#ref-peyreComputationalOptimalTransport2019">2019</a>)</span> for optimal transport, <span class="citation" data-cites="boyd2004_convex">Boyd (<a href="#ref-boyd2004_convex">2004</a>)</span> for convex optimization (<a href="https://web.stanford.edu/~boyd/cvxbook/">freely available online</a>), and <span class="citation" data-cites="nocedal2006_numer">Nocedal (<a href="#ref-nocedal2006_numer">2006</a>)</span> for numerical optimization. <span class="citation" data-cites="kochenderfer2019_algor">Kochenderfer (<a href="#ref-kochenderfer2019_algor">2019</a>)</span><span><label for="sn-6" class="margin-toggle">⊕</label><input type="checkbox" id="sn-6" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/kochenderfer.jpg" width="200" /><br />
|
||||
<br />
|
||||
</span></span> is not in the list (because it is very recent) but is also excellent, with examples in Julia covering nearly every kind of optimization algorithms.</p>
|
||||
<h3 id="online-courses">Online courses</h3>
|
||||
<p>If you would like to watch video lectures, there are a few good opportunities freely available online, in particular on <a href="https://ocw.mit.edu/index.htm">MIT OpenCourseWare</a>. The list of courses at MIT is available <a href="https://orc.mit.edu/academics/course-offerings">on their webpage</a>. I haven’t actually looked in details at the courses content<span><label for="sn-4" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-4" class="margin-toggle" /><span class="sidenote">I am more comfortable reading books than watching lecture videos online. Although I liked attending classes during my studies, I do not have the same feeling in front of a video. When I read, I can re-read three times the same sentence, pause to look up something, or skim a few paragraphs. I find that the inability to do that with a video diminishes greatly my ability to concentrate.<br />
|
||||
<p>If you would like to watch video lectures, there are a few good opportunities freely available online, in particular on <a href="https://ocw.mit.edu/index.htm">MIT OpenCourseWare</a>. The list of courses at MIT is available <a href="https://orc.mit.edu/academics/course-offerings">on their webpage</a>. I haven’t actually looked in details at the courses content<span><label for="sn-7" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-7" class="margin-toggle" /><span class="sidenote">I am more comfortable reading books than watching lecture videos online. Although I liked attending classes during my studies, I do not have the same feeling in front of a video. When I read, I can re-read three times the same sentence, pause to look up something, or skim a few paragraphs. I find that the inability to do that with a video diminishes greatly my ability to concentrate.<br />
|
||||
<br />
|
||||
</span></span>, so I cannot vouch for them directly, but MIT courses are generally of excellent quality. Most courses are also taught by Bertsimas and Bertsekas, who are very famous and wrote many excellent books.</p>
|
||||
<p>Of particular notes are:</p>
|
||||
|
|
@ -50,12 +62,16 @@
|
|||
<li><a href="https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-972-algebraic-techniques-and-semidefinite-optimization-spring-2006/">Algebraic Techniques and Semidefinite Optimization</a>,</li>
|
||||
<li><a href="https://ocw.mit.edu/courses/sloan-school-of-management/15-083j-integer-programming-and-combinatorial-optimization-fall-2009/">Integer Programming and Combinatorial Optimization</a>.</li>
|
||||
</ul>
|
||||
<p>Another interesting course I found online is <a href="https://www.ams.jhu.edu/~wcook12/dl/index.html">Deep Learning in Discrete Optimization</a>, at Johns Hopkins<span><label for="sn-5" class="margin-toggle">⊕</label><input type="checkbox" id="sn-5" class="margin-toggle" /><span class="marginnote"> It is taught by William Cook, who is the author of <a href="https://press.princeton.edu/books/paperback/9780691163529/in-pursuit-of-the-traveling-salesman"><em>In Pursuit of the Traveling Salesman</em></a>, a nice introduction to the TSP problem in a readable form.<br />
|
||||
<p>Another interesting course I found online is <a href="https://www.ams.jhu.edu/~wcook12/dl/index.html">Deep Learning in Discrete Optimization</a>, at Johns Hopkins<span><label for="sn-8" class="margin-toggle">⊕</label><input type="checkbox" id="sn-8" class="margin-toggle" /><span class="marginnote"> It is taught by William Cook, who is the author of <a href="https://press.princeton.edu/books/paperback/9780691163529/in-pursuit-of-the-traveling-salesman"><em>In Pursuit of the Traveling Salesman</em></a>, a nice introduction to the TSP problem in a readable form.<br />
|
||||
<br />
|
||||
</span></span>. It contains an interesting overview of deep learning and integer programming, with a focus on connections, and applications to recent research areas in ML (reinforcement learning, attention, etc.).</p>
|
||||
<h2 id="solvers-and-computational-resources">Solvers and computational resources <span id="solvers"></span></h2>
|
||||
<p>When you start reading about modelling and algorithms, I recommend you try solving a few problems yourself, either by hand for small instances, or using an existing solver. It will allow you to follow the examples in books, while also practising your modelling skills. You will also get an intuition of what is difficult to model and to solve.</p>
|
||||
<p>There are many solvers available, both free and commercial, with various capabilities. I recommend you use the fantastic <a href="https://github.com/JuliaOpt/JuMP.jl">JuMP</a> library for Julia, which exposes a domain-specific language for modelling, along with interfaces to nearly all major solver packages. (Even if you don’t know Julia, this is a great and easy way to start!) If you’d rather use Python, you can use Google’s <a href="https://developers.google.com/optimization/introduction/python">OR-Tools</a> or <a href="https://github.com/coin-or/pulp">PuLP</a> for linear programming.</p>
|
||||
<p>There are many solvers available, both free and commercial, with various capabilities. I recommend you use the fantastic <a href="https://github.com/JuliaOpt/JuMP.jl">JuMP</a><span><label for="sn-9" class="margin-toggle">⊕</label><input type="checkbox" id="sn-9" class="margin-toggle" /><span class="marginnote"><br />
|
||||
<br />
|
||||
<img src="/images/or_references/jump.svg" width="250" /><br />
|
||||
<br />
|
||||
</span></span> library for Julia, which exposes a domain-specific language for modelling, along with interfaces to nearly all major solver packages. (Even if you don’t know Julia, this is a great and easy way to start!) If you’d rather use Python, you can use Google’s <a href="https://developers.google.com/optimization/introduction/python">OR-Tools</a> or <a href="https://github.com/coin-or/pulp">PuLP</a> for linear programming.</p>
|
||||
<p>Regarding solvers, there is a <a href="http://www.juliaopt.org/JuMP.jl/stable/installation/#Getting-Solvers-1">list of solvers</a> on JuMP’s documentation, with their capabilities and their license. Free solvers include <a href="https://www.gnu.org/software/glpk/">GLPK</a> (linear programming), <a href="https://github.com/coin-or/Ipopt">Ipopt</a> (non-linear programming), and <a href="https://scip.zib.de/">SCIP</a> (mixed-integer linear programming).</p>
|
||||
<p>Commercial solvers often have better performance, and some of them propose a free academic license: <a href="https://www.mosek.com/">MOSEK</a>, <a href="https://www.gurobi.com/">Gurobi</a>, and <a href="https://www.ibm.com/analytics/cplex-optimizer">IBM CPLEX</a> in particular all offer free academic licenses and work very well with JuMP.</p>
|
||||
<p>Another awesome resource is the <a href="https://neos-server.org/neos/">NEOS Server</a>. It offers free computing resources for numerical optimization, including all major free and commercial solvers! You can submit jobs on it in a standard format, or interface your favourite programming language with it. The fact that such an amazing resource exists for free, for everyone is extraordinary. They also have an accompanying book, the <a href="https://neos-guide.org/">NEOS Guide</a>, containing many case studies and description of problem types. The <a href="https://neos-guide.org/content/optimization-taxonomy">taxonomy</a> may be particularly useful.</p>
|
||||
|
|
|
|||
1
images/or_references/jump.svg
Normal file
1
images/or_references/jump.svg
Normal file
|
|
@ -0,0 +1 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 377.42 122.03"><defs><style>.cls-1{fill:#231f20;}.cls-2,.cls-3,.cls-5,.cls-7,.cls-9{fill:none;}.cls-2,.cls-3{stroke:#231f20;stroke-linecap:round;stroke-linejoin:round;}.cls-2{stroke-width:6.63px;}.cls-3{stroke-width:6.63px;fill-rule:evenodd;}.cls-4{fill:#aa7dc0;}.cls-5{stroke:#945bb0;}.cls-5,.cls-7,.cls-9{stroke-miterlimit:10;stroke-width:3px;}.cls-6{fill:#6bab5b;}.cls-7{stroke:#3b972e;}.cls-8{fill:#d66661;}.cls-9{stroke:#c93d39;}</style></defs><title>Thin-Border-Logo-Text</title><g id="Page_1" data-name="Page 1"><g id="Layer_1-2" data-name="Layer 1"><polygon class="cls-1" points="313.71 117.21 326.3 117.21 326.3 50.53 311.9 50.53 291.01 79.2 270.12 50.53 255.62 50.53 255.62 117.21 268.21 117.21 268.21 68.72 289.48 97.78 291.96 97.78 313.71 68.81 313.71 117.21 313.71 117.21"/><path class="cls-1" d="M172.45,50.54H185.2V93.37c0,3.86.29,7.71-.65,11.5a19.26,19.26,0,0,1-7.72,11.33,19.75,19.75,0,0,1-9.07,3.42,20.85,20.85,0,0,1-22.67-16.11l12.38-3.16a8.45,8.45,0,0,0,5.17,6.09,8,8,0,0,0,3.67.45,7.09,7.09,0,0,0,3.2-1.16c2.77-1.93,2.94-5.34,2.94-8.34V50.54Zm26.46,0h12.52V74.35c0,4.61.31,7.84.95,9.63a8.46,8.46,0,0,0,3.06,4.19,8.76,8.76,0,0,0,5.2,1.51,9.15,9.15,0,0,0,5.24-1.47,8.68,8.68,0,0,0,3.2-4.34q.76-2.13.76-9.12V50.57h12.43V71.49c0,8.61-.69,14.52-2,17.69a19.21,19.21,0,0,1-7.34,8.9q-4.86,3.11-12.34,3.11-8.13,0-13.13-3.62a19.27,19.27,0,0,1-7.06-10.12c-.94-3-1.45-8.44-1.45-16.32V50.57Zm138.87,0h13.46q11,0,15.76,2a16,16,0,0,1,7.63,6.54,20.21,20.21,0,0,1,2.79,10.8,18.23,18.23,0,0,1-3.65,11.61A18.82,18.82,0,0,1,363.86,88q-3.66,1-13.37,1v28.26H337.78V50.54Zm12.71,26.08h4a22.42,22.42,0,0,0,6.63-.68,5.82,5.82,0,0,0,2.92-2.25,6.53,6.53,0,0,0,1.07-3.79,6.09,6.09,0,0,0-3-5.62C360.69,63.41,358,63,354.06,63h-3.57V76.62Z"/><line class="cls-2" x1="19.64" y1="67.28" x2="120.16" y2="28.42"/><polyline class="cls-3" points="3.32 118.72 89.75 32.29 118.72 3.32"/><path class="cls-4" d="M106.29,120.18a17.17,17.17,0,1,1,12.16-5A17.06,17.06,0,0,1,106.29,120.18Z"/><path class="cls-5" d="M106.29,120.18a17.17,17.17,0,1,1,12.16-5A17.06,17.06,0,0,1,106.29,120.18Z"/><path class="cls-6" d="M106.29,79.33a17.19,17.19,0,1,1,12.16-5A17.07,17.07,0,0,1,106.29,79.33Z"/><path class="cls-7" d="M106.29,79.33a17.19,17.19,0,1,1,12.16-5A17.07,17.07,0,0,1,106.29,79.33Z"/><path class="cls-8" d="M65.28,120.18a17.19,17.19,0,1,1,12.15-5A17,17,0,0,1,65.28,120.18Z"/><path class="cls-9" d="M65.28,120.18a17.19,17.19,0,1,1,12.15-5A17,17,0,0,1,65.28,120.18Z"/></g></g></svg>
|
||||
|
After Width: | Height: | Size: 2.5 KiB |
BIN
images/or_references/kochenderfer.jpg
Normal file
BIN
images/or_references/kochenderfer.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
BIN
images/or_references/wentzel.jpg
Normal file
BIN
images/or_references/wentzel.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 108 KiB |
BIN
images/or_references/williams.jpg
Normal file
BIN
images/or_references/williams.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 52 KiB |
|
|
@ -53,17 +53,21 @@ programming, stochastic processes, etc.
|
|||
For an overall introduction, I recommend cite:wentzel1988_operat. It
|
||||
is an old book, published by Mir Publications, a Soviet publisher
|
||||
which published many excellent scientific textbooks[fn:mir]. It is out
|
||||
of print, but it is available [[https://archive.org/details/WentzelOperationsResearchMir1983][on Archive.org]]. The book is quite old,
|
||||
but everything presented is still extremely relevant today. It
|
||||
requires absolutely no background, and covers everything: a general
|
||||
introduction to the field, linear programming, dynamic programming,
|
||||
Markov processes and queues, Monte Carlo methods, and game
|
||||
theory. Even if you already know some of these topics, the
|
||||
of print, but it is available [[https://archive.org/details/WentzelOperationsResearchMir1983][on Archive.org]][fn:wentzel]. The book is
|
||||
quite old, but everything presented is still extremely relevant
|
||||
today. It requires absolutely no background, and covers everything: a
|
||||
general introduction to the field, linear programming, dynamic
|
||||
programming, Markov processes and queues, Monte Carlo methods, and
|
||||
game theory. Even if you already know some of these topics, the
|
||||
presentations is so clear that it is a pleasure to read! (In
|
||||
particular, it is one of the best presentations of dynamic programming
|
||||
that I have ever read. The explanation of the simplex algorithm is
|
||||
also excellent.)
|
||||
|
||||
[fn:wentzel] {-}
|
||||
#+ATTR_HTML: :width 200px
|
||||
[[file:/images/or_references/wentzel.jpg]]
|
||||
|
||||
[fn:mir] {-} Mir also published [[https://mirtitles.org/2011/06/03/physics-for-everyone/][/Physics for Everyone/]] by Lev Landau
|
||||
and Alexander Kitaigorodsky, a three-volume introduction to physics
|
||||
that is really accessible. Together with Feynman's famous [[https://www.feynmanlectures.caltech.edu/][lectures]], I
|
||||
|
|
@ -82,18 +86,23 @@ of light on the overall process, and will give you more insight in
|
|||
general on the reasoning behind algorithms.
|
||||
|
||||
The best book I have read on the subject is
|
||||
cite:williams2013_model. It contains a lot of concrete, step-by-step
|
||||
examples on concrete applications, in a multitude of domains, and
|
||||
remains very easy to read and to follow. It covers nearly every type
|
||||
of problem, so it is very useful as a reference. When you encounter a
|
||||
concrete problem in real life afterwards, you will know how to
|
||||
construct an appropriate model, and in the process you will often
|
||||
identify a common type of problem. The book then gives plenty of
|
||||
cite:williams2013_model[fn:williams]. It contains a lot of concrete,
|
||||
step-by-step examples on concrete applications, in a multitude of
|
||||
domains, and remains very easy to read and to follow. It covers nearly
|
||||
every type of problem, so it is very useful as a reference. When you
|
||||
encounter a concrete problem in real life afterwards, you will know
|
||||
how to construct an appropriate model, and in the process you will
|
||||
often identify a common type of problem. The book then gives plenty of
|
||||
advice on how to approach each type of problem. Finally, it is also a
|
||||
great resource to build a "mental map" of the field, avoiding getting
|
||||
lost in the jungle of linear, stochastic, mixed integer, quadratic,
|
||||
and other network problems.
|
||||
|
||||
[fn:williams] {-}
|
||||
#+ATTR_HTML: :width 200px
|
||||
[[file:/images/or_references/williams.jpg]]
|
||||
|
||||
|
||||
Another interesting resource is the freely available [[https://docs.mosek.com/modeling-cookbook/index.html][MOSEK Modeling
|
||||
Cookbook]], covering many types of problems, with more mathematical
|
||||
details than in cite:williams2013_model. It is built for people
|
||||
|
|
@ -135,9 +144,15 @@ a lot of useful references, including most of the above. Of particular
|
|||
note are cite:peyreComputationalOptimalTransport2019 for optimal
|
||||
transport, cite:boyd2004_convex for convex optimization ([[https://web.stanford.edu/~boyd/cvxbook/][freely
|
||||
available online]]), and cite:nocedal2006_numer for numerical
|
||||
optimization. cite:kochenderfer2019_algor is not in the list (because
|
||||
it is very recent) but is also excellent, with examples in Julia
|
||||
covering nearly every kind of optimization algorithms.
|
||||
optimization. cite:kochenderfer2019_algor[fn:kochenderfer] is not in
|
||||
the list (because it is very recent) but is also excellent, with
|
||||
examples in Julia covering nearly every kind of optimization
|
||||
algorithms.
|
||||
|
||||
[fn:kochenderfer] {-}
|
||||
#+ATTR_HTML: :width 200px
|
||||
[[file:/images/or_references/kochenderfer.jpg]]
|
||||
|
||||
|
||||
** Online courses
|
||||
|
||||
|
|
@ -186,12 +201,17 @@ skills. You will also get an intuition of what is difficult to model
|
|||
and to solve.
|
||||
|
||||
There are many solvers available, both free and commercial, with
|
||||
various capabilities. I recommend you use the fantastic [[https://github.com/JuliaOpt/JuMP.jl][JuMP]] library
|
||||
for Julia, which exposes a domain-specific language for modelling,
|
||||
along with interfaces to nearly all major solver packages. (Even if
|
||||
you don't know Julia, this is a great and easy way to start!) If you'd
|
||||
rather use Python, you can use Google's [[https://developers.google.com/optimization/introduction/python][OR-Tools]] or [[https://github.com/coin-or/pulp][PuLP]] for linear
|
||||
programming.
|
||||
various capabilities. I recommend you use the fantastic [[https://github.com/JuliaOpt/JuMP.jl][JuMP]][fn:jump]
|
||||
library for Julia, which exposes a domain-specific language for
|
||||
modelling, along with interfaces to nearly all major solver
|
||||
packages. (Even if you don't know Julia, this is a great and easy way
|
||||
to start!) If you'd rather use Python, you can use Google's [[https://developers.google.com/optimization/introduction/python][OR-Tools]]
|
||||
or [[https://github.com/coin-or/pulp][PuLP]] for linear programming.
|
||||
|
||||
[fn:jump] {-}
|
||||
#+ATTR_HTML: :width 250px
|
||||
[[file:/images/or_references/jump.svg]]
|
||||
|
||||
|
||||
Regarding solvers, there is a [[http://www.juliaopt.org/JuMP.jl/stable/installation/#Getting-Solvers-1][list of solvers]] on JuMP's documentation,
|
||||
with their capabilities and their license. Free solvers include [[https://www.gnu.org/software/glpk/][GLPK]]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue