146 lines
7.2 KiB
Org Mode
146 lines
7.2 KiB
Org Mode
---
|
|
title: "ICLR 2020 Notes"
|
|
date: 2020-05-05
|
|
---
|
|
|
|
ICLR is one of the most important conferences in machine learning, and
|
|
as such, I was very excited to have the opportunity to volunteer and
|
|
attend the first fully-virtual edition of the event. The whole content
|
|
of the conference has been made [[https://iclr.cc/virtual_2020/index.html][publicly available]], only a few days
|
|
after the end of the event!
|
|
|
|
I would like to thank the [[https://iclr.cc/Conferences/2020/Committees][organizing committee]] for this incredible
|
|
event, and the possibility to volunteer to help other
|
|
participants[fn:volunteer].
|
|
|
|
The many volunteers, the online-only nature of the event, and the low
|
|
registration fees also allowed for what felt like a very diverse,
|
|
inclusive event. Many graduate students and researchers from industry
|
|
(like me), who do not generally have the time or the resources to
|
|
travel to conferences like this, were able to attend, and make the
|
|
exchanges richer.
|
|
|
|
In this post, I will try to give my impressions on the event, and
|
|
share the most interesting events and papers I saw.
|
|
|
|
[fn:volunteer] To better organize the event, and help people navigate
|
|
the various online tools, they brought in 500(!) volunteers, waved our
|
|
registration fees, and asked us to do simple load-testing and tech
|
|
support. This was a very generous offer, and felt very rewarding for
|
|
us, as we could attend the conference, and give back to the
|
|
organization a little bit.
|
|
|
|
* The Format of the Virtual Conference
|
|
|
|
As a result of global travel restrictions, the conference was made
|
|
fully-virtual. It was supposed to take place in Addis Ababa, Ethiopia,
|
|
which is great for people who are often the target of restrictive visa
|
|
policies in Northern American countries.
|
|
|
|
The thing I appreciated most about the conference format was its
|
|
emphasis on /asynchronous/ communication. Given how little time they
|
|
had to plan the conference, they could have made all poster
|
|
presentations via video-conference and call it a day. Instead, each
|
|
poster had to record a 5-minute video summarising their
|
|
research. Alongside each presentation, there was a dedicated
|
|
Rocket.Chat channel[fn:rocketchat] where anyone could ask a question
|
|
to the authors, or just show their appreciation for the work. This was
|
|
a fantastic idea as it allowed any participant to interact with papers
|
|
and authors at any time they please, which is especially important in
|
|
a setting where people were spread all over the globe.
|
|
|
|
There were also Zoom session where authors were available for direct,
|
|
face-to-face discussions, allowing for more traditional
|
|
conversations. But asking questions on the channel had also the
|
|
advantage of keeping a track of all questions that were asked by other
|
|
people. As such, I quickly acquired the habit of watching the video,
|
|
looking at the chat to see the previous discussions (even if they
|
|
happened in the middle of the night in my timezone!), and then
|
|
skimming the paper or asking questions myself.
|
|
|
|
All of these excellent ideas were implemented by an [[https://iclr.cc/virtual_2020/papers.html?filter=keywords][amazing website]],
|
|
collecting all papers in a searchable, easy-to-use interface, and even
|
|
a nice [[https://iclr.cc/virtual_2020/paper_vis.html][visualisation]] of papers as a point cloud!
|
|
|
|
[fn:rocketchat] [[https://rocket.chat/][Rocket.Chat]] seems to be an [[https://github.com/RocketChat/Rocket.Chat][open-source]] alternative to
|
|
Slack. Overall, the experience was great, and I appreciate the efforts
|
|
of the organizers to use open source software instead of proprietary
|
|
applications. I hope other conferences will do the same, and perhaps
|
|
even avoid Zoom, because of recent privacy concerns (maybe try
|
|
[[https://jitsi.org/][Jitsi]]?).
|
|
|
|
* Speakers
|
|
|
|
Overall, there were 8 speakers (two for each day of the main
|
|
conference). They made a 40-minute presentation, and then there was a
|
|
Q&A both via the chat and via Zoom. I only saw 4 of them, but I expect
|
|
I will be watching the others in the near future.
|
|
|
|
** Prof. Leslie Kaelbling, [[https://iclr.cc/virtual_2020/speaker_2.html][Doing for Our Robots What Nature Did For Us]]
|
|
|
|
This talk was fascinating. It is about robotics, and especially how to
|
|
design the "software" of our robots. We want to program a robot in a
|
|
way that it could work the best it can over all possible domains it
|
|
can encounter. I loved the discussion on how to describe the space of
|
|
distributions over domains, from the point of view of the robot
|
|
factory:
|
|
- The domain could be very narrow (e.g. playing a specific Atari game)
|
|
or very broad and complex (performing a complex task in an open
|
|
world).
|
|
- The factory could know in advance in which domain the robot will
|
|
evolve, or have a lot of uncertainty around it.
|
|
|
|
There are many ways to describe a policy (i.e. the software running in
|
|
the robot's head), and many ways to obtain them. If you are familiar
|
|
with recent advances in reinforcement learning, this talk is a great
|
|
occasion to take a step back, and review the relevant background ideas
|
|
from engineering and control theory.
|
|
|
|
Finally, the most important take-away from this talk is the importance
|
|
of /abstractions/. Whatever the methods we use to program our robots,
|
|
we still need a lot of human insights to give them good structural
|
|
biases. There are many more insights, on the cost of experience,
|
|
(hierarchical) planning, learning constraints, etc, so I strongly
|
|
encourage you to watch the talk!
|
|
|
|
** Dr. Laurent Dinh, [[https://iclr.cc/virtual_2020/speaker_4.html][Invertible Models and Normalizing Flows]]
|
|
|
|
This is a talk about an area of ML research I do not know very well,
|
|
but very clearly presented. I really like the approach of teaching a
|
|
set of methods from a "historical", personal point of view. Laurent
|
|
Dinh shows us how he arrived at this topic, what he finds interesting,
|
|
in a very personal and relatable manner. This has the double advantage
|
|
of introducing us to a topic that he is passionate about, while also
|
|
giving us a glimpse of a researcher's process, without hiding the
|
|
momentary disillusions and disappointments, but emphasising the great
|
|
achievements. Normalizing flows are also very interesting because it
|
|
is grounded in strong theoretical results, that brings together a lot
|
|
of different methods.
|
|
|
|
** Profs. Yann LeCun and Yoshua Bengio, [[https://iclr.cc/virtual_2020/speaker_7.html][Reflections from the Turing Award Winners]]
|
|
|
|
This talk was very interesting, and yet felt very familiar, as if I
|
|
already saw a very similar one elsewhere. Especially for Yann LeCun,
|
|
who clearly reuses the same slides for many presentations at various
|
|
events. They both came back to their favourite subjects:
|
|
self-supervised learning for Yann LeCun, and system 1/system 2 for
|
|
Yoshua Bengio. All in all, they are very good speakers, and their
|
|
presentations are always insightful. Yann LeCun gives a lot of
|
|
references on recent technical advances, which is great if you want to
|
|
go deeper in the approaches he recommends. Yoshua Bengio is also very
|
|
good at broadening the debate around deep learning, and introducing
|
|
very important concepts from cognitive science.
|
|
|
|
** Prof. Michael I. Jordan, [[https://iclr.cc/virtual_2020/speaker_8.html][The Decision-Making Side of Machine Learning: Dynamical, Statistical and Economic Perspectives]]
|
|
|
|
TODO
|
|
|
|
* Some Interesting Papers
|
|
|
|
** Natural Language Processing
|
|
|
|
** Reinforcement Learning
|
|
|
|
** ML and Neural Network Theory
|
|
|
|
* Workshops
|