200 lines
11 KiB
Org Mode
200 lines
11 KiB
Org Mode
---
|
|
title: "ICLR 2020 Notes: Speakers and Workshops"
|
|
date: 2020-05-05
|
|
---
|
|
|
|
ICLR is one of the most important conferences in machine learning, and
|
|
as such, I was very excited to have the opportunity to volunteer and
|
|
attend the first fully-virtual edition of the event. The whole content
|
|
of the conference has been made [[https://iclr.cc/virtual_2020/index.html][publicly available]], only a few days
|
|
after the end of the event!
|
|
|
|
I would like to thank the [[https://iclr.cc/Conferences/2020/Committees][organizing committee]] for this incredible
|
|
event, and the possibility to volunteer to help other
|
|
participants[fn:volunteer].
|
|
|
|
The many volunteers, the online-only nature of the event, and the low
|
|
registration fees also allowed for what felt like a very diverse,
|
|
inclusive event. Many graduate students and researchers from industry
|
|
(like me), who do not generally have the time or the resources to
|
|
travel to conferences like this, were able to attend, and make the
|
|
exchanges richer.
|
|
|
|
In this post, I will try to give my impressions on the event, the
|
|
speakers, and the workshops that I could attend. I will do a quick
|
|
recap of the most interesting papers I saw in a future post.
|
|
|
|
[fn:volunteer] To better organize the event, and help people navigate
|
|
the various online tools, they brought in 500(!) volunteers, waved our
|
|
registration fees, and asked us to do simple load-testing and tech
|
|
support. This was a very generous offer, and felt very rewarding for
|
|
us, as we could attend the conference, and give back to the
|
|
organization a little bit.
|
|
|
|
* The Format of the Virtual Conference
|
|
|
|
As a result of global travel restrictions, the conference was made
|
|
fully-virtual. It was supposed to take place in Addis Ababa, Ethiopia,
|
|
which is great for people who are often the target of restrictive visa
|
|
policies in Northern American countries.
|
|
|
|
The thing I appreciated most about the conference format was its
|
|
emphasis on /asynchronous/ communication. Given how little time they
|
|
had to plan the conference, they could have made all poster
|
|
presentations via video-conference and call it a day. Instead, each
|
|
poster had to record a 5-minute video[fn:slideslive] summarising their
|
|
research. Alongside each presentation, there was a dedicated
|
|
Rocket.Chat channel[fn:rocketchat] where anyone could ask a question
|
|
to the authors, or just show their appreciation for the work. This was
|
|
a fantastic idea as it allowed any participant to interact with papers
|
|
and authors at any time they please, which is especially important in
|
|
a setting where people were spread all over the globe.
|
|
|
|
There were also Zoom session where authors were available for direct,
|
|
face-to-face discussions, allowing for more traditional
|
|
conversations. But asking questions on the channel had also the
|
|
advantage of keeping a track of all questions that were asked by other
|
|
people. As such, I quickly acquired the habit of watching the video,
|
|
looking at the chat to see the previous discussions (even if they
|
|
happened in the middle of the night in my timezone!), and then
|
|
skimming the paper or asking questions myself.
|
|
|
|
All of these excellent ideas were implemented by an [[https://iclr.cc/virtual_2020/papers.html?filter=keywords][amazing website]],
|
|
collecting all papers in a searchable, easy-to-use interface, and even
|
|
including a nice [[https://iclr.cc/virtual_2020/paper_vis.html][visualisation]] of papers as a point cloud!
|
|
|
|
[fn:slideslive] The videos are streamed using [[https://library.slideslive.com/][SlidesLive]], which is a
|
|
great solution for synchronising videos and slides. It is very
|
|
comfortable to navigate through the slides and synchronising the video
|
|
to the slides and vice-versa. As a result, SlidesLive also has a very
|
|
nice library of talks, including major conferences. This is much
|
|
better than browsing YouTube randomly.
|
|
|
|
[fn:rocketchat] [[https://rocket.chat/][Rocket.Chat]] seems to be an [[https://github.com/RocketChat/Rocket.Chat][open-source]] alternative to
|
|
Slack. Overall, the experience was great, and I appreciate the efforts
|
|
of the organizers to use open source software instead of proprietary
|
|
applications. I hope other conferences will do the same, and perhaps
|
|
even avoid Zoom, because of recent privacy concerns (maybe try
|
|
[[https://jitsi.org/][Jitsi]]?).
|
|
|
|
* Speakers
|
|
|
|
Overall, there were 8 speakers (two for each day of the main
|
|
conference). They made a 40-minute presentation, and then there was a
|
|
Q&A both via the chat and via Zoom. I only saw a few of them, but I
|
|
expect I will be watching the others in the near future.
|
|
|
|
** Prof. Leslie Kaelbling, [[https://iclr.cc/virtual_2020/speaker_2.html][Doing for Our Robots What Nature Did For Us]]
|
|
|
|
This talk was fascinating. It is about robotics, and especially how to
|
|
design the "software" of our robots. We want to program a robot in a
|
|
way that it could work the best it can over all possible domains it
|
|
can encounter. I loved the discussion on how to describe the space of
|
|
distributions over domains, from the point of view of the robot
|
|
factory:
|
|
- The domain could be very narrow (e.g. playing a specific Atari game)
|
|
or very broad and complex (performing a complex task in an open
|
|
world).
|
|
- The factory could know in advance in which domain the robot will
|
|
evolve, or have a lot of uncertainty around it.
|
|
|
|
There are many ways to describe a policy (i.e. the software running in
|
|
the robot's head), and many ways to obtain them. If you are familiar
|
|
with recent advances in reinforcement learning, this talk is a great
|
|
occasion to take a step back, and review the relevant background ideas
|
|
from engineering and control theory.
|
|
|
|
Finally, the most important take-away from this talk is the importance
|
|
of /abstractions/. Whatever the methods we use to program our robots,
|
|
we still need a lot of human insights to give them good structural
|
|
biases. There are many more insights, on the cost of experience,
|
|
(hierarchical) planning, learning constraints, etc, so I strongly
|
|
encourage you to watch the talk!
|
|
|
|
** Dr. Laurent Dinh, [[https://iclr.cc/virtual_2020/speaker_4.html][Invertible Models and Normalizing Flows]]
|
|
|
|
This is a very clear presentation of an area of ML research I do not
|
|
know very well. I really like the approach of teaching a set of
|
|
methods from a "historical", personal point of view. Laurent Dinh
|
|
shows us how he arrived at this topic, what he finds interesting, in a
|
|
very personal and relatable manner. This has the double advantage of
|
|
introducing us to a topic that he is passionate about, while also
|
|
giving us a glimpse of a researcher's process, without hiding the
|
|
momentary disillusions and disappointments, but emphasising the great
|
|
achievements. Normalizing flows are also very interesting because it
|
|
is grounded in strong theoretical results, that brings together a lot
|
|
of different methods.
|
|
|
|
** Profs. Yann LeCun and Yoshua Bengio, [[https://iclr.cc/virtual_2020/speaker_7.html][Reflections from the Turing Award Winners]]
|
|
|
|
This talk was very interesting, and yet felt very familiar, as if I
|
|
already saw a very similar one elsewhere. Especially for Yann LeCun,
|
|
who clearly reuses the same slides for many presentations at various
|
|
events. They both came back to their favourite subjects:
|
|
self-supervised learning for Yann LeCun, and system 1/system 2 for
|
|
Yoshua Bengio. All in all, they are very good speakers, and their
|
|
presentations are always insightful. Yann LeCun gives a lot of
|
|
references on recent technical advances, which is great if you want to
|
|
go deeper in the approaches he recommends. Yoshua Bengio is also very
|
|
good at broadening the debate around deep learning, and introducing
|
|
very important concepts from cognitive science.
|
|
|
|
# ** Prof. Michael I. Jordan, [[https://iclr.cc/virtual_2020/speaker_8.html][The Decision-Making Side of Machine Learning: Dynamical, Statistical and Economic Perspectives]]
|
|
|
|
# TODO
|
|
|
|
* Workshops
|
|
|
|
On Sunday, there were [[https://iclr.cc/virtual_2020/workshops.html][15 different workshops]]. All of them were
|
|
recorded, and are available on the website. As always, unfortunately,
|
|
there are too many interesting things to watch everything, but I saw
|
|
bits and pieces of different workshops.
|
|
|
|
** [[https://iclr.cc/virtual_2020/workshops_12.html][Beyond 'tabula rasa' in reinforcement learning: agents that remember, adapt, and generalize]]
|
|
|
|
A lot of pretty advanced talks about RL. The general theme was
|
|
meta-learning, aka "learning to learn". This is a very active area of
|
|
research, which goes way beyond classical RL theory, and offer many
|
|
interesting avenues to adjacent fields (both inside ML and outside,
|
|
especially cognitive science). The [[http://www.betr-rl.ml/2020/abs/101/][first talk]], by Martha White, about
|
|
inductive biases, was a very interesting and approachable introduction
|
|
to the problems and challenges of the field. There was also a panel
|
|
with Jürgen Schmidhuber. We hear a lot about him from the various
|
|
controversies, but it's nice to see him talking about research and
|
|
future developments in RL.
|
|
|
|
** [[https://iclr.cc/virtual_2020/workshops_14.html][Causal Learning For Decision Making]]
|
|
|
|
Ever since I read Judea Pearl's [[https://www.goodreads.com/book/show/36204378-the-book-of-why][/The Book of Why/]] on causality, I have
|
|
been interested in how we can incorporate causality reasoning in
|
|
machine learning. This is a complex topic, and I'm not sure yet that
|
|
it is a complete revolution as Judea Pearl likes to portray it, but it
|
|
nevertheless introduces a lot of new fascinating ideas. Yoshua Bengio
|
|
gave an interesting talk[fn:bengioworkshop] (even though very similar
|
|
to his keynote talk) on causal priors for deep learning.
|
|
|
|
[fn:bengioworkshop] You can find it at 4:45:20 in the [[https://slideslive.com/38926830/workshop-on-causal-learning-for-decision-making][livestream]] of
|
|
the workshop.
|
|
|
|
** [[https://iclr.cc/virtual_2020/workshops_4.html][Bridging AI and Cognitive Science]]
|
|
|
|
Cognitive science is fascinating, and I believe that collaboration
|
|
between ML practitioners and cognitive scientists will greatly help
|
|
advance both fields. I only watched [[https://baicsworkshop.github.io/program/baics_45.html][Leslie Kaelbling's presentation]],
|
|
which echoes a lot of things from her talk at the main conference. It
|
|
complements it nicely, with more focus on intelligence, especially
|
|
/embodied/ intelligence. I think she has the right approach to
|
|
relationships between AI and natural science, explicitly listing the
|
|
things from her work that would be helpful to natural scientists, and
|
|
things she wish she knew about natural intelligences. It raises many
|
|
fascinating questions on ourselves, what we build, and what we
|
|
understand. I felt it was very motivational!
|
|
|
|
** [[https://iclr.cc/virtual_2020/workshops_5.html][Integration of Deep Neural Models and Differential Equations]]
|
|
|
|
I didn't attend this workshop, but I think I will watch the
|
|
presentations if I can find the time. I have found the intersection of
|
|
differential equations and ML very interesting, ever since the famous
|
|
[[https://papers.nips.cc/paper/7892-neural-ordinary-differential-equations][NeurIPS best paper]] on Neural ODEs. I think that such improvements to
|
|
ML theory from other fields in mathematics would be extremely
|
|
beneficial to a better understanding of the systems we build.
|