dimitri/blog
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add workshop notes

Browse source
This commit is contained in:
Dimitri Lozeve 2020-05-05 12:12:00 +02:00
parent 6f50c32ed1
commit 17b25af9c5
6 changed files with 113 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

74
posts/iclr-2020-notes.org
View file

@ -1,5 +1,5 @@
---
title: "ICLR 2020 Notes"
title: "ICLR 2020 Notes: Speakers and Workshops"
date: 2020-05-05
---
@ -41,7 +41,7 @@ The thing I appreciated most about the conference format was its
emphasis on /asynchronous/ communication. Given how little time they
had to plan the conference, they could have made all poster
presentations via video-conference and call it a day. Instead, each
poster had to record a 5-minute video summarising their
poster had to record a 5-minute video[fn:slideslive] summarising their
research. Alongside each presentation, there was a dedicated
Rocket.Chat channel[fn:rocketchat] where anyone could ask a question
to the authors, or just show their appreciation for the work. This was
@ -62,6 +62,13 @@ All of these excellent ideas were implemented by an [[https://iclr.cc/virtual_20
collecting all papers in a searchable, easy-to-use interface, and even
a nice [[https://iclr.cc/virtual_2020/paper_vis.html][visualisation]] of papers as a point cloud!
[fn:slideslive] The videos are streamed using [[https://library.slideslive.com/][SlidesLive]], which is a
great solution for synchronising videos and slides. It is very
comfortable to navigate through the slides and synchronising the video
to the slides and vice-versa. As a result, SlidesLive also has a very
nice library of talks, including major conferences. This is much
better than browsing YouTube randomly.
[fn:rocketchat] [[https://rocket.chat/][Rocket.Chat]] seems to be an [[https://github.com/RocketChat/Rocket.Chat][open-source]] alternative to
Slack. Overall, the experience was great, and I appreciate the efforts
of the organizers to use open source software instead of proprietary
@ -135,12 +142,59 @@ very important concepts from cognitive science.
TODO
* Some Interesting Papers
** Natural Language Processing
** Reinforcement Learning
** ML and Neural Network Theory
* Workshops
On Sunday, there were [[https://iclr.cc/virtual_2020/workshops.html][15 different workshops]]. All of them were
recorded, and are available on the website. As always, unfortunately,
there are too many interesting things to watch everything, but I saw
bits and pieces of different workshops.
** [[https://iclr.cc/virtual_2020/workshops_12.html][Beyond 'tabula rasa' in reinforcement learning: agents that remember, adapt, and generalize]]
A lot of pretty advanced talks about RL. The general theme was
meta-learning, aka "learning to learn". This is a very active area of
research, which goes way beyond classical RL theory, and offer many
interesting avenues to adjacent fields (both inside ML and outside,
especially cognitive science). The [[http://www.betr-rl.ml/2020/abs/101/][first talk]], by Martha White, about
inductive biases, was a very interesting and approachable introduction
to the problems and challenges of the field. There was also a panel
with Jürgen Schmidhuber. We hear a lot about him from the various
controversies, but it's nice to see him talking about research and
future developments in RL.
** [[https://iclr.cc/virtual_2020/workshops_14.html][Causal Learning For Decision Making]]
Ever since I read Judea Pearl's [[https://www.goodreads.com/book/show/36204378-the-book-of-why][/The Book of Why/]] on causality, I have
been interested in how we can incorporate causality reasoning in
machine learning. This is a complex topic, and I'm not sure yet that
it is a complete revolution as Judea Pearl likes to portray it, but it
nevertheless introduces a lot of new fascinating ideas. Yoshua Bengio
gave an interesting talk[fn:bengioworkshop] (even though very similar
to his keynote talk) on causal priors for deep learning.
[fn:bengioworkshop] You can find it at 4:45:20 in the [[https://slideslive.com/38926830/workshop-on-causal-learning-for-decision-making][livestream]] of
the workshop.
** [[https://iclr.cc/virtual_2020/workshops_4.html][Bridging AI and Cognitive Science]]
Cognitive science is fascinating, and I believe that collaboration
between ML practitioners and cognitive scientists will greatly help
advance both fields. I only watched [[https://baicsworkshop.github.io/program/baics_45.html][Leslie Kaelbling's presentation]],
which echoes a lot of things from her talk at the main conference. It
complements it nicely, with more focus on intelligence, especially
/embodied/ intelligence. I think she has the rights approach to
relationships between AI and natural science, explicitly listing the
things from her work that would be helpful to natural scientists, and
things she wish she knew about natural intelligences. It raises many
fascinating questions on ourselves, what we build, and what we
understand. I felt it was very motivational!
** [[https://iclr.cc/virtual_2020/workshops_5.html][Integration of Deep Neural Models and Differential Equations]]
I didn't attend this workshop, but I think I will watch the
presentations if I can find some time. I have found the intersection
of differential equations and ML very interesting, ever since the
famous [[https://papers.nips.cc/paper/7892-neural-ordinary-differential-equations][NeurIPS best paper]] on Neural ODEs. I think that such
improvements to ML theory from other fields in mathematics would be
extremely beneficial to a better understanding of the systems we
build.