Deep Thinkers on Deep Learning
I moderated a legendary panel of machine intelligence pioneers in Toronto today: Russ Salakhutdinov (the new head of AI at Apple), Rich Sutton (Univ. of Alberta, author of Reinforcement Learning), Geoff Hinton (Univ. of Toronto, Google, DL patron saint), and Yoshua Bengio (Univ. of Montreal and head of MILA),
All were deeply inspired by the biology of the brain and wanted to focus their future work on deeper biomimicry.
Ruslan: focusing on embedding memory for natural language understanding, dialog, Q&A and deep reinforcement learning. Unsupervised, one-shot and transfer learning.
Sutton: “AIs will compete and cooperate with us just like other people, but with greater diversity and asymmetries. So, we need to set up mechanisms (social, legal, political, cultural) to ensure this works out well. Inevitably, conventional humans will be less important. Step 1 – Lose your sense of entitlement. Step 2 – Include AIs in your circle of empathy.”
Hinton: Wants to find a better axon activation function (sigmoid and RELU being used for so long) and a better model of the synapse (with instantaneous shifts and retained state). His motivation is to understand the brain better. The only reason he applies his work to deep learning is that Google does a better job funding his research than medical research sources. Current neural nets are not good at coincidences or detecting sameness in time (the XOR problem). Turing did not believe the brain was a turing machine. He did work on neural networks.
Event Agenda. Thanks to Shivon for the photo.