Danijar Hafner's guest appearances

Danijar Hafner

Towards Data Science

TalkRL: The Reinforcement Learning Podcast

Rohin Shah

Ben Goertzel

Jeremy Howard

Roland Memisevic

Kamyar Azizzadenesheli

Max Welling

Ayanna Howard

Mike Tung

Margaret Mitchell

Melanie Mitchell

Gillian Hadfield

Anders Sandberg

Brian Christian

Joel Grus

Jeffrey Ding

OwlTail

Danijar Hafner on the DreamerV3 agent and world models, the Director agent and heirarchical RL,  realtime RL on robots with DayDreamer, and his framework for unsupervised agent design! Danijar Hafner is a PhD candidate at the University of Toronto with Jimmy Ba, a visiting student at UC Berkeley with Pieter Abbeel, and an intern at DeepMind.  He has been our guest before back on episode 11.  Featured References   Mastering Diverse Domains through World Models [ blog ] DreaverV3 Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, Timothy Lillicrap  DayDreamer: World Models for Physical Robot Learning [ blog ]  Philipp Wu, Alejandro Escontrela, Danijar Hafner, Ken Goldberg, Pieter Abbeel  Deep Hierarchical Planning from Pixels [ blog ]  Danijar Hafner, Kuang-Huei Lee, Ian Fischer, Pieter Abbeel   Action and Perception as Divergence Minimization [ blog ]  Danijar Hafner, Pedro A. Ortega, Jimmy Ba, Thomas Parr, Karl Friston, Nicolas Heess  Additional References    Mastering Atari with Discrete World Models [ blog ] DreaverV2 ; Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, Jimmy Ba    Dream to Control: Learning Behaviors by Latent Imagination [ blog ] Dreamer ; Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi    Planning to Explore via Self-Supervised World Models ; Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak   

Danijar Hafner on the DreamerV3 agent and world models, the Director agent and heirarchical RL,  realtime RL on robots with DayDreamer, and his framework for unsupervised agent design! <a href="https://danijar.com/" rel="nofollow" target="_blank">Danijar Hafner</a> is a PhD candidate at the University of Toronto with Jimmy Ba, a visiting student at UC Berkeley with Pieter Abbeel, and an intern at DeepMind.  He has been our guest before back on episode 11.   Featured References    <a href="https://arxiv.org/abs/2301.04104v1" rel="nofollow" target="_blank">Mastering Diverse Domains through World Models</a> [ <a href="https://danijar.com/project/dreamerv3/" rel="nofollow" target="_blank">blog</a> ] DreaverV3 Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, Timothy Lillicrap   <a href="https://arxiv.org/abs/2206.14176" rel="nofollow" target="_blank">DayDreamer: World Models for Physical Robot Learning</a> [ <a href="https://danijar.com/project/daydreamer/" rel="nofollow" target="_blank">blog</a> ]  Philipp Wu, Alejandro Escontrela, Danijar Hafner, Ken Goldberg, Pieter Abbeel  <a href="https://arxiv.org/abs/2206.04114" rel="nofollow" target="_blank">Deep Hierarchical Planning from Pixels</a> [ <a href="https://danijar.com/project/director/" rel="nofollow" target="_blank">blog</a> ]  Danijar Hafner, Kuang-Huei Lee, Ian Fischer, Pieter Abbeel   <a href="https://arxiv.org/abs/2009.01791" rel="nofollow" target="_blank">Action and Perception as Divergence Minimization</a> [ <a href="https://danijar.com/project/apd/" rel="nofollow" target="_blank">blog</a> ]  Danijar Hafner, Pedro A. Ortega, Jimmy Ba, Thomas Parr, Karl Friston, Nicolas Heess  Additional References  <ul> <li> <a href="https://arxiv.org/abs/2010.02193" rel="nofollow" target="_blank">Mastering Atari with Discrete World Models</a> [ <a href="https://danijar.com/project/dreamerv2/" rel="nofollow" target="_blank">blog</a> ] DreaverV2 ; Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, Jimmy Ba  </li> <li> <a href="https://arxiv.org/abs/1912.01603" rel="nofollow" target="_blank">Dream to Control: Learning Behaviors by Latent Imagination</a> [ <a href="https://danijar.com/project/dreamer/" rel="nofollow" target="_blank">blog</a> ] Dreamer ; Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi  </li> <li> <a href="https://arxiv.org/abs/2005.05960" rel="nofollow" target="_blank">Planning to Explore via Self-Supervised World Models</a> ; Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak  </li> </ul>

Danijar Hafner 2

Until recently, AI systems have been narrow — they’ve only been able to perform the specific tasks that they were explicitly trained for. And while narrow systems are clearly useful, the holy grain of AI is to build more flexible, general systems. But that can’t be done without good performance metrics that we can optimize for — or that we can at least use to measure generalization ability. Somehow, we need to figure out what number needs to go up in order to bring us closer to generally-capable agents. That’s the question we’ll be exploring on this episode of the podcast, with Danijar Hafner. Danijar is a PhD student in artificial intelligence at the University of Toronto with Jimmy Ba and Geoffrey Hinton and researcher at Google Brain and the Vector Institute. Danijar has been studying the problem of performance measurement and benchmarking for RL agents with generalization abilities. As part of that work, he recently released Crafter, a tool that can procedurally generate complex environments that are a lot like Minecraft, featuring resources that need to be collected, tools that can be developed, and enemies who need to be avoided or defeated. In order to succeed in a Crafter environment, agents need to robustly plan, explore and test different strategies, which allow them to unlock certain in-game achievements. Crafter is part of a growing set of strategies that researchers are exploring to figure out how we can benchmark and measure the performance of general-purpose AIs, and it also tells us something interesting about the state of AI: increasingly, our ability to define tasks that require the right kind of generalization abilities is becoming just as important as innovating on AI model architectures. Danijar joined me to talk about Crafter, reinforcement learning, and the big challenges facing AI researchers as they work towards general intelligence on this episode of the TDS podcast. *** Intro music: - Artist: Ron Gelinas - Track Title: Daybreak Chill Blend (original mix) - Link to Track: https://youtu.be/d8Y2sKIgFWc *** Chapters:  0:00 Intro 2:25 Measuring generalization 5:40 What is Crafter? 11:10 Differences between Crafter and Minecraft 20:10 Agent behavior 25:30 Merging scaled models and reinforcement learning 29:30 Data efficiency 38:00 Hierarchical learning 43:20 Human-level systems 48:40 Cultural overlap 49:50 Wrap-up

Until recently, AI systems have been narrow — they’ve only been able to perform the specific tasks that they were explicitly trained for. And while narrow systems are clearly useful, the holy grain of AI is to build more flexible, general systems. But that can’t be done without good performance metrics that we can optimize for — or that we can at least use to measure generalization ability. Somehow, we need to figure out what number needs to go up in order to bring us closer to generally-capable agents. That’s the question we’ll be exploring on this episode of the podcast, with Danijar Hafner. Danijar is a PhD student in artificial intelligence at the <a href="http://learning.cs.toronto.edu/" rel="noopener noreferrer noopener nofollow" target="_blank">University of Toronto</a> with <a href="https://scholar.google.com/citations?user=ymzxRhAAAAAJ&amp;hl=en&amp;oi=ao" rel="noopener noreferrer noopener nofollow" target="_blank">Jimmy Ba</a> and <a href="https://scholar.google.com/citations?user=JicYPdAAAAAJ&amp;hl=en&amp;oi=ao" rel="noopener noreferrer noopener nofollow" target="_blank">Geoffrey Hinton</a> and researcher at <a href="https://research.google/teams/brain/" rel="noopener noreferrer noopener nofollow" target="_blank">Google Brain</a> and the <a href="https://vectorinstitute.ai/" rel="noopener noreferrer noopener nofollow" target="_blank">Vector Institute</a>. Danijar has been studying the problem of performance measurement and benchmarking for RL agents with generalization abilities. As part of that work, he recently released Crafter, a tool that can procedurally generate complex environments that are a lot like Minecraft, featuring resources that need to be collected, tools that can be developed, and enemies who need to be avoided or defeated. In order to succeed in a Crafter environment, agents need to robustly plan, explore and test different strategies, which allow them to unlock certain in-game achievements. Crafter is part of a growing set of strategies that researchers are exploring to figure out how we can benchmark and measure the performance of general-purpose AIs, and it also tells us something interesting about the state of AI: increasingly, our ability to define tasks that require the right kind of generalization abilities is becoming just as important as innovating on AI model architectures. Danijar joined me to talk about Crafter, reinforcement learning, and the big challenges facing AI researchers as they work towards general intelligence on this episode of the TDS podcast. *** Intro music: - Artist: Ron Gelinas - Track Title: Daybreak Chill Blend (original mix) - Link to Track: <a href="https://youtu.be/d8Y2sKIgFWc" rel="nofollow" target="_blank">https://youtu.be/d8Y2sKIgFWc</a> *** <h4>Chapters:</h4> <ul> <li>0:00 Intro</li> <li>2:25 Measuring generalization</li> <li>5:40 What is Crafter?</li> <li>11:10 Differences between Crafter and Minecraft</li> <li>20:10 Agent behavior</li> <li>25:30 Merging scaled models and reinforcement learning</li> <li>29:30 Data efficiency</li> <li>38:00 Hierarchical learning</li> <li>43:20 Human-level systems</li> <li>48:40 Cultural overlap</li> <li>49:50 Wrap-up</li></ul>

109. Danijar Hafner - Gaming our way to AGI

Danijar Hafner is a PhD student at the University of Toronto, and a student researcher at Google Research, Brain Team and the Vector Institute.  He holds a Masters of Research from University College London. Featured References   A deep learning framework for neuroscience Blake A. Richards, Timothy P. Lillicrap , Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, Colleen J. Gillon , Danijar Hafner, Adam Kepecs, Nikolaus Kriegeskorte, Peter Latham , Grace W. Lindsay, Kenneth D. Miller , Richard Naud , Christopher C. Pack, Panayiota Poirazi , Pieter Roelfsema , João Sacramento, Andrew Saxe, Benjamin Scellier, Anna C. Schapiro , Walter Senn, Greg Wayne, Daniel Yamins, Friedemann Zenke, Joel Zylberberg, Denis Therien, Konrad P. Kording   Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson   Dream to Control: Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi   Planning to Explore via Self-Supervised World Models Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak  Additional References  Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model Schrittwieser et al   Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm Silver et al   Shaping Belief States with Generative Environment Models for RL  Gregor et al   Model-Based Active Exploration Shyam et al   Errata [Robin] Around 1:37 I say "some ... world models get confused by random noise". I meant "some curiosity formulations", not "world models" 

<a href="https://danijar.com/" rel="nofollow" target="_blank">Danijar Hafner</a> is a PhD student at the University of Toronto, and a student researcher at Google Research, Brain Team and the Vector Institute.  He holds a Masters of Research from University College London. Featured References <ul> <li> <a href="https://www.nature.com/articles/s41593-019-0520-2.epdf?shared_access_token=n1zyUZ6-ypeHWkeaEs1FPNRgN0jAjWel9jnR3ZoTv0N5dsTXXcjpcGP7i54eL_L9GTMgy1V6NUDPE4-SxE_8Ip1gIa5G35VU4LeqRZ56IGy5uMJKd6aUZ4JeYonqPfWkstTCNFgazGPl8xJGrQAvuw%3D%3D" rel="nofollow" target="_blank">A deep learning framework for neuroscience</a> Blake A. Richards, Timothy P. Lillicrap , Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, Colleen J. Gillon , Danijar Hafner, Adam Kepecs, Nikolaus Kriegeskorte, Peter Latham , Grace W. Lindsay, Kenneth D. Miller , Richard Naud , Christopher C. Pack, Panayiota Poirazi , Pieter Roelfsema , João Sacramento, Andrew Saxe, Benjamin Scellier, Anna C. Schapiro , Walter Senn, Greg Wayne, Daniel Yamins, Friedemann Zenke, Joel Zylberberg, Denis Therien, Konrad P. Kording </li> <li> <a href="https://arxiv.org/abs/1811.04551" rel="nofollow" target="_blank">Learning Latent Dynamics for Planning from Pixels</a> Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson </li> <li> <a href="https://arxiv.org/abs/1912.01603" rel="nofollow" target="_blank">Dream to Control: Learning Behaviors by Latent Imagination</a> Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi </li> <li> <a href="https://arxiv.org/abs/2005.05960" rel="nofollow" target="_blank">Planning to Explore via Self-Supervised World Models</a> Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, Deepak Pathak </li> </ul> Additional References<ul> <li> <a href="https://arxiv.org/abs/1911.08265" rel="nofollow" target="_blank">Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model</a> Schrittwieser et al </li> <li> <a href="https://arxiv.org/abs/1712.01815" rel="nofollow" target="_blank">Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm</a> Silver et al </li> <li> <a href="https://arxiv.org/abs/1906.09237" rel="nofollow" target="_blank">Shaping Belief States with Generative Environment Models for RL</a>  Gregor et al </li> <li> <a href="https://arxiv.org/abs/1810.12162" rel="nofollow" target="_blank">Model-Based Active Exploration</a> Shyam et al </li> </ul> Errata <ul><li>[Robin] Around 1:37 I say "some ... world models get confused by random noise". I meant "some curiosity formulations", not "world models" </li></ul>