Anirudh Goyal
anirudhgoyal9119 at gmail dot com

I am a graduate student in CS at University of Montreal. I am a part of Mila, advised by Prof. Yoshua Bengio.

Before graduate school, I received a Bachelors in Computer Science at IIIT Hyderabad, where I worked on several research projects at CVIT under Prof. C.V Jawahar. I have also spent time at Google.

Google Scholar  /  GitHub

News
Invited Talks and Lectures
Mentoring
Current
Aniket Didolkar
Vedant Shah
Nikhil Barhate
Frederik Träuble
Mihir Prabhudesai
Past
Soumye Singhal
Samarth Sinha
Sarthak Mittal
Phanideep Gampa
Ossama Ahmed
Kartikeya Badola
Kanika Madan
Yashas Annadani

Publications

Opinion Piece/Review Papers

Inductive Biases for Deep Learning of Higher-Level Cognition
Anirudh Goyal, Yoshua Bengio
arXiv

Towards Causal Representation Learning
Bernhard Schölkopf , Francesco Locatello , Nan Rosemary Ke , Stefan Bauer , Nal Kalchbrenner , Anirudh Goyal, Yoshua Bengio
arXiv

Preprints

Transformers with Competitive Ensembles of Independent Mechanisms
Alex Lamb , Di He, Anirudh Goyal, Guolin Ke, Chien-Feng Liao, Mirco Ravanelli, arXiv

Coordination Among Neural Modules Through a Shared Global Workspace
Anirudh Goyal, Aniket Didolkar, Alex Lamb , Kartikeya Badola , Nan Rosemary Ke , Nasim Rahaman , Jonathan Binas , Charles Blundell , Michael C. Mozer, Yoshua Bengio
arXiv / code

Learning Neural Causal Models from Active Interventions
Nino Scherrer , Olexa Bilaniuk , Yashas Annadani , Anirudh Goyal, Patrick Schwab , Bernhard Schölkopf, Michael C. Mozer, Yoshua Bengio , Stefan Bauer , Nan Rosemary Ke , arXiv

Learning Neural Causal Models from unknown Interventions
Nan Rosemary Ke , Olexa Bilaniuk , Anirudh Goyal, Stefan Bauer , Bernhard Schölkopf, Michael C. Mozer, Hugo Larochelle, Chris Pal , Yoshua Bengio
arXiv / code

Maximum Entropy Generators for Energy-Based Models
Rithesh Kumar , Sherjil Ozair , Anirudh Goyal, Aaron Courville , Yoshua Bengio
arXiv / code

Maximum Entropy Models for Fast Adaptation
Samarth Sinha , Anirudh Goyal, Hugo Larochelle, Animesh Garg
arXiv

2021

Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Nan Rosemary Ke , Aniket Didolkar , Sarthak Mittal , Anirudh Goyal, Stefan Bauer , Danilo Rezende
Christopher Pal , Michael C. Mozer
Neural Information Processing Systems (NeurIPS, Dataset Track) , 2021
arXiv / code

Neural Production Systems
Anirudh Goyal, Aniket Didolkar , Nan Rosemary Ke , Charles Blundell , Philippe Beaudoin , Nicolas Heess , Michael C. Mozer, Yoshua Bengio
Neural Information Processing Systems (NeurIPS) , 2021
arXiv / code

Discrete-Valued Neural Communication
Dianbo Liu , Kenji Kawaguchi, Anirudh Goyal, Chen Sun, Michael C. Mozer, Yoshua Bengio
Neural Information Processing Systems (NeurIPS) , 2021
arXiv

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
Ossama Ahmed , Frederik Träuble , Anirudh Goyal, Alexander Neitz , Yoshua Bengio , Bernhard Schölkopf, Manuel Wüthrich , Stefan Bauer
International Conference on Learning Representations (ICLR), 2021
arXiv / code

Fast and Slow Learning of Recurrent Independent Mechanisms
Kanika Madan , Nan Rosemary Ke , Anirudh Goyal, Bernhard Schölkopf, Yoshua Bengio
International Conference on Learning Representations (ICLR), 2021
arXiv

Recurrent Independent Mechanisms
Anirudh Goyal, Alex Lamb, Jordan Hoffmann , Shagun Sodhani , Sergey Levine, Bernhard Schölkopf, Yoshua Bengio
International Conference on Learning Representations (ICLR), 2021
arXiv / code

Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems
Anirudh Goyal, Alex Lamb, Phanideep Gampa , Philippe Beaudoin, Sergey Levine, Charles Blundell , Yoshua Bengio , Michael C. Mozer
International Conference on Learning Representations (ICLR), 2021
arXiv

S2RMs: Spatially Structured Recurrent Modules
Nasim Rahaman , Anirudh Goyal, Muhammad Waleed Gondal , Manuel Wüthrich, Stefan Bauer , Yoshua Bengio , Bernhard Schölkopf
International Conference on Learning Representations (ICLR), 2021
arXiv

Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers
Alex Lamb, Anirudh Goyal, Agnieszka Słowik , Michael C. Mozer
Philippe Beaudoin, Yoshua Bengio
International Conference on Artificial Intelligence and Statistics (AISTATS) , 2021
arXiv

Is Independence all you need? On the Generalization of Representations Learned from Correlated Data
Frederik Träuble , Elliot Creager , Niki Kilbertus , Anirudh Goyal, Francesco Locatello , Bernhard Schölkopf, Stefan Bauer
International Conference on Machine Learning (ICML), 2021
arXiv

DIBS: Diversity inducing Information Bottleneck in Model Ensembles
Samarth Sinha , Homanga Bharadhwaj , Anirudh Goyal, Hugo Larochelle, Animesh Garg , Florian Shkurti
AAAI Conference on Artificial Intelligence (AAAI) , 2021
arXiv

2020

Untangling tradeoffs between recurrence and self-attention in neural networks
Giancarlo Kerg , Bhargav Kanuparthi , Anirudh Goyal, Kyle Goyette, Yoshua Bengio , Guillaume Lajoie
Neural Information Processing Systems (NeurIPS) , 2020
arXiv

Top-K Training of GANs: Improving Generators by Making Critics Less Critical
Samarth Sinha , Anirudh Goyal, Colin Raffel , Augustus Odena
Neural Information Processing Systems (NeurIPS), 2020
arXiv

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules
Sarthak Mittal , Alex Lamb, Anirudh Goyal, Murray Shanahan , Guillaume Lajoie , Michael C. Mozer, Yoshua Bengio
International Conference on Machine Learning (ICML), 2020
arXiv

Small-GAN: Speeding Up GAN Training Using Core-sets
Samarth Sinha , Han Zhang , Anirudh Goyal, Yoshua Bengio , Hugo Larochelle, Augustus Odena
International Conference on Machine Learning (ICML), 2020
arXiv

Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives
Anirudh Goyal, Shagun Sodhani, Jonathan Binas , Xue Bin (Jason) Peng , Sergey Levine, Yoshua Bengio
International Conference on Learning Representations (ICLR), 2020
arXiv

The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget
Anirudh Goyal, Yoshua Bengio , Matthew Botvinick , Sergey Levine
International Conference on Learning Representations (ICLR), 2020
arXiv

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
Yoshua Bengio , Tristan Deleu , Nasim Rahaman , Nan Rosemary Ke , Sebastien Lachapelle , Olexa Bilaniuk , Anirudh Goyal, Chris Pal
International Conference on Learning Representations (ICLR), 2020
arXiv / code

Learning the Arrow of Time for Problems in Reinforcement Learning
Nasim Rahaman , Steffen Wolf , Anirudh Goyal, Roman Remme , Yoshua Bengio
International Conference on Learning Representations (ICLR), 2020
arXiv / code

2019

State-Reification Networks: Improving Generalization by Modeling the Distribution of Hidden Representations
Alex Lamb, Jonathan Binas , Anirudh Goyal, Sandeep Subramanian , Yoshua Bengio , Michael C. Mozer
International Conference on Machine Learning (ICML), 2019
arXiv / code

Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
Nan Rosemary Ke , Amanpreet Singh , Ahmed Touati , Anirudh Goyal, Yoshua Bengio , Devi Parikh , Dhruv Batra
International Conference on Learning Representations (ICLR), 2019
arXiv / code

InfoBot: Transfer and Exploration via the Information Bottleneck
Anirudh Goyal, Riashat Islam , Daniel Strouse , Zafarali Ahmed , Matthew Botvinick , Hugo Larochelle, Sergey Levine, Yoshua Bengio
International Conference on Learning Representations (ICLR), 2019
arXiv

Recall Traces: Backtracking Models for Efficient Reinforcement Learning
Anirudh Goyal, Philemon Brakel , William Fedus , Soumye Singhal , Timothy Lillicrap , Hugo Larochelle, Sergey Levine, Yoshua Bengio
International Conference on Learning Representations (ICLR), 2019
arXiv / code

2018

Sparse Attentive Backtracking: Temporal Credit Assignment Through Reminding
Nan Rosemary Ke , Anirudh Goyal, Olexa Bilaniuk , Jonathan Binas , Michael C. Mozer, Chris Pal, Yoshua Bengio
Advances in Neural Information Processing Systems (NIPS) , 2018
arXiv / code

Generalization of Equilibrium Propagation to Vector Field Dynamics
Benjamin Scellier , Anirudh Goyal, Jonathan Binas , Yoshua Bengio
International Conference on Learning Representations Workshop (ICLR), 2018
arXiv

2017

Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal, Alessandro Sordoni , Marc-Alexandre Côté , Nan Rosemary Ke , Yoshua Bengio
Advances in Neural Information Processing Systems (NIPS) , 2017
arXiv / code

Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net
Anirudh Goyal, Nan Rosemary Ke , Surya Ganguli , Yoshua Bengio
Advances in Neural Information Processing Systems (NIPS) , 2017
arXiv / code

Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
David Krueger , Tegan Maharaj , János Kramár , Mohammad Pezeshki , Nicolas Ballas , Nan Rosemary Ke , Anirudh Goyal, Yoshua Bengio , Aaron Courville , Chris Pal
International Conference on Learning Representations (ICLR), 2017
arXiv / code

An Actor-Critic Algorithm for Sequence Prediction
Dzmitry Bahdanau , Philemon Brakel , Kelvin Xu , Anirudh Goyal, Ryan Lowe , Joelle Pineau , Aaron Courville , Yoshua Bengio
International Conference on Learning Representations (ICLR), 2017
arXiv / code

2016

Professor Forcing: A New Algorithm for Training Recurrent Networks
Anirudh Goyal, Alex Lamb , Ying Zhang , Saizheng Zhang , Aaron Courville , Yoshua Bengio
Advances in Neural Information Processing Systems (NIPS) , 2016
arXiv / code