How Modular Should Neural Module Networks Be for Systematic Generalization?

Vanessa D’Amario; Tomotake Sasaki; Xavier Boix

How Modular Should Neural Module Networks Be for Systematic Generalization?

Vanessa D’Amario
, Tomotake Sasaki
, Xavier Boix

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Neural Module Networks (NMNs) aim at Visual Question Answering (VQA) via composition of modules that tackle a sub-task. NMNs are a promising strategy to achieve systematic generalization, ie. overcoming biasing factors in the training distribution. However, the aspects of NMNs that facilitate systematic generalization are not fully understood. In this paper, we demonstrate that the degree of modularity of the NMN have large influence on systematic generalization. In a series of experiments on three VQA datasets (VQA-MNIST, SQOOP, and CLEVR-CoGenT), our results reveal that tuning the degree of modularity, especially at the image encoder stage, reaches substantially higher systematic generalization. These findings lead to new NMN architectures that outperform previous ones in terms of systematic generalization.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021
Editors	Marc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan
Publisher	Neural information processing systems foundation
Pages	23374-23385
Number of pages	12
ISBN (Electronic)	9781713845393
State	Published - 2021
Externally published	Yes
Event	35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online Duration: Dec 6 2021 → Dec 14 2021

Publication series

Name	Advances in Neural Information Processing Systems
Volume	28
ISSN (Print)	1049-5258

Conference

Conference	35th Conference on Neural Information Processing Systems, NeurIPS 2021
City	Virtual, Online
Period	12/6/21 → 12/14/21

Bibliographical note

Publisher Copyright:
© 2021 Neural information processing systems foundation. All rights reserved.

ASJC Scopus Subject Areas

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

D’Amario, V., Sasaki, T., & Boix, X. (2021). How Modular Should Neural Module Networks Be for Systematic Generalization? In MA. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. Wortman Vaughan (Eds.), Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021 (pp. 23374-23385). (Advances in Neural Information Processing Systems; Vol. 28). Neural information processing systems foundation.

How Modular Should Neural Module Networks Be for Systematic Generalization? / D’Amario, Vanessa; Sasaki, Tomotake; Boix, Xavier.
Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. ed. / Marc'Aurelio Ranzato; Alina Beygelzimer; Yann Dauphin; Percy S. Liang; Jenn Wortman Vaughan. Neural information processing systems foundation, 2021. p. 23374-23385 (Advances in Neural Information Processing Systems; Vol. 28).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

D’Amario, V, Sasaki, T & Boix, X 2021, How Modular Should Neural Module Networks Be for Systematic Generalization? in MA Ranzato, A Beygelzimer, Y Dauphin, PS Liang & J Wortman Vaughan (eds), Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. Advances in Neural Information Processing Systems, vol. 28, Neural information processing systems foundation, pp. 23374-23385, 35th Conference on Neural Information Processing Systems, NeurIPS 2021, Virtual, Online, 12/6/21.

D’Amario V, Sasaki T, Boix X. How Modular Should Neural Module Networks Be for Systematic Generalization? In Ranzato MA, Beygelzimer A, Dauphin Y, Liang PS, Wortman Vaughan J, editors, Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. Neural information processing systems foundation. 2021. p. 23374-23385. (Advances in Neural Information Processing Systems).

D’Amario, Vanessa ; Sasaki, Tomotake ; Boix, Xavier. / How Modular Should Neural Module Networks Be for Systematic Generalization?. Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. editor / Marc'Aurelio Ranzato ; Alina Beygelzimer ; Yann Dauphin ; Percy S. Liang ; Jenn Wortman Vaughan. Neural information processing systems foundation, 2021. pp. 23374-23385 (Advances in Neural Information Processing Systems).

@inproceedings{cd922334c0a84778a90ba9073818cce8,

title = "How Modular Should Neural Module Networks Be for Systematic Generalization?",

abstract = "Neural Module Networks (NMNs) aim at Visual Question Answering (VQA) via composition of modules that tackle a sub-task. NMNs are a promising strategy to achieve systematic generalization, ie. overcoming biasing factors in the training distribution. However, the aspects of NMNs that facilitate systematic generalization are not fully understood. In this paper, we demonstrate that the degree of modularity of the NMN have large influence on systematic generalization. In a series of experiments on three VQA datasets (VQA-MNIST, SQOOP, and CLEVR-CoGenT), our results reveal that tuning the degree of modularity, especially at the image encoder stage, reaches substantially higher systematic generalization. These findings lead to new NMN architectures that outperform previous ones in terms of systematic generalization.",

author = "Vanessa D{\textquoteright}Amario and Tomotake Sasaki and Xavier Boix",

note = "Publisher Copyright: {\textcopyright} 2021 Neural information processing systems foundation. All rights reserved.; 35th Conference on Neural Information Processing Systems, NeurIPS 2021 ; Conference date: 06-12-2021 Through 14-12-2021",

year = "2021",

language = "English",

series = "Advances in Neural Information Processing Systems",

publisher = "Neural information processing systems foundation",

pages = "23374--23385",

editor = "Marc'Aurelio Ranzato and Alina Beygelzimer and Yann Dauphin and Liang, \{Percy S.\} and \{Wortman Vaughan\}, Jenn",

booktitle = "Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021",