CVPR 2023 - Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoning

In this episode we discuss Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoning by Shi Chen, Qi Zhao. The paper proposes a new framework for visual reasoning inspired by human reasoning, which addresses the limitations of current methods. Existing methods rely on statistical priors and struggle with novel objects or biased question-answer distributions, whereas humans are able to decompose difficult problems and correlate different concepts based on their semantic relationships. The proposed framework uses a principled object factorization method and a neural module network to decompose objects and derive prototypes, which are used to measure similarity on a common semantic space and make decisions with compositional reasoning. The framework is capable of answering questions with diverse objects and provides an interpretable interface for understanding the decision-making process of models.