Drew Hudson: Compositionality in Visual Reasoning and Generation

HAI Weekly Seminar

Compositionality in Visual Reasoning and Generation

The world we live in is inherently compositional: just like a sentence is built upon phrases and words, a visual scene comprises a collection of interacting objects and entities, which in turn are derived from the sum of their parts. This compositionality plays a critical role in our ability to understand the world, organize the acquired knowledge through a rich set of concepts, and easily adapt them to novel situations and environments. Essentially, it is considered one of the fundamental building blocks of human intelligence. How to incorporate such compositionality into AI models? How can we encourage neural networks to develop semantic understanding of our surroundings? And how can we leverage the emerging structured knowledge to improve in downstream tasks such as question answering or image generation? These are the questions that will be explored in the talk, in which I will present models for multi-step synthesis of and reasoning over multi-object scenes, describe their key design principles and underlying mechanisms, and illustrate the benefits they offer in terms of enhanced controllability, increased data-efficiency, and improved interpretability of their internal representations and reasoning process.

Speakers

PhD Student in Computer Science, Stanford University