Drew Hudson: Compositionality in Visual Reasoning and Generation
In-person registration is open to the Stanford community. If you are not a Stanford affiliate, please register to attend virtually through the link above. If you would like to request an in-person seat, please contact the HAI Events Team at firstname.lastname@example.org.
Compositionality in Visual Reasoning and Generation
The world we live in is inherently compositional: just like a sentence is built upon phrases and words, a visual scene comprises a collection of interacting objects and entities, which in turn are derived from the sum of their parts. This compositionality plays a critical role in our ability to understand the world, organize the acquired knowledge through a rich set of concepts, and easily adapt them to novel situations and environments. Essentially, it is considered one of the fundamental building blocks of human intelligence. How to incorporate such compositionality into AI models? How can we encourage neural networks to develop semantic understanding of our surroundings? And how can we leverage the emerging structured knowledge to improve in downstream tasks such as question answering or image generation? These are the questions that will be explored in the talk, in which I will present models for multi-step synthesis of and reasoning over multi-object scenes, describe their key design principles and underlying mechanisms, and illustrate the benefits they offer in terms of enhanced controllability, increased data-efficiency, and improved interpretability of their internal representations and reasoning process.