HAI Weekly Seminar with Leonidas Guibas
Joint Learning Over Visual and Geometric Data
Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.
Sign Up For Latest News
Joint Learning Over Visual and Geometric Data
We examine the prevalence and productivity dynamics of artificial intelligence (AI) in American manufacturing. Working with the Census Bureau to collect detailed large-scale data for 2017 and 2021, we focus on AI-related technologies with industrial applications.
.png&w=1920&q=100)
We examine the prevalence and productivity dynamics of artificial intelligence (AI) in American manufacturing. Working with the Census Bureau to collect detailed large-scale data for 2017 and 2021, we focus on AI-related technologies with industrial applications.
While Large Language Models (LLMs) show promise in many domains, relying on them for direct policy generation in games often results in illegal moves and poor strategic play.

While Large Language Models (LLMs) show promise in many domains, relying on them for direct policy generation in games often results in illegal moves and poor strategic play.
The possibility that AI will automate most cognitive labor is worth taking seriously. How should we adapt to this transformation? I start from the perspective, articulated in the essay “AI as normal technology”, that the true bottlenecks lie downstream of capabilities and that AI’s impacts will unfold gradually over decades. If this is true, there are major gaps in our current evidence infrastructure, because it over-emphasizes the capability layer.
.png&w=1920&q=100)
The possibility that AI will automate most cognitive labor is worth taking seriously. How should we adapt to this transformation? I start from the perspective, articulated in the essay “AI as normal technology”, that the true bottlenecks lie downstream of capabilities and that AI’s impacts will unfold gradually over decades. If this is true, there are major gaps in our current evidence infrastructure, because it over-emphasizes the capability layer.
Many challenges remain in applying machine learning to domains where obtaining massive annotated data is difficult. We discuss a number of approaches that aim to reduce supervision load for learning algorithms in the visual and geometric domains by leveraging correlations among data, among representations, and among learning tasks -- what we call joint learning. The basic notion is that inference problems do not occur in isolation but rather in a social context that can be exploited to provide self-supervision by enforcing consistency among them, thus improving performance and increasing sample efficiency. An example is shape co-segmentation, where we can use structural correlations between related shapes to regularize the segmentation of any particular shape. Another is the use of cross-task consistency constraints, as in the case of inferring depth and normals from an image, which are obviously related. Even at the level of representations, joint learning can avoid blind-spots of any one individual representation and better adapt to data particularities – just as we get with multiple 2D views of a 3D object. The talk will present a number of examples of joint learning, including the above as well as 3D object detection and pose estimation.
