Skip to main content Skip to secondary navigation
Page Content

Sheng Wang | Generative AI for Multimodal Biomedicine

Event Details

Wednesday, November 6, 2024
12:00 p.m. - 1:15 p.m. PST

Event Type

Location

Hybrid

Contact

Annie Benisch
HAI Seminar with Sheng Wang

Generative AI for Multimodal Biomedicine

Register to attend in-person 

  Register to attend via Zoom webinar

Abstract:

Biomedicine is inherently multimodal, including imaging modalities such as pathology, CT, MRI, X-ray and ultrasounds, as well as omics modality such as genomics, epigenomics and transcriptomics. General domain multimodal approaches are not applicable to biomedicine because biomedical images are very different from general domain images, thus necessitating the development of modality-specific approaches. In this talk, Sheng will introduce three recent works towards building multimodal biomedicine foundation models. 

First,  Sheng will introduce GigaPath, the first whole-slide pathology foundation model that can handle gigapixel-level pathology images. GigaPath exploits a novel vision transformer architecture and achieves the state-of-the-art results on 23 out of 26 cancer tasks, including subtyping and biomarker prediction. Next, he will introduce OCTCube, the first 3D OCT retinal imaging foundation model. OCTCube significantly outperformed 2D models on 27 out of 29 tasks, including retinal disease prediction, cross-modality analysis, cross-device generalization and systemic disease prediction. Finally, Sheng will introduce BiomedParse, a multi-modal foundation model that integrates 9 major biomedical imaging modalities by projecting all of them into the text space, resulting in superior performance on segmentation, detection, and recognition, paving the path for large-scale image-based biomedical discovery. I will conclude this task with discussion on how multi-modal generative AI can advance future medical applications through multi-agent framework and integration with multi-omics datasets.

Sheng Wang Headshot

Sheng Wang

Assistant Professor in the School of Computer Science and Engineering at the University of Washington Seattle

Connect

The official Twitter account of the Stanford Institute for Human-Centered AI, advancing AI research, education, policy, and practice to improve the human condition.

Join the conversation

If you need a disability-related accommodation, please contact: Annie Benisch, Events Planner. Requests should be made at least a week before the event.