Sheng Wang | Generative AI for Multimodal Biomedicine
Event Details
Event Type
Location
Hybrid
Generative AI for Multimodal Biomedicine
Register to attend via Zoom webinar
Abstract:
Biomedicine is inherently multimodal, including imaging modalities such as pathology, CT, MRI, X-ray and ultrasounds, as well as omics modality such as genomics, epigenomics and transcriptomics. General domain multimodal approaches are not applicable to biomedicine because biomedical images are very different from general domain images, thus necessitating the development of modality-specific approaches. In this talk, Sheng will introduce three recent works towards building multimodal biomedicine foundation models.
First, Sheng will introduce GigaPath, the first whole-slide pathology foundation model that can handle gigapixel-level pathology images. GigaPath exploits a novel vision transformer architecture and achieves the state-of-the-art results on 23 out of 26 cancer tasks, including subtyping and biomarker prediction. Next, he will introduce OCTCube, the first 3D OCT retinal imaging foundation model. OCTCube significantly outperformed 2D models on 27 out of 29 tasks, including retinal disease prediction, cross-modality analysis, cross-device generalization and systemic disease prediction. Finally, Sheng will introduce BiomedParse, a multi-modal foundation model that integrates 9 major biomedical imaging modalities by projecting all of them into the text space, resulting in superior performance on segmentation, detection, and recognition, paving the path for large-scale image-based biomedical discovery. I will conclude this task with discussion on how multi-modal generative AI can advance future medical applications through multi-agent framework and integration with multi-omics datasets.
Sheng Wang
Assistant Professor in the School of Computer Science and Engineering at the University of Washington Seattle
Connect
The official Twitter account of the Stanford Institute for Human-Centered AI, advancing AI research, education, policy, and practice to improve the human condition.
If you need a disability-related accommodation, please contact: Annie Benisch, Events Planner. Requests should be made at least a week before the event.