Yuheng Li
BME PhD Proposal Presentation

Date: 2025-12-11
Time: 09:00 AM EST
Location / Meeting Link: https://emory.zoom.us/j/97941751006

Committee Members:
Yun Wang; Xiaofeng Yang; Judy Wawira Gichoya; John Oshinski; Xiao Hu


Title: Developing multi-modal foundation models via self-supervised learning for advancing radiation therapy

Abstract:
Radiation therapy depends on accurate delineation of tumors and organs-at-risk (OARs) on computed tomography (CT), yet current manual contouring is laborious and subject to variability among clinicians. Deep learning–based auto-segmentation holds potential to accelerate planning, but most methods require extensive labeled data and generalize poorly across institutions. Self-supervised learning (SSL) offers a scalable alternative, leveraging large amounts of unlabeled CT imaging; however, existing vision-only SSL neglects the rich semantic knowledge contained in radiology reports and treatment planning notes. Conversely, current vision–language models (VLMs) incorporate textual supervision but lack spatial precision required for radiotherapy. Addressing this gap could transform automated planning and enable new text-guided clinical workflows. In this work, we propose to establish a unified CT foundation model that integrates visual self-supervision with clinical text alignment, enabling generalizable downstream performance across segmentation, classification, and report generation tasks. Aim 1 focuses on developing a 3D masked image modeling framework to learn local anatomical structure from unlabeled CT data, improving segmentation accuracy and robustness. Aim 2 extends this representation to a vision–language paradigm using large-scale CT–report corpora and organ-level captions, enabling text-driven segmentation that is robust to variations in clinical vocabulary. Aim 3 unifies these components into a two-stage self-supervised framework that leverages both image-only and image–text supervision to produce a scalable, transferable foundation model for radiation oncology, evaluated across segmentation, classification, registration, and structured reporting.