Trustworthy Multimodal Medical Foundation Model

The proposed project aims to develop and evaluate trustworthy multimodal medical foundation models, with a particular focus on Vision-Language Models (VLMs) in healthcare. The project emphasizes improving the trustworthiness of VLMs by enhancing their explainability and reliability. Explainability allows users to understand the reasoning behind the model's decisions, while reliability ensures that the model generates outputs grounded in accurate, factual knowledge. To achieve this, retrieval-augmented systems will be explored to reduce hallucinations and improve model transparency. The research aims to incorporate traditional medical tasks such as image segmentation, classification, and detection into the VLM framework. The research will explore the spatial reasoning and measurement ability of VLMs to enhance medical image computing capabilities on 2D and 3D medical images.