A holistic multimodal interaction model between histopathology and genomic profiles for patient survival prediction.

Abstract: Cancer survival prediction requires the integration of pathology whole-slice images (WSI) and genomic atlases, which is a challenging task due to the heterogeneity of the model itself and the complexity of modeling inter- and intra-modality interactions. Current methods usually adopt simple fusion strategies for multimodal feature integration, which cannot fully capture modality-specific and modality-common interactions, resulting in limited understanding of multimodal correlations and poor prediction performance. To compensate for these limitations, this paper proposes a multimodal representation decoupling network (MurreNet) to advance cancer survival analysis. Specifically, we first propose a multimodal representation decomposition (MRD) module to explicitly decompose pairs of input data into modality-specific and modality-common representations, thereby reducing redundancy between modalities. Furthermore, we further refine and update the decoupled representation through a novel training regularization strategy that imposes constraints on the distribution similarity, dissimilarity, and representativeness of modal features. Finally, the enhanced multi-modal features are integrated into the joint representation through the proposed deep ensemble orthogonal fusion (DHOF) strategy. Extensive experiments based on six TCGA cancer cohorts show that our MurreNet achieves state-of-the-art (SOTA) performance in survival prediction. This work was accepted by MICCAI2025, the top international conference in the field of medical image computing.

Mingxin Liu, Chengfei Cai, Jun Li, Pengbo Xu, Jinze Li, Jiquan Ma, and Jun Xu, MurreNet: Modeling Holistic Multimodal Interactions Between Histopathology and Genomic Profiles for Survival Prediction, MICCAI 2025: the 28th International Conference on Medical Image Computing and Computer Assisted Intervention , September 23-27, 2025. [ Link to paper ]

Translated from the original Chinese source.