MULTIMODAL FUSION OF MRI AND GENOMIC DATA USING TRANSFORMER NETWORKS FOR ALZHEIMER’S PREDICTION

Abstract

Alzheimer's disease (AD) is a growing neurodegenerative illness that is significantly influenced by both physical alterations in the brain and a genetic susceptibility. Traditional unimodal approaches that rely solely on MRI or genomic data often overlook the complex relationships between SNP-level alterations and neuroanatomical atrophy. In this study, we suggest a multimodal transformer-based system that integrates structural MRI and SNP genomic data via bidirectional cross-attention fusion. A Vision Transformer encoder handles the MRI modality, while a transformer-based SNP encoder simulates genetic variations.

The model can learn significant inter-modal connections thanks to cross-attention, which permits fine-grained alignment between genetic biomarkers and brain areas. The suggested framework achieves an overall accuracy of 92% for Alzheimer's disease prediction, outperforming both unimodal and conventional fusion techniques, according to experiments done on the ADNI dataset. Additionally, the model has excellent production in AD vs. CN, MCI vs. AD, and MCI-to-AD conversion tasks, underscoring the importance of combining genomic and imaging modalities. These findings suggest that transformer-based cross-attention fusion offers an effective and comprehensible basis for early AD detection and customized risk evaluation.