Multi-modal brain tumor segmentation via conditional synthesis with Fourier domain adaptation

Yasmina Al Khalil (Corresponding author), Aymen Ayaz, Cristian Lorenz, Jürgen Weese, Josien Pluim, Marcel Breeuwer

Research output: Contribution to journalArticleAcademicpeer-review

44 Downloads (Pure)

Abstract

Accurate brain tumor segmentation is critical for diagnosis and treatment planning, whereby multi-modal magnetic resonance imaging (MRI) is typically used for analysis. However, obtaining all required sequences and expertly labeled data for training is challenging and can result in decreased quality of segmentation models developed through automated algorithms. In this work, we examine the possibility of employing a conditional generative adversarial network (GAN) approach for synthesizing multi-modal images to train deep learning-based neural networks aimed at high-grade glioma (HGG) segmentation. The proposed GAN is conditioned on auxiliary brain tissue and tumor segmentation masks, allowing us to attain better accuracy and control of tissue appearance during synthesis. To reduce the domain shift between synthetic and real MR images, we additionally adapt the low-frequency Fourier space components of synthetic data, reflecting the style of the image, to those of real data. We demonstrate the impact of Fourier domain adaptation (FDA) on the training of 3D segmentation networks and attain significant improvements in both the segmentation performance and prediction confidence. Similar outcomes are seen when such data is used as a training augmentation alongside the available real images. In fact, experiments on the BraTS2020 dataset reveal that models trained solely with synthetic data exhibit an improvement of up to 4% in Dice score when using FDA, while training with both real and FDA-processed synthetic data through augmentation results in an improvement of up to 5% in Dice compared to using real data alone. This study highlights the importance of considering image frequency in generative approaches for medical image synthesis and offers a promising approach to address data scarcity in medical imaging segmentation.

Original languageEnglish
Article number102332
Number of pages12
JournalComputerized Medical Imaging and Graphics
Volume112
DOIs
Publication statusPublished - Mar 2024

Funding

This research is a part of the openGTN project, supported by the European Union in the Marie Curie Innovative Training Networks (ITN), The Netherlands fellowship program under project No. 764465.

FundersFunder number
European Commission764465

    Keywords

    • Brain tumor segmentation
    • Frequency domain adaptation
    • Image synthesis

    Fingerprint

    Dive into the research topics of 'Multi-modal brain tumor segmentation via conditional synthesis with Fourier domain adaptation'. Together they form a unique fingerprint.

    Cite this