Pathology Synthesis of 3D Consistent Cardiac MR Images Using 2D VAEs and GANs

Sina Amirrajab, Cristian Lorenz, Juergen Weese, Josien Pluim, Marcel Breeuwer

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

1 Citation (Scopus)
38 Downloads (Pure)

Abstract

We propose a method for synthesizing cardiac MR images with plausible heart shapes and realistic appearances for the purpose of generating labeled data for deep-learning (DL) training. It breaks down the image synthesis into label deformation and label-to-image translation tasks. The former is achieved via latent space interpolation in a VAE model, while the latter is accomplished via a conditional GAN model. We devise an approach for label manipulation in the latent space of the trained VAE model, namely pathology synthesis, aiming to synthesize a series of pseudo-pathological synthetic subjects with characteristics of a desired heart disease. Furthermore, we propose to model the relationship between 2D slices in the latent space of the VAE via estimating the correlation coefficient matrix between the latent vectors and utilizing it to correlate elements of randomly drawn samples before decoding to image space. This simple yet effective approach results in generating 3D consistent subjects from 2D slice-by-slice generations. Such an approach could provide a solution to diversify and enrich the available database of cardiac MR images and to pave the way for the development of generalizable DL-based image analysis algorithms. The code will be available at https://github.com/sinaamirrajab/CardiacPathologySynthesis.
Original languageEnglish
Title of host publicationSimulation and Synthesis in Medical Imaging - 7th International Workshop, SASHIMI 2022, Held in Conjunction with MICCAI 2022, Proceedings
Subtitle of host publication7th International Workshop, SASHIMI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 18, 2022, Proceedings
EditorsCan Zhao, David Svoboda, Jelmer M. Wolterink, Maria Escobar
PublisherSpringer
Chapter4
Pages34-42
Number of pages9
ISBN (Electronic)978-3-031-16980-9
ISBN (Print)978-3-031-16979-3
DOIs
Publication statusPublished - Sept 2022
Event25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022 - Singapore, Singapore
Duration: 18 Sept 202222 Sept 2022
Conference number: 25

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13570 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022
Abbreviated titleMICCAI 2022
Country/TerritorySingapore
CitySingapore
Period18/09/2222/09/22

Keywords

  • Cardiac MRI
  • GANs
  • Image synthesis
  • Pathology synthesis
  • VAEs

Fingerprint

Dive into the research topics of 'Pathology Synthesis of 3D Consistent Cardiac MR Images Using 2D VAEs and GANs'. Together they form a unique fingerprint.

Cite this