4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model

Samaneh Abbasi-Sureshjani; Sina Amirrajab; Cristian Lorenz; Juergen Weese; Josien Pluim; Marcel Breeuwer

4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model

Samaneh Abbasi-Sureshjani, Sina Amirrajab, Cristian Lorenz, Juergen Weese, Josien Pluim, Marcel Breeuwer

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic

13 Citations (Scopus)

80 Downloads (Pure)

Abstract

We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De-normalization (SPADE) technique for conditional image synthesis to preserve the semantic spatial information of ground truth anatomy. Using the parameterized motion model of the XCAT heart, we generate labels for 25 time frames of the heart for one cardiac cycle at 18 locations for the short axis view. Subsequently, realistic images are generated from these labels, with modality-specific features that are learned from real CMR image data. We demonstrate that style transfer from another cardiac image can be accomplished by using a style encoder network. Due to the flexibility of XCAT in creating new heart models, this approach can result in a realistic virtual population to address different challenges the medical image analysis research community is facing such as expensive data collection. Our proposed method has a great potential to synthesize 4D controllable CMR images with annotations and adaptable styles to be used in various supervised multi-site, multi-vendor applications in medical image analysis.

Original language	English
Title of host publication	Proceedings of the Third Conference on Medical Imaging with Deep Learning
Editors	Tal Arbel, Ismail Ben Ayed, Marleen de Bruijne, Maxime Descoteaux, Herve Lombaert, Christopher Pal
Publisher	PMLR
Pages	6-18
Number of pages	13
Publication status	Published - 17 Feb 2020
Event	Third Conference on Medical Imaging with Deep Learning - Montreal, Canada Duration: 6 Jul 2020 → 8 Jul 2020

Publication series

Name	Proceedings of Machine Learning Research
Publisher	PMLR
Number	121
ISSN (Print)	2640-3498

Conference

Conference	Third Conference on Medical Imaging with Deep Learning
Country/Territory	Canada
City	Montreal
Period	6/07/20 → 8/07/20

Keywords

eess.IV
cs.CV
cs.LG
stat.ML
generative adversarial network
SPADE GAN
cardiac magnetic resonance imaging
4D semantic image synthesis
XCAT phantom

Access to Document

2002.07089v3Final author version , 1.72 MB
abbasi-sureshjani20aFinal published version, 1.66 MB

Cite this

Abbasi-Sureshjani, S., Amirrajab, S., Lorenz, C., Weese, J., Pluim, J., & Breeuwer, M. (2020). 4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model. In T. Arbel, I. Ben Ayed, M. de Bruijne, M. Descoteaux, H. Lombaert, & C. Pal (Eds.), Proceedings of the Third Conference on Medical Imaging with Deep Learning (pp. 6-18). (Proceedings of Machine Learning Research; No. 121). PMLR. https://arxiv.org/abs/2002.07089

Abbasi-Sureshjani, Samaneh ; Amirrajab, Sina ; Lorenz, Cristian et al. / 4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model. Proceedings of the Third Conference on Medical Imaging with Deep Learning. editor / Tal Arbel ; Ismail Ben Ayed ; Marleen de Bruijne ; Maxime Descoteaux ; Herve Lombaert ; Christopher Pal. PMLR, 2020. pp. 6-18 (Proceedings of Machine Learning Research; 121).

@inproceedings{b6db7e3417794eb499cc028f5269e5dd,

title = "4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model",

abstract = "We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De-normalization (SPADE) technique for conditional image synthesis to preserve the semantic spatial information of ground truth anatomy. Using the parameterized motion model of the XCAT heart, we generate labels for 25 time frames of the heart for one cardiac cycle at 18 locations for the short axis view. Subsequently, realistic images are generated from these labels, with modality-specific features that are learned from real CMR image data. We demonstrate that style transfer from another cardiac image can be accomplished by using a style encoder network. Due to the flexibility of XCAT in creating new heart models, this approach can result in a realistic virtual population to address different challenges the medical image analysis research community is facing such as expensive data collection. Our proposed method has a great potential to synthesize 4D controllable CMR images with annotations and adaptable styles to be used in various supervised multi-site, multi-vendor applications in medical image analysis. ",

keywords = "eess.IV, cs.CV, cs.LG, stat.ML, generative adversarial network, SPADE GAN, cardiac magnetic resonance imaging, 4D semantic image synthesis, XCAT phantom",

author = "Samaneh Abbasi-Sureshjani and Sina Amirrajab and Cristian Lorenz and Juergen Weese and Josien Pluim and Marcel Breeuwer",

year = "2020",

month = feb,

day = "17",

language = "English",

series = "Proceedings of Machine Learning Research",

publisher = "PMLR",

number = "121",

pages = "6--18",

editor = "Tal Arbel and {Ben Ayed}, Ismail and {de Bruijne}, Marleen and Maxime Descoteaux and Herve Lombaert and Christopher Pal",

booktitle = "Proceedings of the Third Conference on Medical Imaging with Deep Learning",

note = "Third Conference on Medical Imaging with Deep Learning ; Conference date: 06-07-2020 Through 08-07-2020",

}

Abbasi-Sureshjani, S, Amirrajab, S, Lorenz, C, Weese, J, Pluim, J & Breeuwer, M 2020, 4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model. in T Arbel, I Ben Ayed, M de Bruijne, M Descoteaux, H Lombaert & C Pal (eds), Proceedings of the Third Conference on Medical Imaging with Deep Learning. Proceedings of Machine Learning Research, no. 121, PMLR, pp. 6-18, Third Conference on Medical Imaging with Deep Learning, Montreal, Canada, 6/07/20. < https://arxiv.org/abs/2002.07089>

4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model. / Abbasi-Sureshjani, Samaneh; Amirrajab, Sina; Lorenz, Cristian et al.
Proceedings of the Third Conference on Medical Imaging with Deep Learning. ed. / Tal Arbel; Ismail Ben Ayed; Marleen de Bruijne; Maxime Descoteaux; Herve Lombaert; Christopher Pal. PMLR, 2020. p. 6-18 (Proceedings of Machine Learning Research; No. 121).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic

TY - GEN

T1 - 4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model

AU - Abbasi-Sureshjani, Samaneh

AU - Amirrajab, Sina

AU - Lorenz, Cristian

AU - Weese, Juergen

AU - Pluim, Josien

AU - Breeuwer, Marcel

PY - 2020/2/17

Y1 - 2020/2/17

N2 - We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De-normalization (SPADE) technique for conditional image synthesis to preserve the semantic spatial information of ground truth anatomy. Using the parameterized motion model of the XCAT heart, we generate labels for 25 time frames of the heart for one cardiac cycle at 18 locations for the short axis view. Subsequently, realistic images are generated from these labels, with modality-specific features that are learned from real CMR image data. We demonstrate that style transfer from another cardiac image can be accomplished by using a style encoder network. Due to the flexibility of XCAT in creating new heart models, this approach can result in a realistic virtual population to address different challenges the medical image analysis research community is facing such as expensive data collection. Our proposed method has a great potential to synthesize 4D controllable CMR images with annotations and adaptable styles to be used in various supervised multi-site, multi-vendor applications in medical image analysis.

AB - We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De-normalization (SPADE) technique for conditional image synthesis to preserve the semantic spatial information of ground truth anatomy. Using the parameterized motion model of the XCAT heart, we generate labels for 25 time frames of the heart for one cardiac cycle at 18 locations for the short axis view. Subsequently, realistic images are generated from these labels, with modality-specific features that are learned from real CMR image data. We demonstrate that style transfer from another cardiac image can be accomplished by using a style encoder network. Due to the flexibility of XCAT in creating new heart models, this approach can result in a realistic virtual population to address different challenges the medical image analysis research community is facing such as expensive data collection. Our proposed method has a great potential to synthesize 4D controllable CMR images with annotations and adaptable styles to be used in various supervised multi-site, multi-vendor applications in medical image analysis.

KW - eess.IV

KW - cs.CV

KW - cs.LG

KW - stat.ML

KW - generative adversarial network

KW - SPADE GAN

KW - cardiac magnetic resonance imaging

KW - 4D semantic image synthesis

KW - XCAT phantom

UR - http://www.scopus.com/inward/record.url?scp=85103424571&partnerID=8YFLogxK

M3 - Conference contribution

T3 - Proceedings of Machine Learning Research

SP - 6

EP - 18

BT - Proceedings of the Third Conference on Medical Imaging with Deep Learning

A2 - Arbel, Tal

A2 - Ben Ayed, Ismail

A2 - de Bruijne, Marleen

A2 - Descoteaux, Maxime

A2 - Lombaert, Herve

A2 - Pal, Christopher

PB - PMLR

T2 - Third Conference on Medical Imaging with Deep Learning

Y2 - 6 July 2020 through 8 July 2020

ER -

Abbasi-Sureshjani S, Amirrajab S, Lorenz C, Weese J, Pluim J , Breeuwer M. 4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model. In Arbel T, Ben Ayed I, de Bruijne M, Descoteaux M, Lombaert H, Pal C, editors, Proceedings of the Third Conference on Medical Imaging with Deep Learning. PMLR. 2020. p. 6-18. (Proceedings of Machine Learning Research; 121).

4D semantic cardiac magnetic resonance image synthesis on XCAT anatomical model

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this