Automatic cardiac landmark localization by a recurrent neural network

Mike van Zon, Mitko Veta, Shuo Li

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Localization of cardiac anatomical landmarks is an important step towards a more robust and accurate analysis of the heart. A fully automatic hybrid framework is proposed that detects key landmark locations in cardiac magnetic resonance (MR) images. Our method is trained and evaluated for the detection of mitral valve points on long-axis MRI and RV insert points in short-axis MRI. The framework incorporates four key modules for the localization of the landmark points. The first module crops the MR image around the heart using a convolutional neural network (CNN). The second module employs a U-Net to obtain an efficient feature representation of the cardiac image, as well as detect a preliminary location of the landmark points. In the third module, the feature representation of a cardiac image is processed with a Recurrent Neural Network (RNN). The RNN leverages either spatial or temporal dynamics from neighboring slides in time or space and obtains a second prediction for the landmark locations. In the last module the two predictions from the U-Net and RNN are combined and final locations for the landmarks are extracted. The framework is separately trained and evaluated for the localization of each landmark, it achieves a final average error of 2.87 mm for the mitral valve points and an average error of 3.64 mm for the right ventricular insert points. Our method shows that the use of a recurrent neural network for the modeling of additional temporal or spatial dependencies improves localization accuracy and achieves promising results.

LanguageEnglish
Title of host publicationMedical Imaging 2019
Subtitle of host publicationImage Processing
EditorsBennett A. Landman, Elsa D. Angelini
Place of PublicationBellingham
PublisherSPIE
Number of pages13
ISBN (Electronic)9781510625457
DOIs
StatePublished - 1 Jan 2019
EventMedical Imaging 2019: Image Processing - San Diego, United States
Duration: 19 Feb 201921 Feb 2019

Publication series

NameProceedings of SPIE
Volume10949

Conference

ConferenceMedical Imaging 2019: Image Processing
CountryUnited States
CitySan Diego
Period19/02/1921/02/19

Fingerprint

landmarks
Recurrent neural networks
Mitral Valve
Magnetic Resonance Spectroscopy
Magnetic resonance
modules
Magnetic resonance imaging
Spatial Analysis
inserts
Crops
magnetic resonance
spatial dependencies
Neural networks
crops
predictions
chutes

Keywords

  • Cardiac Imaging
  • Convolutional Neural Network
  • Deep Learning
  • Recurrent Neural Network

Cite this

van Zon, M., Veta, M., & Li, S. (2019). Automatic cardiac landmark localization by a recurrent neural network. In B. A. Landman, & E. D. Angelini (Eds.), Medical Imaging 2019: Image Processing [1094916] (Proceedings of SPIE; Vol. 10949). Bellingham: SPIE. DOI: 10.1117/12.2512048
van Zon, Mike ; Veta, Mitko ; Li, Shuo. / Automatic cardiac landmark localization by a recurrent neural network. Medical Imaging 2019: Image Processing. editor / Bennett A. Landman ; Elsa D. Angelini. Bellingham : SPIE, 2019. (Proceedings of SPIE).
@inproceedings{90f8b3fd8e4f4f8ea699056379b0ffa5,
title = "Automatic cardiac landmark localization by a recurrent neural network",
abstract = "Localization of cardiac anatomical landmarks is an important step towards a more robust and accurate analysis of the heart. A fully automatic hybrid framework is proposed that detects key landmark locations in cardiac magnetic resonance (MR) images. Our method is trained and evaluated for the detection of mitral valve points on long-axis MRI and RV insert points in short-axis MRI. The framework incorporates four key modules for the localization of the landmark points. The first module crops the MR image around the heart using a convolutional neural network (CNN). The second module employs a U-Net to obtain an efficient feature representation of the cardiac image, as well as detect a preliminary location of the landmark points. In the third module, the feature representation of a cardiac image is processed with a Recurrent Neural Network (RNN). The RNN leverages either spatial or temporal dynamics from neighboring slides in time or space and obtains a second prediction for the landmark locations. In the last module the two predictions from the U-Net and RNN are combined and final locations for the landmarks are extracted. The framework is separately trained and evaluated for the localization of each landmark, it achieves a final average error of 2.87 mm for the mitral valve points and an average error of 3.64 mm for the right ventricular insert points. Our method shows that the use of a recurrent neural network for the modeling of additional temporal or spatial dependencies improves localization accuracy and achieves promising results.",
keywords = "Cardiac Imaging, Convolutional Neural Network, Deep Learning, Recurrent Neural Network",
author = "{van Zon}, Mike and Mitko Veta and Shuo Li",
year = "2019",
month = "1",
day = "1",
doi = "10.1117/12.2512048",
language = "English",
series = "Proceedings of SPIE",
publisher = "SPIE",
editor = "Landman, {Bennett A.} and Angelini, {Elsa D.}",
booktitle = "Medical Imaging 2019",
address = "United States",

}

van Zon, M, Veta, M & Li, S 2019, Automatic cardiac landmark localization by a recurrent neural network. in BA Landman & ED Angelini (eds), Medical Imaging 2019: Image Processing., 1094916, Proceedings of SPIE, vol. 10949, SPIE, Bellingham, Medical Imaging 2019: Image Processing, San Diego, United States, 19/02/19. DOI: 10.1117/12.2512048

Automatic cardiac landmark localization by a recurrent neural network. / van Zon, Mike; Veta, Mitko; Li, Shuo.

Medical Imaging 2019: Image Processing. ed. / Bennett A. Landman; Elsa D. Angelini. Bellingham : SPIE, 2019. 1094916 (Proceedings of SPIE; Vol. 10949).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Automatic cardiac landmark localization by a recurrent neural network

AU - van Zon,Mike

AU - Veta,Mitko

AU - Li,Shuo

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Localization of cardiac anatomical landmarks is an important step towards a more robust and accurate analysis of the heart. A fully automatic hybrid framework is proposed that detects key landmark locations in cardiac magnetic resonance (MR) images. Our method is trained and evaluated for the detection of mitral valve points on long-axis MRI and RV insert points in short-axis MRI. The framework incorporates four key modules for the localization of the landmark points. The first module crops the MR image around the heart using a convolutional neural network (CNN). The second module employs a U-Net to obtain an efficient feature representation of the cardiac image, as well as detect a preliminary location of the landmark points. In the third module, the feature representation of a cardiac image is processed with a Recurrent Neural Network (RNN). The RNN leverages either spatial or temporal dynamics from neighboring slides in time or space and obtains a second prediction for the landmark locations. In the last module the two predictions from the U-Net and RNN are combined and final locations for the landmarks are extracted. The framework is separately trained and evaluated for the localization of each landmark, it achieves a final average error of 2.87 mm for the mitral valve points and an average error of 3.64 mm for the right ventricular insert points. Our method shows that the use of a recurrent neural network for the modeling of additional temporal or spatial dependencies improves localization accuracy and achieves promising results.

AB - Localization of cardiac anatomical landmarks is an important step towards a more robust and accurate analysis of the heart. A fully automatic hybrid framework is proposed that detects key landmark locations in cardiac magnetic resonance (MR) images. Our method is trained and evaluated for the detection of mitral valve points on long-axis MRI and RV insert points in short-axis MRI. The framework incorporates four key modules for the localization of the landmark points. The first module crops the MR image around the heart using a convolutional neural network (CNN). The second module employs a U-Net to obtain an efficient feature representation of the cardiac image, as well as detect a preliminary location of the landmark points. In the third module, the feature representation of a cardiac image is processed with a Recurrent Neural Network (RNN). The RNN leverages either spatial or temporal dynamics from neighboring slides in time or space and obtains a second prediction for the landmark locations. In the last module the two predictions from the U-Net and RNN are combined and final locations for the landmarks are extracted. The framework is separately trained and evaluated for the localization of each landmark, it achieves a final average error of 2.87 mm for the mitral valve points and an average error of 3.64 mm for the right ventricular insert points. Our method shows that the use of a recurrent neural network for the modeling of additional temporal or spatial dependencies improves localization accuracy and achieves promising results.

KW - Cardiac Imaging

KW - Convolutional Neural Network

KW - Deep Learning

KW - Recurrent Neural Network

UR - http://www.scopus.com/inward/record.url?scp=85068342099&partnerID=8YFLogxK

U2 - 10.1117/12.2512048

DO - 10.1117/12.2512048

M3 - Conference contribution

T3 - Proceedings of SPIE

BT - Medical Imaging 2019

PB - SPIE

CY - Bellingham

ER -

van Zon M, Veta M, Li S. Automatic cardiac landmark localization by a recurrent neural network. In Landman BA, Angelini ED, editors, Medical Imaging 2019: Image Processing. Bellingham: SPIE. 2019. 1094916. (Proceedings of SPIE). Available from, DOI: 10.1117/12.2512048