Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks

Arash Pourtaherian, Farhad Ghazvinian Zanjani, Svitlana Zinger, Nenad Mihajlovic, Gary C. Ng, Hendrikus H.M. Korsten, Peter H.N. de With

Research output: Contribution to journalArticleAcademicpeer-review

36 Citations (Scopus)
121 Downloads (Pure)


Purpose: During needle interventions, successful automated detection of the needle immediately after insertion is necessary to allow the physician identify and correct any misalignment of the needle and the target at early stages, which reduces needle passes and improves health outcomes. Methods: We present a novel approach to localize partially inserted needles in 3D ultrasound volume with high precision using convolutional neural networks. We propose two methods based on patch classification and semantic segmentation of the needle from orthogonal 2D cross-sections extracted from the volume. For patch classification, each voxel is classified from locally extracted raw data of three orthogonal planes centered on it. We propose a bootstrap resampling approach to enhance the training in our highly imbalanced data. For semantic segmentation, parts of a needle are detected in cross-sections perpendicular to the lateral and elevational axes. We propose to exploit the structural information in the data with a novel thick-slice processing approach for efficient modeling of the context. Results: Our introduced methods successfully detect 17 and 22 G needles with a single trained network, showing a robust generalized approach. Extensive ex-vivo evaluations on datasets of chicken breast and porcine leg show 80 and 84% F1-scores, respectively. Furthermore, very short needles are detected with tip localization errors of less than 0.7 mm for lengths of only 5 and 10 mm at 0.2 and 0.36 mm voxel sizes, respectively. Conclusion: Our method is able to accurately detect even very short needles, ensuring that the needle and its tip are maximally visible in the visualized plane during the entire intervention, thereby eliminating the need for advanced bi-manual coordination of the needle and transducer.

Original languageEnglish
Pages (from-to)1321-1333
Number of pages13
JournalInternational Journal of Computer Assisted Radiology and Surgery
Issue number9
Publication statusPublished - 1 Sept 2018


  • 3D ultrasound
  • Convolutional neural networks
  • Needle detection


Dive into the research topics of 'Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks'. Together they form a unique fingerprint.

Cite this