Implicit Neural Representations for 3D Reconstruction of X-ray Coronary Angiography

Coronary artery disease (CAD) is the most common heart disease worldwide, caused by a narrowing (stenosis) of the coronary arteries that supply oxygen-rich blood to the heart. The standard treatment is percutaneous coronary intervention (PCI), involving stent deployment. X-ray coronary angiography (XCA) is the standard imaging technique to diagnose CAD and guide PCI, capturing X-ray videos while contrast agent injection visualizes vessel anatomy and motion. However, XCA provides two-dimensional (2D) projections of three-dimensional (3D) anatomy, causing depth distortion that can hinder stenosis assessment and lead to inaccurate stent selection, with potential clinical complications. Although acquiring images from multiple angles can partially mitigate this issue, to minimize radiation exposure, typically only two to four views are obtained, which is insufficient for reliable stenosis assessment. 3D reconstruction of XCA can overcome these limitations by providing accurate vessel anatomy suitable for stent planning. Numerous methods have been proposed, ranging from model-based to machine learning approaches. Yet, several data characteristics of XCA complicate reconstruction: sparse-views, vessel morphology, and poor foreground-background contrast. Furthermore, cardiac and respiratory motion cause temporal inconsistencies requiring dynamic 3D (4D) solutions. As a result, existing methods typically require a large number of views or rely on error-prone (semi-)automatic segmentations, preventing their integration into clinical workflows.Recently, Neural Radiance Fields (NeRFs) have demonstrated high-fidelity reconstruction for natural imaging scenarios. NeRFs encode 3D scenes by mapping spatial coordinates and viewing directions to density and color using continuous functions represented by neural networks, also referred to as implicit neural representations (INRs). For XCA reconstruction, NeRFs seem particularly promising, given their ability to automatically reconstruct from dynamic sparse-view imaging, directly addressing the limitations of previous works. However, their application and clinical potential for XCA remain unexplored. In this dissertation, we explore INRs for the 4D reconstruction of XCA. We (1) investigate the feasibility of INRs for XCA reconstruction; (2) develop INR- and NeRF-based 4D reconstruction and motion estimation methods tailored to XCA data; and (3) introduce techniques to understand the inner workings of NeRFs. In Chapter 4, we conduct an experimental analysis of NeRFs for reconstructing XCA, focusing on XCA-specific challenges, quantitatively and qualitatively evaluating their impact on reconstruction quality with phantom datasets. We highlight strengths, such as the ability to reconstruct from sparse-views, and limitations, including poor foreground-background contrast, identifying key areas that should be addressed in further developments. Shifting from quantitative to clinical analysis, Chapter 5 explores NeRFs’ feasibility for clinical applications like view-planning to optimize acquisition angles. Results, evaluated in a clinician user study, focused on qualitative, clinically relevant characteristics. While promising, they underscored the need for qualitative evaluation and the development of robust models. Having analyzed the feasibility of NeRFs, we next focus on developing 4D reconstruction methods addressing the identified limitations. Specifically, in Chapter 6, we introduce NeRF-CA, a method for dynamic reconstruction of sparse-view XCA. Addressing cardiac motion and the poor foreground-background distinction, NeRF-CA utilizes static and dynamic decomposition using neural representations, achieving reasonable reconstructions from four views. Building on NeRF-CA, Chapter 7 introduces NerT-CA, which combines neural and tensorial representations to achieve more efficient reconstructions. This approach reduces reconstruction time from hours to minutes while producing higher-quality results from only three views, bringing the method closer to clinical applicability. NeRF-CA and NerT-CA demonstrate significant contributions in phantom settings with cardiac-induced motion. However, clinical data also involves unmeasured respiratory motion, which introduces additional non-rigid displacement. In Chapter 8, we propose INSPIRE, an INR-based method to estimate respiratory motion directly from XCA sequences. This estimated respiratory motion signal could be used to guide further developments to achieve 4D reconstructions in real-world XCA settings. While the previous chapters highlight the potential of INRs for XCA, they do not address a key limitation: their “black-box” nature. The clinical feasibility study of Chapter 5 highlights the need to assess model quality and detect unreliable predictions. To address this, we introduce NeRVis (Chapter 9), a visualization approach for exploring uncertainty in 3D NeRF reconstructions. NeRVis enables users to identify critical views for accurate reconstruction in synthetic settings, laying the foundation for clinical applications like view-planning.All in all, this dissertation addresses the 4D reconstruction of XCA through the lens of INRs. We evaluated their feasibility from both methodological and clinical perspectives, developed 4D reconstruction and motion estimation methods, and introduced an approach to enhance model understanding. Together, these contributions advance 4D reconstruction of XCA by reducing clinical requirements and improving model reliability, ultimately supporting better diagnosis and treatment of CAD.