TY - JOUR
T1 - Active Inference and Deep Generative Modeling for Cognitive Ultrasound
AU - van Sloun, Ruud J.G.
N1 - Publisher Copyright:
© 1986-2012 IEEE.
PY - 2024/11
Y1 - 2024/11
N2 - Ultrasound (US) has the unique potential to offer access to medical imaging to anyone, everywhere. Devices have become ultraportable and cost-effective, akin to the stethoscope. Nevertheless, and despite many advances, US image quality and diagnostic efficacy are still highly operator-and patient-dependent. In difficult-To-image patients, image quality is often insufficient for reliable diagnosis. In this article, we put forth the idea that US imaging systems can be recast as information-seeking agents that engage in reciprocal interactions with their anatomical environment. Such agents autonomously adapt their transmit-receive sequences to fully personalize imaging and actively maximize information gain in situ. To that end, we will show that the sequence of pulse-echo experiments that a US system performs can be interpreted as a perception-Action loop: The action is the data acquisition, probing tissue with acoustic waves and recording reflections at the detection array, and perception is the inference of the anatomical and or functional state, potentially including associated diagnostic quantities. We then equip systems with a mechanism to actively reduce uncertainty and maximize diagnostic value across a sequence of experiments, treating action and perception jointly using Bayesian inference given generative models of the environment and action-conditional pulse-echo observations. Since the representation capacity of the generative models dictates both the quality of inferred anatomical states and the effectiveness of inferred sequences of future imaging actions, we will be greatly leveraging the enormous advances in deep generative modeling (generative AI), which are currently disrupting many fields and society at large. Finally, we show some examples of cognitive, closed-loop, US systems that perform active beamsteering and adaptive scanline selection based on deep generative models that track anatomical belief states.
AB - Ultrasound (US) has the unique potential to offer access to medical imaging to anyone, everywhere. Devices have become ultraportable and cost-effective, akin to the stethoscope. Nevertheless, and despite many advances, US image quality and diagnostic efficacy are still highly operator-and patient-dependent. In difficult-To-image patients, image quality is often insufficient for reliable diagnosis. In this article, we put forth the idea that US imaging systems can be recast as information-seeking agents that engage in reciprocal interactions with their anatomical environment. Such agents autonomously adapt their transmit-receive sequences to fully personalize imaging and actively maximize information gain in situ. To that end, we will show that the sequence of pulse-echo experiments that a US system performs can be interpreted as a perception-Action loop: The action is the data acquisition, probing tissue with acoustic waves and recording reflections at the detection array, and perception is the inference of the anatomical and or functional state, potentially including associated diagnostic quantities. We then equip systems with a mechanism to actively reduce uncertainty and maximize diagnostic value across a sequence of experiments, treating action and perception jointly using Bayesian inference given generative models of the environment and action-conditional pulse-echo observations. Since the representation capacity of the generative models dictates both the quality of inferred anatomical states and the effectiveness of inferred sequences of future imaging actions, we will be greatly leveraging the enormous advances in deep generative modeling (generative AI), which are currently disrupting many fields and society at large. Finally, we show some examples of cognitive, closed-loop, US systems that perform active beamsteering and adaptive scanline selection based on deep generative models that track anatomical belief states.
KW - Active inference
KW - adaptive compressed sensing
KW - cognitive imaging
KW - computational imaging
KW - deep generative models
KW - deep learning
KW - perception and action
KW - ultrasound (US) imaging
UR - http://www.scopus.com/inward/record.url?scp=85204990358&partnerID=8YFLogxK
U2 - 10.1109/TUFFC.2024.3466290
DO - 10.1109/TUFFC.2024.3466290
M3 - Article
C2 - 39312433
AN - SCOPUS:85204990358
SN - 0885-3010
VL - 71
SP - 1478
EP - 1490
JO - IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
JF - IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
IS - 11
ER -