Myocardial blood flow can be quantified from dynamic contrast-enhanced magnetic resonance (MR) images through the fitting of tracer-kinetic models to the observed imaging data. The use of multi-compartment exchange models is desirable as they are physiologically motivated and resolve directly for both blood flow and microvascular function. However, the parameter estimates obtained with such models can be unreliable. This is due to the complexity of the models relative to the observed data which is limited by the low signal-to-noise ratio, the temporal resolution, the length of the acquisitions and other complex imaging artefacts. In this work, a Bayesian inference scheme is proposed which allows the reliable estimation of the parameters of the two-compartment exchange model from myocardial perfusion MR data. The Bayesian scheme allows the incorporation of prior knowledge on the physiological ranges of the model parameters and facilitates the use of the additional information that neighbouring voxels are likely to have similar kinetic parameter values. Hierarchical priors are used to avoid making a priori assumptions on the health of the patients. We provide both a theoretical introduction to Bayesian inference for tracer-kinetic modelling and specific implementation details for this application. This approach is validated in both in silico and in vivo settings. In silico, there was a significant reduction in mean-squared error with the ground-truth parameters using Bayesian inference as compared to using the standard non-linear least squares fitting. When applied to patient data the Bayesian inference scheme returns parameter values that are in-line with those previously reported in the literature, as well as giving parameter maps that match the independant clinical diagnosis of those patients.