We develop a model for the magnonic contribution to the unidirectional spin Hall magnetoresistance (USMR) of heavy metal/ferromagnetic insulator bilayer films. We show that diffusive transport of Holstein-Primakoff magnons leads to an accumulation of spin near the bilayer interface, giving rise to a magnoresistance which is not invariant under inversion of the current direction. Unlike the electronic contribution described by Zhang and Vignale [Phys. Rev. B 94, 140411 (2016)2469-995010.1103/PhysRevB.94.140411], which requires an electrically conductive ferromagnet, the magnonic contribution can occur in ferromagnetic insulators such as yttrium iron garnet. We show that the magnonic USMR is, to leading order, cubic in the spin Hall angle of the heavy metal, as opposed to the linear relation found for the electronic contribution. We estimate that the maximal magnonic USMR in Pt|YIG bilayers is on the order of 10-8 but may reach values of up to 10-5 if the magnon gap is suppressed and can thus become comparable to the electronic contribution in, e.g., Pt|Co. We show that the magnonic USMR at a finite magnon gap may be enhanced by an order of magnitude if the magnon diffusion length is decreased to a specific optimal value that depends on various system parameters.