Oxygen is often detected as impurity in metal and metal nitride films prepared by atomic layer deposition (ALD) and its presence has profound and adverse effects on the material properties. In this work, we present the case study of HfNx films prepared by plasma-assisted ALD by alternating exposures of CpHf(NMe2)3 and H2 plasma. First, we identify the primary source of O contamination in the film. Specifically, we find that the extent of O incorporation in HfNx films is determined by the flux of background H2O/O2 residual gases reaching the HfNx surface during the ALD process and leads to the formation of Hf-O bonds. Then, we report on the decrease in the concentration of Hf-O bonds in the film upon application of an external radiofrequency (rf) substrate bias during the H2 plasma step. The experimental work is accompanied by first principles calculations to gain insights into the O incorporation and its mitigation upon the impingement of energetic ions on the surface. Specifically, we find that the dissociative binding of H2O on a bare HfN surface is highly favored, resulting in surface Hf-OH groups and concomitant increase in the oxidation state of Hf. We also show that energetic cations (H+, H2+ and H3+) lead to the dissociation of surface Hf-OH bonds, H2O formation, and its subsequent desorption from the surface. The latter is followed by reduction of the Hf oxidation state, presumably by H radicals. The atomic-level understanding obtained in this work on O incorporation and its abstraction are expected to be crucial to prevent O impurities in the HfNx films and contribute to the fabrication of other technologically relevant low resistivity ALD-grown transition metal nitride films.