The combination of digital image correlation (DIC) and scanning electron microscopy (SEM) enables to extract high resolution full field displacement data, based on the high spatial resolution of SEM and the sub-pixel accuracy of DIC. However, SEM images may exhibit a considerable amount of imaging artifacts, which may seriously compromise the accuracy of the displacements and strains measured from these images. The current study proposes a unified general framework to correct for the three dominant types of SEM artifacts, i.e. spatial distortion, drift distortion and scan line shifts. The artifact fields are measured alongside the mechanical deformations to minimize the artifact induced errors in the latter. To this purpose, Integrated DIC (IDIC) is extended with a series of hierarchical mapping functions that describe the interaction of the imaging process with the mechanics. A new IDIC formulation based on these mapping functions is derived and the potential of the framework is tested by a number of virtual experiments. The effect of noise in the images and different regularization options for the artifact fields are studied. The error in the mechanical displacement fields measured for noise levels up to 5% is within the usual DIC accuracy range for all the cases studied, while it is more than 4 pixels if artifacts are ignored. A validation on real SEM images at three different magnifications confirms that all three distortion fields are accurately captured. The results of all virtual and real experiments demonstrate the accuracy of the methodology proposed, as well as its robustness in terms of convergence.