Psychophysical evaluation of image quality : from judgment to impression

Designs of imaging systems, image processing algorithms etc. usually take for granted that methods for assessing perceived image quality produce unbiased estimates of the viewers' quality impression. Quality judgments, however, are affected by the judgment strategies induced by the experimental procedures. In this paper the results of two experiments are presented illustrating the influence judgment strategies can have on quality judgments. The first experiment concerns contextual effects due to the composition of the stimulus set. Subjects assessed the sharpness of two differently composed sets of blurred versions of one static image. The sharpness judgments for the blurred images present in both stimulus sets were found to be dependent on the composition of the set. The magnitude of this effect was determined by the scaling technique employed. In the second experiment subjects assessed either the overall quality or the overall impairment of JPEG-coded images containing two main artifacts. Two scaling methods were used: single stimulus scaling and comparison scaling. The results indicate a systematic difference between the quality and impairment judgments that could be interpreted as an instruction-based different weighting of the two artifacts. Again, some influence of scaling technique was observed. The results of both experiments underscore the important role judgment strategies play in the psychophysical evaluation of image quality. Ignoring this influence on quality judgments may lead to invalid conclusions about the viewers' impression of image quality. Accordingly, knowledge about judgment strategies is indispensable in designing and improving evaluation techniques.