For the purpose of quantifying models of letter recognition, similarities are often specified in terms of stimulus properties. In this paper, an approach is advocated based on similarities between internal letter representations or internal letter images, i.e. it is argued that optical and retinal factors play a more prominent role in letter confusions than is usually assumed. To illustrate this, letter images were calculated on the basis of earlier experimentally determined point spread functions (Blommaert et al., Spatial Vision 2, 99-115, 1987). Next, data on confusion matrices from Bouma (Vision Res. 11, 459-474, 1971) were taken to evaluate different measures which might be useful for quantifying similarities between internal letter representations. In the analysis of experimental data, Luce's (In: Handbook of Mathematical Psychology, 1963) choice model was used. It was found that if similarities were expressed in terms of differences between image contours, a fair first order approximation of Bouma's experimental results could be formulated (overall correlation coefficient of 0.95). Other measures like correlations between spatial frequency spectra of letter images were found to be less successful. The method used provides a means to relate quantitatively stimulus features and optical and early-visual factors to letter confusions.