We have determined the size, shape, and composition of InAs/GaAs quantum dots (QDs) and InAs QDs embedded in an AlAs barrier by cross-sectional scanning tunneling microscopy. The outward relaxation and lattice constant of the cleaved surface of the QDs and their wetting layers were calculated using continuum elasticity theory and compared with experimental data in order to determine the indium concentration of the dots. Based on the structural results we have calculated the electronic ground states of the dots using a single band, effective mass approach. We find that the calculated ground state photoluminescence energy of the InAs/GaAs dots is in excellent agreement with the measured energy. The observed large width of the PL spectrum of InAs/AlAs dots can be attributed to G-G electron-hole recombination within an ensemble of dots with sizes varying between 2.4–4.2 nm in height and 10–20 nm along the base diagonal. We find that the electron-hole wave function overlap of small InAs/AlAs QDs is 7.6 times larger than that of InAs/GaAs QDs grown under the same conditions. This supports the explanation of the long decay times in InAs/AlAs dots by an enhanced exciton exchange splitting.