Using a thin capping layer to engineer the structural and optical properties of InAs/GaAs quantum dots (QDs) has become common practice in the last decade. Traditionally, the main parameter considered has been the strain in the QD/capping layer system. With the advent of more exotic alloys, it has become clear that other mechanisms significantly alter the QD size and shape as well. Larger bond strengths, surfactants, and phase separation are known to act on QD properties but are far from being fully understood. In this study, we investigate at the atomic scale the influence of these effects on the morphology of capped QDs with cross-sectional scanning tunneling microscopy. A broad range of capping materials (InGaAs, GaAsSb, GaAsN, InGaAsN, and GaAsSbN) are compared. The QD morphology is related to photoluminescence characteristics.