Polarization-resolved reflection measurements are performed on nearly circular InAs/GaAs quantum dots (QDs) by means of time-resolved differential reflection spectroscopy. We observe linear polarization anisotropy of the differential absorption, revealing the dichroic character of the QD reflection. The observed magnitude of the dichroism is Theta/Theta=1.07. The polarization has a preferential direction orientated along the  crystal axis, which is confirmed by polarization-resolved photoluminescence. We observe that the polarization anisotropy of the reflectivity is strongly dependent on the pump excitation density, decreasing from Rho=0.14 at low excitation to Rho=0.06 at high excitation. The pump power dependence is described by a binomial model taking into account the statistics of carrier capture into a limited number of QDs.