The effect of side chain length on the photovoltaic properties of conjugated polymers is systematically investigated with two sets of polymers that bear different alkyl side chain lengths based on benzodithiophene and benzo[2,1,3]thiadiazole or 5,6-difluorobenzo[2,1,3]thiadiazole. Characterization of the photovoltaic cells reveals a strong interdependency between the side chain length of conjugated polymers and photovoltaic performances (power conversion efficiency, short-circuit current, and fill factor) of the resulting bulk-heterojunction (BHJ) solar cells. Charge carrier transport and external quantum efficiency (EQE) measurements in combination with morphology characterization suggest that too long side chains lead to deteriorated charge transport, suboptimal BHJ morphology, considerable bimolecular recombination, and consequently poor photovoltaic performances. On the other hand, when the side chains are too short, they cannot afford a high enough solubility and molecular weight for the resulting polymers and produce poor solar cell performance as well. This study shows that side chain optimization is of significant importance to maximize the potential of photovoltaic active conjugated polymers, which indicates the fruitful molecular design rules toward highly efficient BHJ polymer solar cells.