The photo-oxidation behavior of three different polymers—namely, poly(3-hexylthiophene) (P3HT), poly[2,6-(4,4-bis(2-ethylhexyl)-4H-cyclopenta[2,1-b;3,4-b']dithiophene)-alt-4,7-(2,1,3-benzothiadiazole)] (C-PCPDTBT), and poly[2,6-(4,4-bis(2-ethylhexyl)dithieno[3,2-b:2',3'-d]silole)-alt-4,7-(2,1,3-benzothiadiazole)] (Si-PCPDTBT)—is investigated in neat polymer films and in blends with [6,6]-phenyl-C61-butyric acid methyl ester (PCBM) for different polymer:PCBM ratios. PCBM is shown to have both stabilizing and destabilizing effects, the extent of which is dependent on the type of polymer with which it is blended. Screening of the polymer from incident light by PCBM turns out to play an only minor role in the stabilization of P3HT. Quenching of the polymer excited states is also not a significant stabilization mechanism, as demonstrated by the comparison of the reduction of photo-oxidation rates to the extent of photoluminescence quenching by PCBM and 2,7-dinitrofluorenone (DNF). Photoinduced absorption spectroscopy reveals that the enhanced degradation of C-PCPDTBT in blend films with PCBM correlates with the population of the polymer triplet state via the polymer:PCBM charge-transfer state.