We present a comparative study of excited states in push-pull oligomers of PCPDTBT and PSBTBT and prototypical complexes with a C60 acceptor using many-body Green's functions theory within the GW approximation and the Bethe-Salpeter equation. We analyze excitations in oligomers up to a length of 5 nm and find that for both materials the absorption energy practically saturates for structures larger than two repeat units due to the localized nature of the excitation. In the bimolecular complexes with C60, the transition from Frenkel to charge transfer excitons is generally exothermic and strongly influenced by the acceptor's position and orientation. The high CT binding energy of the order of 2 eV results from the lack of an explicit molecular environment. External polarization effects are then modeled in a GW-BSE based QM/MM approach by embedding the donor-acceptor complex into a polarizable lattice. The lowest charge transfer exciton is energetically stabilized by about 0.5 eV, while its binding energy is reduced to about 0.3 eV. We also identify a globally unbound charge transfer state with a more delocalized hole at higher energy while still within the absorption spectrum, which opens another potential pathway for charge separation. For both PCPDTBT and PSBTBT, the energetics are largely similar with respect to absorption and the driving force to form intermediate charge transfer excitations for free charge generation. These results support that the higher power conversion efficiency observed for solar cells using PSBTBT as donor material is a result of molecular packing rather than of the electronic structure of the polymer.