Growing interests have been devoted to the synthesis of polymer acceptors as alternatives to fullerene derivatives to realize high-performance and stable all-polymer solar cells (all-PSCs). So far, one of the key factors that limit the performance of all-PSCs is low photocurrent density (normally < 14 mA/cm2). One potential solution is to improve the dielectric constants (εr) of polymer:polymer blends, which tend to reduce the binding energy of excitons, thus boosting the exciton dissociation efficiencies. Nevertheless, the correlation between εr and photovoltaic performance has been rarely investigated for all-PSCs. In this work, five fluorinated naphthalene diimide (NDI)-based acceptor polymers, with different content of fluorine were synthesized. The incorporation of fluorine increased the εr of the acceptor polymers and blend films, which improved the charge generation and overall photocurrent of the all-PSCs. As a result, the PTB7-Th:PNDI-FT10 all-PSC attained a high power conversion efficiency (PCE) of 7.3% with a photocurrent density of 14.7 mA/cm2, which surpassed the values reported for the all-PSC based on the non-fluorinated acceptor PNDI-T10. Interestingly, similarly high photovoltaic performance was maintained regardless of a large variation of donor:acceptor ratios, which revealed the good morphological tolerance and the potential for robust production capability of all-PSCs.