For polymer tandem solar cells it is not easy to determine the individual internal electric fields across each of the two individual photoactive layers because the intermediate contact is often buried in the device stack and cannot be contacted. Here we explore electroabsorption (EA) spectroscopy as a possible tool to investigate these electric fields in polymer tandem solar cells. EA spectroscopy is a noninvasive electro-optical technique that can be used to probe internal electrical fields in organic thin film devices. The tandem cell investigated consists of a wide band gap front cell and a small band gap back cell, connected via an intermediate recombination contact. We demonstrate that the EA spectra of the individual subcells and their dependence on applied external bias can be distinguished in the EA spectrum of the tandem cells but that a quantitative interpretation of the EA spectra in terms of individual internal fields is hampered by a nonlinear bias dependence of the EA intensity on the applied external bias, especially for the small band gap cell. By studying the corresponding single junction wide and small band gap cells, we establish that the nonlinear bias dependence finds its origin in a spectral shift of the EA signal due to a change in the ratio of two different components contributing to the EA spectrum. The two components have been identified as being due to the Stark effect and to charges induced into the active layer in the EA experiment.