Recently, there has been much interest in using high work function materials as hole-selective contacts for silicon solar cells instead of boron doped amorphous silicon films. The main advantage of these high work function materials is that they typically have lower absorption coefficient compared to doped amorphous silicon and consequently will have a potential for higher short circuit current densities when used at the sunny side of a solar cell. In this work we use SunSolve, the photovoltaic module ray tracer from PV Lighthouse, to assess the optical performance of the most popular hole selective contacts using either amorphous silicon or aluminium oxide interface passivation layers. Our results show that the optical losses can be reduced by 20% relative when using a non-absorbing interface passivation layer resulting in an absolute gain of up to 0.8 mA/cm2 in short-circuit current density at the module level. The absorption in the high work function materials is found to be negligible compared to the transparent conductive oxide and amorphous silicon layers. We also show that the thickness of the hole-selective contact layer affects the current loss in the interface passivation layer and the transparent conductive oxide. Consequently, this work shows the potential optical gain when using high work function materials in conventional front and back contacted solar cells.