The electrical resistivity of thin gold films deposited on amorphous substrates is found to increase with decreasing film thickness. The temperature-dependent part of the resistivity, however, is independent of the film thickness. This observation cannot be reconciled with existing theories that deal with surface scattering effects. We show that electron scattering at grain boundaries explains our measurements. Moreover, the main effect of the surface appears to be a change in the effective Debye temperature.