The linear and nonlinear optical properties of metal nanoparticles are highly tunable by variation of parameters such as particle size, shape, composition, and environment. To fully exploit this tunability, however, quantitative information on nonlinear absorption cross sections is required, as well as a sufficient understanding of the physical mechanism underlying these nonlinearities. In this work, we present a detailed and systematic investigation of the wavelength-dependent nonlinear optical properties of Ag nanoparticles embedded in a glass host, in which the most important parameters determining the nonlinear behavior of the system are characterized. This allows a proper quantification of absorption cross sections and elucidation of the excitation mechanism. Based on small-angle X-ray scattering measurements average particle diameters of 3 and 17 nm are estimated for the studied samples. The nonlinear optical properties of the nanoparticle-glass composite are studied in an extended wavelength range with the open aperture z-scan technique. The experiments reveal a strong dependence of the nonlinear optical response on the excitation wavelength. Based on the wavelength-dependent response, excited-state absorption is determined as the excitation mechanism of the nanoparticles. Electromagnetic simulations demonstrate that the contributions from electric field enhancement and plasmonic coupling between the particles in the diluted glasses are limited, which implies that the very high two-photon absorption cross section at 460 nm ((6.9 ± 1.6) × 106 GM for the 3 nm particles and (19.5 ± 2.2) × 109 GM for the 17 nm particles) is an intrinsic property. In addition, irradiance-dependent measurements elucidate the role of saturation of the excited-state absorption process on the observed nonlinearities.