Atomic layer deposition (ALD) is used in applications where inorganic material layers with uniform thickness down to the nanometer range are required. For such thicknesses, the growth mode, defining how the material is arranged on the surface during the growth, is of critical importance. In this work, the growth mode of the zirconium tetrachloride/water and the trimethyl aluminum/water ALD process on hydrogen-terminated silicon was investigated by combining information on the total amount of material deposited with information on the surface fraction of the material. The total amount of material deposited was measured by Rutherford backscattering, x-ray fluorescence, and inductively coupled plasma-optical emission spectroscopy, and the surface fractions by low-energy ion scattering. Growth mode modeling was made assuming two-dimensional growth or random deposition (RD), with a "shower model" of RD recently developed for ALD. Experimental surface fractions of the ALD-grown zirconium oxide and aluminum oxide films were lower than the surface fractions calculated assuming RD, suggesting the occurrence of island growth. Island growth was confirmed with transmission electron microscopy (TEM) measurements, from which the island size and number of islands per unit surface area could also be estimated. The conclusion of island growth for the aluminum oxide deposition on hydrogen-terminated silicon contradicts earlier observations. In this work, physical aluminum oxide islands were observed in TEM after 15 ALD reaction cycles. Earlier, thicker aluminum oxide layers have been analyzed, where islands have not been observed because they have already coalesced to form a continuous film. The unreactivity of hydrogen-terminated silicon surface towards the ALD reactants, except for reactive defect areas, is proposed as the origin of island growth. Consequently, island growth can be regarded as "undesired surface-selective ALD."