We describe an interfacial rheometry technique based on pairs of micrometer-sized magnetic particles at a fluid-fluid interface. The particles are repeatedly attracted and repelled by well-controlled magnetic dipole-dipole forces, so-called interfacial rheometry by intra-pair magnetophoresis (IPM). From the forces (∼pN), displacements (∼μm) and velocities (∼μm s-1) of the particles we are able to quantify the interfacial drag coefficient of particles within a few seconds and over very long timescales. The use of local dipole-dipole forces makes the system insensitive to fluid flow and suited for simultaneously recording many particles in parallel over a long period of time. We apply IPM to study the time-dependent adsorption of an oil-soluble amino-modified silicone polymer at a water-oil interface using carboxylated magnetic particles. At low polymer concentration the carboxylated particles remain on the water side of the water-oil interface, while at high polymer concentrations the particles transit into the oil phase. Both conditions show a drag coefficient that does not depend on time. However, at intermediate polymer concentrations data show an increase of the interfacial drag coefficient as a function of time, with an increase over more than three orders of magnitude (10-7 to 10-4 N s m-1), pointing to a strong polymer-polymer interaction at the interface. The time-dependence of the interfacial drag appears to be highly sensitive to the polymer concentration and to the ionic strength of the aqueous phase. We foresee that IPM will be a very convenient technique to study fluid-fluid interfaces for a broad range of materials systems.