A class of dynamic models to describe finite anisotropic elasto-viscoplastic behavior in an Eulerian setting are examined from the perspective of non-equilibrium thermodynamics. A proper description of anisotropic elastic behavior, using the deformation gradient as an internal variable, serves as a starting point. A general model for elasto-viscoplastic deformation is then formulated by allowing isochoric relaxation of the total deformation gradient, giving rise to an elastic deformation gradient and a plastic strain rate. Thermodynamic restrictions on specific forms of the plastic strain rate for anisotropic materials, using the representation theorem for tensor functions, are discussed. In addition, the thermodynamic admissibility of a multi-mode formulation of the elasto-viscoplastic model is demonstrated. Finally, the model is applied to describe rate-dependent inelastic deformation of an amorphous polymer glass, and anisotropic strain-rate dependent crystalline slip as found in metal plasticity.