This article presents a comprehensive comparative study between two nonlinear solvers for the hybrid analytical modeling (HAM) formulation. The Newton-Raphson method (NRM) and the fixed-point method (FPM) are compared in terms of their convergence rate, computational time, and accuracy. A new HAM formulation using the loop-based magnetic equivalent circuit (MEC) instead of the node-based one is proposed to improve the convergence and condition number. The loop-based formulation is coupled with both NRM and FPM nonlinear solvers to perform the magnetostatic analysis of a 12/10 variable flux reluctance machine (VFRM) under local magnetic saturation. It is shown that both methods can achieve convergence for various saturation levels, mesh, and harmonic refinements. However, FPM exhibits a 0.4 larger convergence rate than NRM. It is also observed that the accuracy of NRM decreases under deep magnetic saturation, and the number of required iterations of NRM increases with the model refinement. However, FPM is able to converge for all analyzed refinements with less than six iterations.