Delayed Workload Shifting in Many-server Systems

Motivated by the desire to shift workload during periods of overload, we extend established square-root capacity sizing rules for many-server systems in the Quality-and-Efficiency Driven (QED) regime. We propose Delayed Workload Shifting (DWS) which has two defining features: when there are n users in the system, newly arriving users are no longer admitted directly. Instead, these users will reattempt getting access after a stochastic delay until they are successful. The goal of DWS is to release pressure from the system during overloaded periods, and indeed we show that the performance gain can be substantial. We derive nontrivial corrections to classical QED approximations to account for DWS, and leverage these to control stationary and time-varying system behavior.