Continuum frameworks of dislocation based plasticity theories are gaining prominence in the researchcommunity. In these theories, the underlying discrete lattice defects are represented by an averagedcontinuous description of a signed dislocation density. The long range stress fields are accuratelycharacterized but the short range interactions are modeled phenomenologically. In this paper, wedemonstrate by a rigorous analysis that short-range interactions resulting from certain aspects of theunderlying discreteness cannot be neglected. An idealized problem of dislocation pile-ups against a hardobstacle is used to illustrate this observation. It is also demonstrated that the modeling of short rangeinteractions by a local gradient of dislocation distribution has limitations. It is realized that even though thestress contribution for distant dislocations is relatively small, it is the accumulation of these stresscontributions from numerous such dislocations which culminates in substantial contributions. It would beinaccurate to neglect these effects. Our benchmark problem can be used for calibration of current and futuretheories of plasticity that attempt to accurately model short range interactions.