Interval arithmetic is based on the fact that for intervals on the real line, the element-wise product of two intervals is also an interval. This property is not always true: e.g., it is not true if we consider intervals on the set of integers instead of intervals on the set of real numbers. When is an element-wise product or a sum of two intervals always an interval? In this paper, we analyze this problem in a general algebraic setting: we need the corresponding algebraic structures to have (related) addition, multiplication, and order; thus, we consider (consistently) ordered rings. We describe all consistently ordered rings for which the element-wise product and sum of two intervals are always intervals.