This paper is concerned with the numerical evaluation of multi-echelon production systems. Each stage requires a fixed predetermined leadtime; furthermore, we assume a stochastic, stationary end-time demand process. In a previous paper, we have developed an analytical framework for determining optimal control policies for such systems under an average cost criterion. The current paper is based on this analytical theory but discusses computational aspects, in particular for serial and assembly systems. A hierarchical (exact) decomposition of these systems can be obtained by considering echelon stocks and by transforming penalty and holding costs accordingly. The one-dimensional problems arising after this decomposition however involve incomplete convolutions of distribution functions, which are only recursively defined. We develop numerical procedures for analysing these incomplete convolutions; these procedures are based on approximations of distribution functions by mixtures of Erlang distributions. Combining the analytically obtained (exact) decomposition results with these numerical procedures enables us to quickly determine optimal order-up-to levels for all stages. Moreover, expressions for the customer service level of such a multi-stage are obtained, yielding the possibility to determine policies which minimize average inventory holding costs, given a service level constraint.