We consider a single-product make-to-stock manufacturing-remanufacturing system. Returned products require remanufacturing before they can be sold. The manufacturing and remanufacturing operations are executed by the same single server, where switching from one activity to another does not involve time or cost and can be done at an arbitrary moment in time. Customer demand can be fulfilled by either newly manufactured or remanufactured products. The times for manufacturing and remanufacturing a product are exponentially distributed. Demand and used products arrive via mutually independent Poisson processes. Disposal of products is not allowed and all used products that are returned have to be accepted. Using Markov decision processes, we investigate the optimal manufacture-remanufacture policy that minimizes holding, backorder, manufacturing and remanufacturing costs per unit of time over an infinite horizon. For a subset of system parameter values we are able to completely characterize the optimal continuous-review dynamic preemptive policy. We provide an efficient algorithm based on quasi-birth-death processes to compute the optimal policy parameter values. For other sets of system parameter values, we present some structural properties and insights related to the optimal policy and the performance of some simple threshold policies.