Recent breakthroughs in artificial neural networks (ANNs) have spurred interest in efficient computational paradigms where the energy and time costs for training and inference are reduced. One promising contender for efficient ANN implementation is crossbar arrays of resistive memory elements that emulate the synaptic strength between neurons within the ANN. Organic nonvolatile redox memory has recently been demonstrated as a promising device for neuromorphic computing, offering a continuous range of linearly programmable resistance states and tunable electronic and electrochemical properties, opening a path toward massively parallel and energy efficient ANN implementation. However, one of the key issues with implementations relying on electrochemical gating of organic materials is the state-retention time and device stability. Here, revealed are the mechanisms leading to state loss and cycling instability in redox-gated neuromorphic devices: parasitic redox reactions and out-diffusion of reducing additives. The results of this study are used to design an encapsulation structure which shows an order of magnitude improvement in state retention and cycling stability for poly(3,4-ethylenedioxythiophene)/polyethyleneimine:poly(styrene sulfonate) devices by tuning the concentration of additives, implementing a solid-state electrolyte, and encapsulating devices in an inert environment. Finally, a comparison is made between programming range and state retention to optimize device operation.