Implementation of dynamic programming for optimal control problems with continuous states

K. van Berkel, Bram de Jager, T. Hofman, M. Steinbuch

Research output: Contribution to journalArticleAcademicpeer-review

21 Citations (Scopus)
135 Downloads (Pure)


Dynamic programming is a numerical method to solve a dynamic optimal control problem. Due to its numerical framework, it is very suitable to describe discrete dynamics, nonlinear characteristics, and nonconvex constraints. The implementation of continuous states in the discrete framework, however, may lead to optimization inaccuracies. This brief addresses implementation methods with fundamentally different utilizations of the nodes in the quantized time-state space. A new implementation method is presented, which combines the advantages of numerical and analytical optimization techniques to substantially improve the optimization accuracy for a given quantization of the continuous state. If desired, the computation time can be substantially reduced for a given accuracy by lowering the quantization resolution. As a case study, the optimal energy controller is computed for a mechanical hybrid powertrain, which is characterized by switched dynamics, active state constraints, and nonconvex control constraints. Results show that the optimization accuracy of the new method is superior to that of the conventional method based on nearest neighbor rounding. For a given desired accuracy, the computation time is reduced by an order of magnitude.

Original languageEnglish
Article number6917009
Pages (from-to)1172-1179
Number of pages8
JournalIEEE Transactions on Control Systems Technology
Issue number3
Publication statusPublished - 1 May 2015


  • Accuracy
  • Dynamic programming
  • Interpolation
  • Optimal control
  • Optimization
  • Quantization (signal)
  • Trajectory
  • energy management


Dive into the research topics of 'Implementation of dynamic programming for optimal control problems with continuous states'. Together they form a unique fingerprint.

Cite this