Model-based multi-objective reinforcement learning

M.A. Wiering; M. Withagen; M.M. Drugan

doi:10.1109/ADPRL.2014.7010622

Model-based multi-objective reinforcement learning

M.A. Wiering
, M. Withagen
, M.M. Drugan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

40 Citations (Scopus)

Abstract

This paper describes a novel multi-objective reinforcement learning algorithm. The proposed algorithm first learns a model of the multi-objective sequential decision making problem, after which this learned model is used by a multi-objective dynamic programming method to compute Pareto optimal policies. The advantage of this model-based multi-objective reinforcement learning method is that once an accurate model has been estimated from the experiences of an agent in some environment, the dynamic programming method will compute all Pareto optimal policies. Therefore it is important that the agent explores the environment in an intelligent way by using a good exploration strategy. In this paper we have supplied the agent with two different exploration strategies and compare their effectiveness in estimating accurate models within a reasonable amount of time. The experimental results show that our method with the best exploration strategy is able to quickly learn all Pareto optimal policies for the Deep Sea Treasure problem.

Original language	English
Title of host publication	2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 9-12 December 2014, Orlando, Florida
Place of Publication	Piscataway
Publisher	Institute of Electrical and Electronics Engineers
Pages	1-6
ISBN (Electronic)	978-1-4799-4552-8
ISBN (Print)	9781479945535
DOIs	https://doi.org/10.1109/ADPRL.2014.7010622
Publication status	Published - 14 Jan 2014
Externally published	Yes
Event	2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014) - Orlando, United States Duration: 9 Dec 2014 → 12 Dec 2014

Conference

Conference	2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014)
Abbreviated title	ADPRL 2014
Country/Territory	United States
City	Orlando
Period	9/12/14 → 12/12/14

Access to Document

10.1109/ADPRL.2014.7010622

Cite this

@inproceedings{73cd4c08f3e8476f9dd912908bcb1819,

title = "Model-based multi-objective reinforcement learning",

abstract = "This paper describes a novel multi-objective reinforcement learning algorithm. The proposed algorithm first learns a model of the multi-objective sequential decision making problem, after which this learned model is used by a multi-objective dynamic programming method to compute Pareto optimal policies. The advantage of this model-based multi-objective reinforcement learning method is that once an accurate model has been estimated from the experiences of an agent in some environment, the dynamic programming method will compute all Pareto optimal policies. Therefore it is important that the agent explores the environment in an intelligent way by using a good exploration strategy. In this paper we have supplied the agent with two different exploration strategies and compare their effectiveness in estimating accurate models within a reasonable amount of time. The experimental results show that our method with the best exploration strategy is able to quickly learn all Pareto optimal policies for the Deep Sea Treasure problem.",

author = "M.A. Wiering and M. Withagen and M.M. Drugan",

year = "2014",

month = jan,

day = "14",

doi = "10.1109/ADPRL.2014.7010622",

language = "English",

isbn = "9781479945535",

pages = "1--6",

booktitle = "2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 9-12 December 2014, Orlando, Florida",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014), ADPRL 2014 ; Conference date: 09-12-2014 Through 12-12-2014",

}

Wiering, MA, Withagen, M & Drugan, MM 2014, Model-based multi-objective reinforcement learning. in 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 9-12 December 2014, Orlando, Florida., 7010622, Institute of Electrical and Electronics Engineers, Piscataway, pp. 1-6, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014), Orlando, Florida, United States, 9/12/14. https://doi.org/10.1109/ADPRL.2014.7010622

Model-based multi-objective reinforcement learning. / Wiering, M.A.; Withagen, M.; Drugan, M.M.
2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 9-12 December 2014, Orlando, Florida. Piscataway: Institute of Electrical and Electronics Engineers, 2014. p. 1-6 7010622.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Model-based multi-objective reinforcement learning

AU - Wiering, M.A.

AU - Withagen, M.

AU - Drugan, M.M.

PY - 2014/1/14

Y1 - 2014/1/14

N2 - This paper describes a novel multi-objective reinforcement learning algorithm. The proposed algorithm first learns a model of the multi-objective sequential decision making problem, after which this learned model is used by a multi-objective dynamic programming method to compute Pareto optimal policies. The advantage of this model-based multi-objective reinforcement learning method is that once an accurate model has been estimated from the experiences of an agent in some environment, the dynamic programming method will compute all Pareto optimal policies. Therefore it is important that the agent explores the environment in an intelligent way by using a good exploration strategy. In this paper we have supplied the agent with two different exploration strategies and compare their effectiveness in estimating accurate models within a reasonable amount of time. The experimental results show that our method with the best exploration strategy is able to quickly learn all Pareto optimal policies for the Deep Sea Treasure problem.

AB - This paper describes a novel multi-objective reinforcement learning algorithm. The proposed algorithm first learns a model of the multi-objective sequential decision making problem, after which this learned model is used by a multi-objective dynamic programming method to compute Pareto optimal policies. The advantage of this model-based multi-objective reinforcement learning method is that once an accurate model has been estimated from the experiences of an agent in some environment, the dynamic programming method will compute all Pareto optimal policies. Therefore it is important that the agent explores the environment in an intelligent way by using a good exploration strategy. In this paper we have supplied the agent with two different exploration strategies and compare their effectiveness in estimating accurate models within a reasonable amount of time. The experimental results show that our method with the best exploration strategy is able to quickly learn all Pareto optimal policies for the Deep Sea Treasure problem.

UR - https://www.scopus.com/pages/publications/84946688279

U2 - 10.1109/ADPRL.2014.7010622

DO - 10.1109/ADPRL.2014.7010622

M3 - Conference contribution

AN - SCOPUS:84946688279

SN - 9781479945535

SP - 1

EP - 6

BT - 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 9-12 December 2014, Orlando, Florida

PB - Institute of Electrical and Electronics Engineers

CY - Piscataway

T2 - 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014)

Y2 - 9 December 2014 through 12 December 2014

ER -

Model-based multi-objective reinforcement learning

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this