Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off

M.M. Drugan

doi:10.1007/978-3-319-27947-3_7

Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off

M.M. Drugan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Downloads (Pure)

Abstract

We focus on the effect of the exploration/exploitation tradeoff strategies on the algorithmic design off multi-armed bandits (MAB) with reward vectors. Pareto dominance relation assesses the quality of reward vectors in infinite horizon MABs, like the UCB1 and UCB2 algorithms. In single objective MABs, there is a trade-off between the exploration of the suboptimal arms, and exploitation of a single optimal arm. Pareto dominance based MABs fairly exploit all Pareto optimal arms, and explore suboptimal arms. We study the exploration vs exploitation trade-off for two UCB like algorithms for reward vectors. We analyse the properties of the proposed MAB algorithms in terms of upper regret bounds and we experimentally compare their exploration vs exploitation trade-off on a bi-objective Bernoulli environment coming from control theory.

Original language	English
Title of host publication	Agents and Artificial Intelligence
Subtitle of host publication	7th International Conference, ICAART 2015, Lisbon, Portugal, January 10-12, 2015, Revised Selected Papers
Editors	B. Duval, J. van den Herik, St. Loiseau, J. Filipe
Place of Publication	Berlin
Publisher	Springer
Pages	128-144
Number of pages	17
ISBN (Electronic)	978-3-319-27947-3
ISBN (Print)	9783319279466
DOIs	https://doi.org/10.1007/978-3-319-27947-3_7
Publication status	Published - 2015
Externally published	Yes
Event	7th International Conference on Agents and Artificial Intelligence (ICAART 2015) - Lisbon, Portugal Duration: 10 Jan 2015 → 12 Jan 2015 Conference number: 7 http://www.icaart.org/?y=2015

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9494
ISSN (Print)	03029743
ISSN (Electronic)	16113349

Conference

Conference	7th International Conference on Agents and Artificial Intelligence (ICAART 2015)
Abbreviated title	ICAART 2015
Country/Territory	Portugal
City	Lisbon
Period	10/01/15 → 12/01/15
Other	Conference held in conjunction with the 4th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2015) and the 4th International Conference on Operations Research and Enterprise Systems (ICORES 2015)
Internet address	http://www.icaart.org/?y=2015

Keywords

Infinite horizon policies
Multi-armed bandits
Multi-objective optimisation
Pareto dominance relation

Access to Document

10.1007/978-3-319-27947-3_7

Cite this

Drugan, M. M. (2015). Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off. In B. Duval, J. van den Herik, S. Loiseau, & J. Filipe (Eds.), Agents and Artificial Intelligence : 7th International Conference, ICAART 2015, Lisbon, Portugal, January 10-12, 2015, Revised Selected Papers (pp. 128-144). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9494). Springer. https://doi.org/10.1007/978-3-319-27947-3_7

Drugan, M.M. / Infinite horizon multi-armed bandits with reward vectors : exploration/exploitation trade-off. Agents and Artificial Intelligence : 7th International Conference, ICAART 2015, Lisbon, Portugal, January 10-12, 2015, Revised Selected Papers. editor / B. Duval ; J. van den Herik ; St. Loiseau ; J. Filipe. Berlin : Springer, 2015. pp. 128-144 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0ab7c0dc389949fc9f6abde84ba0208b,

title = "Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off",

abstract = "We focus on the effect of the exploration/exploitation tradeoff strategies on the algorithmic design off multi-armed bandits (MAB) with reward vectors. Pareto dominance relation assesses the quality of reward vectors in infinite horizon MABs, like the UCB1 and UCB2 algorithms. In single objective MABs, there is a trade-off between the exploration of the suboptimal arms, and exploitation of a single optimal arm. Pareto dominance based MABs fairly exploit all Pareto optimal arms, and explore suboptimal arms. We study the exploration vs exploitation trade-off for two UCB like algorithms for reward vectors. We analyse the properties of the proposed MAB algorithms in terms of upper regret bounds and we experimentally compare their exploration vs exploitation trade-off on a bi-objective Bernoulli environment coming from control theory.",

keywords = "Infinite horizon policies, Multi-armed bandits, Multi-objective optimisation, Pareto dominance relation",

author = "M.M. Drugan",

year = "2015",

doi = "10.1007/978-3-319-27947-3\_7",

language = "English",

isbn = "9783319279466",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "128--144",

editor = "B. Duval and \{van den Herik\}, J. and Loiseau, \{St. \} and J. Filipe",

booktitle = "Agents and Artificial Intelligence",

address = "Germany",

note = "7th International Conference on Agents and Artificial Intelligence (ICAART 2015), ICAART 2015 ; Conference date: 10-01-2015 Through 12-01-2015",

url = "http://www.icaart.org/?y=2015",

}

Drugan, MM 2015, Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off. in B Duval, J van den Herik, S Loiseau & J Filipe (eds), Agents and Artificial Intelligence : 7th International Conference, ICAART 2015, Lisbon, Portugal, January 10-12, 2015, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9494, Springer, Berlin, pp. 128-144, 7th International Conference on Agents and Artificial Intelligence (ICAART 2015), Lisbon, Portugal, 10/01/15. https://doi.org/10.1007/978-3-319-27947-3_7

Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off. / Drugan, M.M.
Agents and Artificial Intelligence : 7th International Conference, ICAART 2015, Lisbon, Portugal, January 10-12, 2015, Revised Selected Papers. ed. / B. Duval; J. van den Herik; St. Loiseau; J. Filipe. Berlin: Springer, 2015. p. 128-144 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9494).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Infinite horizon multi-armed bandits with reward vectors

T2 - 7th International Conference on Agents and Artificial Intelligence (ICAART 2015)

AU - Drugan, M.M.

N1 - Conference code: 7

PY - 2015

Y1 - 2015

N2 - We focus on the effect of the exploration/exploitation tradeoff strategies on the algorithmic design off multi-armed bandits (MAB) with reward vectors. Pareto dominance relation assesses the quality of reward vectors in infinite horizon MABs, like the UCB1 and UCB2 algorithms. In single objective MABs, there is a trade-off between the exploration of the suboptimal arms, and exploitation of a single optimal arm. Pareto dominance based MABs fairly exploit all Pareto optimal arms, and explore suboptimal arms. We study the exploration vs exploitation trade-off for two UCB like algorithms for reward vectors. We analyse the properties of the proposed MAB algorithms in terms of upper regret bounds and we experimentally compare their exploration vs exploitation trade-off on a bi-objective Bernoulli environment coming from control theory.

AB - We focus on the effect of the exploration/exploitation tradeoff strategies on the algorithmic design off multi-armed bandits (MAB) with reward vectors. Pareto dominance relation assesses the quality of reward vectors in infinite horizon MABs, like the UCB1 and UCB2 algorithms. In single objective MABs, there is a trade-off between the exploration of the suboptimal arms, and exploitation of a single optimal arm. Pareto dominance based MABs fairly exploit all Pareto optimal arms, and explore suboptimal arms. We study the exploration vs exploitation trade-off for two UCB like algorithms for reward vectors. We analyse the properties of the proposed MAB algorithms in terms of upper regret bounds and we experimentally compare their exploration vs exploitation trade-off on a bi-objective Bernoulli environment coming from control theory.

KW - Infinite horizon policies

KW - Multi-armed bandits

KW - Multi-objective optimisation

KW - Pareto dominance relation

UR - https://www.scopus.com/pages/publications/84952802546

U2 - 10.1007/978-3-319-27947-3_7

DO - 10.1007/978-3-319-27947-3_7

M3 - Conference contribution

AN - SCOPUS:84952802546

SN - 9783319279466

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 128

EP - 144

BT - Agents and Artificial Intelligence

A2 - Duval, B.

A2 - van den Herik, J.

A2 - Loiseau, St.

A2 - Filipe, J.

PB - Springer

CY - Berlin

Y2 - 10 January 2015 through 12 January 2015

ER -

Drugan MM. Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off. In Duval B, van den Herik J, Loiseau S, Filipe J, editors, Agents and Artificial Intelligence : 7th International Conference, ICAART 2015, Lisbon, Portugal, January 10-12, 2015, Revised Selected Papers. Berlin: Springer. 2015. p. 128-144. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-27947-3_7

Infinite horizon multi-armed bandits with reward vectors: exploration/exploitation trade-off

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this