Abstract
Multi-objective multi-armed bandits (MOMAB) are multi-armed bandits (MAB) extended to reward vectors. We use the Pareto dominance relation to assess the quality of reward vectors, as opposite to scalarization functions. In this paper, we study the exploration vs exploitation trade-off in infinite horizon MOMABs algorithms. Single objective MABs explore the suboptimal arms and exploit a single optimal arm. MOMABs explore the suboptimal arms, but they also need to exploit fairly all optimal arms. We study the exploration vs exploitation trade-off of the Pareto UCB1 algorithm. We extend UCB2 that is another popular infinite horizon MAB algorithm to rewards vectors using the Pareto dominance relation. We analyse the properties of the proposed MOMAB algorithms in terms of upper regret bounds. We experimentally compare the exploration vs exploitation trade-off of the proposed MOMAB algorithms on a bi-objective Bernoulli environment coming from control theory.
Original language | English |
---|---|
Title of host publication | Proceedings of the International Conference on Agents and Artificial Intelligence : Lisbon, Portugal, 10-12 January 2015 |
Place of Publication | s.l. |
Publisher | SciTePress Digital Library |
Pages | 66-77 |
Number of pages | 12 |
Volume | 2 |
ISBN (Print) | 9789897580741 |
Publication status | Published - 2015 |
Externally published | Yes |
Event | 7th International Conference on Agents and Artificial Intelligence (ICAART 2015) - Lisbon, Portugal Duration: 10 Jan 2015 → 12 Jan 2015 Conference number: 7 http://www.icaart.org/?y=2015 |
Conference
Conference | 7th International Conference on Agents and Artificial Intelligence (ICAART 2015) |
---|---|
Abbreviated title | ICAART 2015 |
Country/Territory | Portugal |
City | Lisbon |
Period | 10/01/15 → 12/01/15 |
Other | Conference held in conjunction with the 4th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2015) and the 4th International Conference on Operations Research and Enterprise Systems (ICORES 2015) |
Internet address |
Keywords
- Infinite horizon policies
- Multi-armed bandits
- Multi-objective optimisation
- Pareto dominance relation