Reinforcement learning method for ad networks ordering in real-time bidding

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

3 Citations (Scopus)


High turnover of online advertising and especially real time bidding makes this ad market very attractive to beneficiary stakeholders. For publishers, it is as easy as placing some slots in their webpages and sell these slots in the available online auctions. It is important to determine which online auction market to send their slots to. Based on the traditional Waterfall Strategy, publishers have a fixed ordering of preferred online auction markets, and sell the ad slots by trying these markets sequentially. This fixed-order strategy replies heavily on the experience of publishers, and often it does not provide highest revenue. In this paper, we propose a method for dynamically deciding on the ordering of auction markets for each available ad slot. This method is based on reinforcement learning (RL) and learns the state-action through a tabular method. Since the state-action space is sparse, a prediction model is used to solve this sparsity. We analyze a real-time bidding dataset, and then show that the proposed RL method on this dataset leads to higher revenues. In addition, a sensitivity analysis is performed on the parameters of the method.
Original languageEnglish
Title of host publicationAgents and Artificial Intelligence
Subtitle of host publication11th International Conference, ICAART 2019, Prague, Czech Republic, February 19–21, 2019, Revised Selected Papers
EditorsJaap van den Herik, Ana Paula Rocha, Luc Steels
Place of PublicationBerlin
Number of pages21
ISBN (Electronic)978-3-030-37494-5
ISBN (Print)978-3-030-37493-8
Publication statusPublished - 2019
Event11th International Conference on Agents and Artificial Intelligence, ICAART 2019 - Prague, Czech Republic
Duration: 19 Feb 201921 Feb 2019

Publication series

NameLecture Notes in Computer Science
NameLecture Notes in Artificial Intelligence


Conference11th International Conference on Agents and Artificial Intelligence, ICAART 2019
Abbreviated titleICAART2019
Country/TerritoryCzech Republic
Internet address


  • Real time bidding
  • Reinforcement learning
  • Waterfall strategy


Dive into the research topics of 'Reinforcement learning method for ad networks ordering in real-time bidding'. Together they form a unique fingerprint.

Cite this