Reinforcement learning method for ad networks ordering in real-time bidding

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)

Abstract

High turnover of online advertising and especially real time bidding makes this ad market very attractive to beneficiary stakeholders. For publishers, it is as easy as placing some slots in their webpages and sell these slots in the available online auctions. It is important to determine which online auction market to send their slots to. Based on the traditional Waterfall Strategy, publishers have a fixed ordering of preferred online auction markets, and sell the ad slots by trying these markets sequentially. This fixed-order strategy replies heavily on the experience of publishers, and often it does not provide highest revenue. In this paper, we propose a method for dynamically deciding on the ordering of auction markets for each available ad slot. This method is based on reinforcement learning (RL) and learns the state-action through a tabular method. Since the state-action space is sparse, a prediction model is used to solve this sparsity. We analyze a real-time bidding dataset, and then show that the proposed RL method on this dataset leads to higher revenues. In addition, a sensitivity analysis is performed on the parameters of the method.
Original languageEnglish
Title of host publicationAgents and Artificial Intelligence
Subtitle of host publication11th International Conference, ICAART 2019, Prague, Czech Republic, February 19–21, 2019, Revised Selected Papers
EditorsJaap van den Herik, Ana Paula Rocha, Luc Steels
Place of PublicationBerlin
PublisherSpringer
Pages16-36
Number of pages21
ISBN (Electronic)978-3-030-37494-5
ISBN (Print)978-3-030-37493-8
DOIs
Publication statusPublished - 2019
Event11th International Conference on Agents and Artificial Intelligence, ICAART 2019 - Prague, Czech Republic
Duration: 19 Feb 201921 Feb 2019
http://www.icaart.org/

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume11978
NameLecture Notes in Artificial Intelligence
Volume11978

Conference

Conference11th International Conference on Agents and Artificial Intelligence, ICAART 2019
Abbreviated titleICAART2019
Country/TerritoryCzech Republic
CityPrague
Period19/02/1921/02/19
Internet address

Keywords

  • Real time bidding
  • Reinforcement learning
  • Waterfall strategy

Fingerprint

Dive into the research topics of 'Reinforcement learning method for ad networks ordering in real-time bidding'. Together they form a unique fingerprint.

Cite this