Abstract
In traditional A/B testing, we have two variants of the same product, a pool of test subjects, and a measure of success. In a randomized experiment, each test subject is presented with one of the two variants, and the measure of success is aggregated per variant. The variant of the product associated with the most success is retained, while the other variant is discarded. This, however, presumes that the company producing the products only has enough capacity to maintain one of the two product variants. If more capacity is available, then advanced data science techniques can extract more profit for the company from the A/B testing results. Exceptional Model Mining is one such advanced data science technique, which specializes in identifying subgroups that behave differently from the overall population. Using the association model class for EMM, we can find subpopulations that prefer variant A where the general population prefers variant B, and vice versa. This data science technique is applied on data from StudyPortals, a global study choice platform that ran an A/B test on the design of aspects of their website.
| Original language | English |
|---|---|
| Title of host publication | Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings |
| Editors | Michelangelo Ceci, Saso Dzeroski, Donato Malerba, Yasemin Altun, Kamalika Das, Jesse Read, Marinka Zitnik, Jerzy Stefanowski, Taneli Mielikäinen |
| Place of Publication | Cham |
| Publisher | Springer |
| Pages | 114-126 |
| Number of pages | 13 |
| ISBN (Electronic) | 978-3-319-71273-4 |
| ISBN (Print) | 978-3-319-71272-7 |
| DOIs | |
| Publication status | Published - 30 Dec 2017 |
| Event | European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017 - Skopje, Macedonia, The Former Yugoslav Republic of Duration: 18 Sep 2017 → 22 Sep 2017 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 10536 LNAI |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017 |
|---|---|
| Country | Macedonia, The Former Yugoslav Republic of |
| City | Skopje |
| Period | 18/09/17 → 22/09/17 |
Fingerprint
Keywords
- A/B testing
- Association
- E-commerce
- Exceptional Model Mining
- Online controlled experiments
- Website optimization
Cite this
}
Have it both ways : from A/B testing to A&B testing with exceptional model mining. / Duivesteijn, Wouter; Farzami, Tara; Putman, Thijs; Peer, Evertjan; Weerts, Hilde J.P.; Adegeest, Jasper N.; Foks, Gerson; Pechenizkiy, Mykola.
Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings. ed. / Michelangelo Ceci; Saso Dzeroski; Donato Malerba; Yasemin Altun; Kamalika Das; Jesse Read; Marinka Zitnik; Jerzy Stefanowski; Taneli Mielikäinen. Cham : Springer, 2017. p. 114-126 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10536 LNAI).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review
TY - GEN
T1 - Have it both ways : from A/B testing to A&B testing with exceptional model mining
AU - Duivesteijn, Wouter
AU - Farzami, Tara
AU - Putman, Thijs
AU - Peer, Evertjan
AU - Weerts, Hilde J.P.
AU - Adegeest, Jasper N.
AU - Foks, Gerson
AU - Pechenizkiy, Mykola
PY - 2017/12/30
Y1 - 2017/12/30
N2 - In traditional A/B testing, we have two variants of the same product, a pool of test subjects, and a measure of success. In a randomized experiment, each test subject is presented with one of the two variants, and the measure of success is aggregated per variant. The variant of the product associated with the most success is retained, while the other variant is discarded. This, however, presumes that the company producing the products only has enough capacity to maintain one of the two product variants. If more capacity is available, then advanced data science techniques can extract more profit for the company from the A/B testing results. Exceptional Model Mining is one such advanced data science technique, which specializes in identifying subgroups that behave differently from the overall population. Using the association model class for EMM, we can find subpopulations that prefer variant A where the general population prefers variant B, and vice versa. This data science technique is applied on data from StudyPortals, a global study choice platform that ran an A/B test on the design of aspects of their website.
AB - In traditional A/B testing, we have two variants of the same product, a pool of test subjects, and a measure of success. In a randomized experiment, each test subject is presented with one of the two variants, and the measure of success is aggregated per variant. The variant of the product associated with the most success is retained, while the other variant is discarded. This, however, presumes that the company producing the products only has enough capacity to maintain one of the two product variants. If more capacity is available, then advanced data science techniques can extract more profit for the company from the A/B testing results. Exceptional Model Mining is one such advanced data science technique, which specializes in identifying subgroups that behave differently from the overall population. Using the association model class for EMM, we can find subpopulations that prefer variant A where the general population prefers variant B, and vice versa. This data science technique is applied on data from StudyPortals, a global study choice platform that ran an A/B test on the design of aspects of their website.
KW - A/B testing
KW - Association
KW - E-commerce
KW - Exceptional Model Mining
KW - Online controlled experiments
KW - Website optimization
UR - http://www.scopus.com/inward/record.url?scp=85040256149&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-71273-4_10
DO - 10.1007/978-3-319-71273-4_10
M3 - Conference contribution
AN - SCOPUS:85040256149
SN - 978-3-319-71272-7
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 114
EP - 126
BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings
A2 - Ceci, Michelangelo
A2 - Dzeroski, Saso
A2 - Malerba, Donato
A2 - Altun, Yasemin
A2 - Das, Kamalika
A2 - Read, Jesse
A2 - Zitnik, Marinka
A2 - Stefanowski, Jerzy
A2 - Mielikäinen, Taneli
PB - Springer
CY - Cham
ER -