Robust importance-weighted cross-validation under sample selection bias

Wouter M. Kouw; Jesse H. Krijthe; Marco Loog

doi:10.1109/MLSP.2019.8918731

Robust importance-weighted cross-validation under sample selection bias

Wouter M. Kouw, Jesse H. Krijthe, Marco Loog

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

4 Citaten (Scopus)

74 Downloads (Pure)

Samenvatting

Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.

Originele taal-2	Engels
Titel	2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019
Plaats van productie	Piscataway
Uitgeverij	Institute of Electrical and Electronics Engineers
Aantal pagina's	6
ISBN van elektronische versie	978-1-7281-0824-7
DOI's	https://doi.org/10.1109/MLSP.2019.8918731
Status	Gepubliceerd - 5 dec. 2019
Evenement	29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019 - University of Pittsburgh, Pittsburgh, Verenigde Staten van Amerika Duur: 13 okt. 2019 → 16 dec. 2019 Congresnummer: 29 https://www.ieeemlsp.cc/

Congres

Congres	29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019
Verkorte titel	MLSP 2019
Land/Regio	Verenigde Staten van Amerika
Stad	Pittsburgh
Periode	13/10/19 → 16/12/19
Internet adres	https://www.ieeemlsp.cc/

Toegang tot document

10.1109/MLSP.2019.8918731

arXiv versionFinale auteursversie , 330 KB

Andere bestanden en links

Link to publication in Scopus

Citeer dit

@inproceedings{b5afd26c16514878bb3ee489626681dc,

title = "Robust importance-weighted cross-validation under sample selection bias",

abstract = "Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.",

keywords = "Sample selection bias, cross-validation",

author = "Kouw, {Wouter M.} and Krijthe, {Jesse H.} and Marco Loog",

year = "2019",

month = dec,

day = "5",

doi = "10.1109/MLSP.2019.8918731",

language = "English",

booktitle = "2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019, MLSP 2019 ; Conference date: 13-10-2019 Through 16-12-2019",

url = "https://www.ieeemlsp.cc/",

}

Kouw, WM, Krijthe, JH & Loog, M 2019, Robust importance-weighted cross-validation under sample selection bias. in 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019., 8918731, Institute of Electrical and Electronics Engineers, Piscataway, 29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019, Pittsburgh, Pennsylvania, Verenigde Staten van Amerika, 13/10/19. https://doi.org/10.1109/MLSP.2019.8918731

Robust importance-weighted cross-validation under sample selection bias. / Kouw, Wouter M.; Krijthe, Jesse H.; Loog, Marco.
2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019. Piscataway: Institute of Electrical and Electronics Engineers, 2019. 8918731.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - Robust importance-weighted cross-validation under sample selection bias

AU - Kouw, Wouter M.

AU - Krijthe, Jesse H.

AU - Loog, Marco

N1 - Conference code: 29

PY - 2019/12/5

Y1 - 2019/12/5

N2 - Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.

AB - Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.

KW - Sample selection bias

KW - cross-validation

UR - http://www.scopus.com/inward/record.url?scp=85077676600&partnerID=8YFLogxK

U2 - 10.1109/MLSP.2019.8918731

DO - 10.1109/MLSP.2019.8918731

M3 - Conference contribution

BT - 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019

PB - Institute of Electrical and Electronics Engineers

CY - Piscataway

T2 - 29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019

Y2 - 13 October 2019 through 16 December 2019

ER -

Robust importance-weighted cross-validation under sample selection bias

Samenvatting

Congres

Toegang tot document

Andere bestanden en links

Vingerafdruk

Citeer dit