Simple and effective way for data preprocessing selection based on design of experiments

J. Gerretzen, E. Szymańska, J.J. Jansen, J. Bart, H.J. van Manen, E.R. van den Heuvel, L.M.C. Buydens

Research output: Contribution to journalArticleAcademicpeer-review

41 Citations (Scopus)
1 Downloads (Pure)

Abstract

The selection of optimal preprocessing is among the main bottlenecks in chemometric data analysis. Preprocessing currently is a burden, since a multitude of different preprocessing methods is available for, e.g., baseline correction, smoothing, and alignment, but it is not clear beforehand which method(s) should be used for which data set. The process of preprocessing selection is often limited to trial-and-error and is therefore considered somewhat subjective. In this paper, we present a novel, simple, and effective approach for preprocessing selection. The defining feature of this approach is a design of experiments. On the basis of the design, model performance of a few well-chosen preprocessing methods, and combinations thereof (called strategies) is evaluated. Interpretation of the main effects and interactions subsequently enables the selection of an optimal preprocessing strategy. The presented approach is applied to eight different spectroscopic data sets, covering both calibration and classification challenges. We show that the approach is able to select a preprocessing strategy which improves model performance by at least 50% compared to the raw data; in most cases, it leads to a strategy very close to the true optimum. Our approach makes preprocessing selection fast, insightful, and objective. © 2015 American Chemical Society.
Original languageEnglish
Pages (from-to)12096-12103
Number of pages8
JournalAnalytical Chemistry
Volume87
Issue number24
DOIs
Publication statusPublished - 2015

Bibliographical note

Cited By :1

Export Date: 29 June 2016

CODEN: ANCHA

Fingerprint Dive into the research topics of 'Simple and effective way for data preprocessing selection based on design of experiments'. Together they form a unique fingerprint.

  • Cite this

    Gerretzen, J., Szymańska, E., Jansen, J. J., Bart, J., van Manen, H. J., van den Heuvel, E. R., & Buydens, L. M. C. (2015). Simple and effective way for data preprocessing selection based on design of experiments. Analytical Chemistry, 87(24), 12096-12103. https://doi.org/10.1021/acs.analchem.5b02832