Sparse nonnegative matrix factorization with ℓ0-constraints

Robert Peharz; Franz Pernkopf

doi:10.1016/j.neucom.2011.09.024

Sparse nonnegative matrix factorization with ℓ0-constraints

Robert Peharz, Franz Pernkopf

Research output: Contribution to journal › Article › Academic › peer-review

134 Citations (Scopus)

Abstract

Although nonnegative matrix factorization (NMF) favors a sparse and part-based representation of nonnegative data, there is no guarantee for this behavior. Several authors proposed NMF methods which enforce sparseness by constraining or penalizing the ℓ ¹-norm of the factor matrices. On the other hand, little work has been done using a more natural sparseness measure, the ℓ ⁰-pseudo-norm. In this paper, we propose a framework for approximate NMF which constrains the ℓ ⁰-norm of the basis matrix, or the coefficient matrix, respectively. For this purpose, techniques for unconstrained NMF can be easily incorporated, such as multiplicative update rules, or the alternating nonnegative least-squares scheme. In experiments we demonstrate the benefits of our methods, which compare to, or outperform existing approaches.

Original language	English
Pages (from-to)	38-46
Number of pages	8
Journal	Neurocomputing
Volume	80
DOIs	https://doi.org/10.1016/j.neucom.2011.09.024
Publication status	Published - 15 Mar 2012
Externally published	Yes

Bibliographical note

Part of special issue:
Special Issue on Machine Learning for Signal Processing 2010
Edited by Jaakko Peltonen, Tapani Raiko, Samuel Kaski

Funding

This work was supported by the Austrian Science Fund (Project number P22488-N23 ). The authors like to thank Sebastian Tschiatschek for his support concerning the Ipopt library, and his useful comments concerning this paper.

Keywords

NMF
Sparse coding
Nonnegative least squares

Access to Document

10.1016/j.neucom.2011.09.024

Cite this

@article{ba735256c26147e7824cb43df93f6e69,

title = "Sparse nonnegative matrix factorization with ℓ0-constraints",

abstract = "Although nonnegative matrix factorization (NMF) favors a sparse and part-based representation of nonnegative data, there is no guarantee for this behavior. Several authors proposed NMF methods which enforce sparseness by constraining or penalizing the ℓ 1-norm of the factor matrices. On the other hand, little work has been done using a more natural sparseness measure, the ℓ 0-pseudo-norm. In this paper, we propose a framework for approximate NMF which constrains the ℓ 0-norm of the basis matrix, or the coefficient matrix, respectively. For this purpose, techniques for unconstrained NMF can be easily incorporated, such as multiplicative update rules, or the alternating nonnegative least-squares scheme. In experiments we demonstrate the benefits of our methods, which compare to, or outperform existing approaches.",

keywords = "NMF, Sparse coding, Nonnegative least squares",

author = "Robert Peharz and Franz Pernkopf",

note = "Part of special issue: Special Issue on Machine Learning for Signal Processing 2010 Edited by Jaakko Peltonen, Tapani Raiko, Samuel Kaski",

year = "2012",

month = mar,

day = "15",

doi = "10.1016/j.neucom.2011.09.024",

language = "English",

volume = "80",

pages = "38--46",

journal = "Neurocomputing",

issn = "0925-2312",

publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Sparse nonnegative matrix factorization with ℓ0-constraints

AU - Peharz, Robert

AU - Pernkopf, Franz

N1 - Part of special issue: Special Issue on Machine Learning for Signal Processing 2010 Edited by Jaakko Peltonen, Tapani Raiko, Samuel Kaski

PY - 2012/3/15

Y1 - 2012/3/15

N2 - Although nonnegative matrix factorization (NMF) favors a sparse and part-based representation of nonnegative data, there is no guarantee for this behavior. Several authors proposed NMF methods which enforce sparseness by constraining or penalizing the ℓ 1-norm of the factor matrices. On the other hand, little work has been done using a more natural sparseness measure, the ℓ 0-pseudo-norm. In this paper, we propose a framework for approximate NMF which constrains the ℓ 0-norm of the basis matrix, or the coefficient matrix, respectively. For this purpose, techniques for unconstrained NMF can be easily incorporated, such as multiplicative update rules, or the alternating nonnegative least-squares scheme. In experiments we demonstrate the benefits of our methods, which compare to, or outperform existing approaches.

AB - Although nonnegative matrix factorization (NMF) favors a sparse and part-based representation of nonnegative data, there is no guarantee for this behavior. Several authors proposed NMF methods which enforce sparseness by constraining or penalizing the ℓ 1-norm of the factor matrices. On the other hand, little work has been done using a more natural sparseness measure, the ℓ 0-pseudo-norm. In this paper, we propose a framework for approximate NMF which constrains the ℓ 0-norm of the basis matrix, or the coefficient matrix, respectively. For this purpose, techniques for unconstrained NMF can be easily incorporated, such as multiplicative update rules, or the alternating nonnegative least-squares scheme. In experiments we demonstrate the benefits of our methods, which compare to, or outperform existing approaches.

KW - NMF

KW - Sparse coding

KW - Nonnegative least squares

UR - http://www.scopus.com/inward/record.url?scp=84855246748&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2011.09.024

DO - 10.1016/j.neucom.2011.09.024

M3 - Article

AN - SCOPUS:84855246748

SN - 0925-2312

VL - 80

SP - 38

EP - 46

JO - Neurocomputing

JF - Neurocomputing

ER -

Sparse nonnegative matrix factorization with ℓ0-constraints

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this