A factor graph approach to automated design of Bayesian signal processing algorithms

Marco Cox (Corresponding author), Thijs van de Laar, Bert de Vries

Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

2 Citaties (Scopus)

Uittreksel

The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed “ForneyLab”2 as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools “Edward” and “Stan”. In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.

TaalEngels
Pagina's185-204
Aantal pagina's20
TijdschriftInternational Journal of Approximate Reasoning
Volume104
DOI's
StatusGepubliceerd - 1 jan 2019

Vingerafdruk

Factor Graph
Signal Processing
Signal processing
Message passing
Message Passing
Probabilistic Programming
Differentiation (calculus)
Automatic Differentiation
Model Comparison
Model Specification
State-space Model
Bayesian inference
Parameter estimation
Monte Carlo method
Learning systems
Parameter Estimation
Machine Learning
Monte Carlo methods
Paradigm
Specifications

Bibliografische nota

Accepted for publication in the International Journal of Approximate Reasoning

Trefwoorden

    Citeer dit

    @article{29135ae69bdb4c78bdfc9c24e98700fc,
    title = "A factor graph approach to automated design of Bayesian signal processing algorithms",
    abstract = "The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed “ForneyLab”2 as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools “Edward” and “Stan”. In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.",
    keywords = "Bayesian inference, Factor graphs, Julia, Message passing, Probabilistic programming",
    author = "Marco Cox and {van de Laar}, Thijs and {de Vries}, Bert",
    note = "Accepted for publication in the International Journal of Approximate Reasoning",
    year = "2019",
    month = "1",
    day = "1",
    doi = "10.1016/j.ijar.2018.11.002",
    language = "English",
    volume = "104",
    pages = "185--204",
    journal = "International Journal of Approximate Reasoning",
    issn = "0888-613X",
    publisher = "Elsevier",

    }

    A factor graph approach to automated design of Bayesian signal processing algorithms. / Cox, Marco (Corresponding author); van de Laar, Thijs; de Vries, Bert.

    In: International Journal of Approximate Reasoning, Vol. 104, 01.01.2019, blz. 185-204.

    Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

    TY - JOUR

    T1 - A factor graph approach to automated design of Bayesian signal processing algorithms

    AU - Cox,Marco

    AU - van de Laar,Thijs

    AU - de Vries,Bert

    N1 - Accepted for publication in the International Journal of Approximate Reasoning

    PY - 2019/1/1

    Y1 - 2019/1/1

    N2 - The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed “ForneyLab”2 as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools “Edward” and “Stan”. In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.

    AB - The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed “ForneyLab”2 as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools “Edward” and “Stan”. In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.

    KW - Bayesian inference

    KW - Factor graphs

    KW - Julia

    KW - Message passing

    KW - Probabilistic programming

    UR - http://www.scopus.com/inward/record.url?scp=85056602296&partnerID=8YFLogxK

    U2 - 10.1016/j.ijar.2018.11.002

    DO - 10.1016/j.ijar.2018.11.002

    M3 - Article

    VL - 104

    SP - 185

    EP - 204

    JO - International Journal of Approximate Reasoning

    T2 - International Journal of Approximate Reasoning

    JF - International Journal of Approximate Reasoning

    SN - 0888-613X

    ER -