Certified policy synthesis for general Markov decision processes: an application in building automation systems

S. Haesaert, N. Cauchi, A. Abate

Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

5 Citaties (Scopus)

Uittreksel

In this paper, we present an industrial application of new approximate similarity relations for Markov models, and show that they are key for the synthesis of control strategies. Typically, modern engineering systems are modelled using complex and high-order models which make the correct-by-design controller construction computationally hard. Using the new approximate similarity relations, this complexity is reduced and we provide certificates on the performance of the synthesised policies. The application deals with stochastic models for the thermal dynamics in a "smart building" setup: such building automation system set-up can be described by discrete-time Markov decision processes evolving over an uncountable state space and endowed with an output quantifying the room temperature. The new similarity relations draw a quantitative connection between different levels of model abstraction, and allow to quantitatively refine over complex models control strategies synthesised on simpler ones. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states and distances between model outputs. We develop a software toolbox supporting the application and the computational implementation of these new relations.

TaalEngels
Pagina's75-103
Aantal pagina's29
TijdschriftPerformance Evaluation
Volume117
DOI's
StatusGepubliceerd - 1 dec 2017

Vingerafdruk

Markov Decision Process
Similarity Relation
Automation
Synthesis
Control Strategy
Set Systems
Output
Uncountable
Systems Engineering
Certificate
Intelligent buildings
Industrial Application
Controller Design
Model
Markov Model
Stochastic Model
Stochastic models
State Space
Discrete-time
Systems engineering

Trefwoorden

    Citeer dit

    @article{649998eb2dff455a9cb6b5195d77e120,
    title = "Certified policy synthesis for general Markov decision processes: an application in building automation systems",
    abstract = "In this paper, we present an industrial application of new approximate similarity relations for Markov models, and show that they are key for the synthesis of control strategies. Typically, modern engineering systems are modelled using complex and high-order models which make the correct-by-design controller construction computationally hard. Using the new approximate similarity relations, this complexity is reduced and we provide certificates on the performance of the synthesised policies. The application deals with stochastic models for the thermal dynamics in a {"}smart building{"} setup: such building automation system set-up can be described by discrete-time Markov decision processes evolving over an uncountable state space and endowed with an output quantifying the room temperature. The new similarity relations draw a quantitative connection between different levels of model abstraction, and allow to quantitatively refine over complex models control strategies synthesised on simpler ones. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states and distances between model outputs. We develop a software toolbox supporting the application and the computational implementation of these new relations.",
    keywords = "Building automation systems, General Markov decision processes, Safety, Synthesis, Temperature control, Verification",
    author = "S. Haesaert and N. Cauchi and A. Abate",
    year = "2017",
    month = "12",
    day = "1",
    doi = "10.1016/j.peva.2017.09.005",
    language = "English",
    volume = "117",
    pages = "75--103",
    journal = "Performance Evaluation",
    issn = "0166-5316",
    publisher = "Elsevier",

    }

    Certified policy synthesis for general Markov decision processes : an application in building automation systems. / Haesaert, S.; Cauchi, N.; Abate, A.

    In: Performance Evaluation, Vol. 117, 01.12.2017, blz. 75-103.

    Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

    TY - JOUR

    T1 - Certified policy synthesis for general Markov decision processes

    T2 - Performance Evaluation

    AU - Haesaert,S.

    AU - Cauchi,N.

    AU - Abate,A.

    PY - 2017/12/1

    Y1 - 2017/12/1

    N2 - In this paper, we present an industrial application of new approximate similarity relations for Markov models, and show that they are key for the synthesis of control strategies. Typically, modern engineering systems are modelled using complex and high-order models which make the correct-by-design controller construction computationally hard. Using the new approximate similarity relations, this complexity is reduced and we provide certificates on the performance of the synthesised policies. The application deals with stochastic models for the thermal dynamics in a "smart building" setup: such building automation system set-up can be described by discrete-time Markov decision processes evolving over an uncountable state space and endowed with an output quantifying the room temperature. The new similarity relations draw a quantitative connection between different levels of model abstraction, and allow to quantitatively refine over complex models control strategies synthesised on simpler ones. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states and distances between model outputs. We develop a software toolbox supporting the application and the computational implementation of these new relations.

    AB - In this paper, we present an industrial application of new approximate similarity relations for Markov models, and show that they are key for the synthesis of control strategies. Typically, modern engineering systems are modelled using complex and high-order models which make the correct-by-design controller construction computationally hard. Using the new approximate similarity relations, this complexity is reduced and we provide certificates on the performance of the synthesised policies. The application deals with stochastic models for the thermal dynamics in a "smart building" setup: such building automation system set-up can be described by discrete-time Markov decision processes evolving over an uncountable state space and endowed with an output quantifying the room temperature. The new similarity relations draw a quantitative connection between different levels of model abstraction, and allow to quantitatively refine over complex models control strategies synthesised on simpler ones. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states and distances between model outputs. We develop a software toolbox supporting the application and the computational implementation of these new relations.

    KW - Building automation systems

    KW - General Markov decision processes

    KW - Safety

    KW - Synthesis

    KW - Temperature control

    KW - Verification

    UR - http://www.scopus.com/inward/record.url?scp=85030849993&partnerID=8YFLogxK

    U2 - 10.1016/j.peva.2017.09.005

    DO - 10.1016/j.peva.2017.09.005

    M3 - Article

    VL - 117

    SP - 75

    EP - 103

    JO - Performance Evaluation

    JF - Performance Evaluation

    SN - 0166-5316

    ER -