Verification of general Markov decision processes by approximate similarity relations and policy refinement

S. Haesaert, S. Esmaeil, S. Soudjani, A. Abate

Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademic

5 Citaties (Scopus)

Uittreksel

In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed with uncountably-infinite state spaces and metric output (or observation) spaces. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states, and distances between model outputs. We show that the new probabilistic similarity relations, inspired by a notion of simulation developed for finite-state models, can be effectively employed over general Markov decision processes for verification purposes, and specifically for control refinement from abstract models.
TaalEngels
Artikelnummer1605.09557v1
Pagina's1-36
TijdschriftarXiv.org, e-Print Archive, Physics
StatusGepubliceerd - 31 mei 2016

Vingerafdruk

Probability distributions

Trefwoorden

  • cs.SY

Citeer dit

@article{c239fae236ed4c2ca3967c05787e80d1,
title = "Verification of general Markov decision processes by approximate similarity relations and policy refinement",
abstract = "In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed with uncountably-infinite state spaces and metric output (or observation) spaces. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states, and distances between model outputs. We show that the new probabilistic similarity relations, inspired by a notion of simulation developed for finite-state models, can be effectively employed over general Markov decision processes for verification purposes, and specifically for control refinement from abstract models.",
keywords = "cs.SY",
author = "S. Haesaert and S. Esmaeil and S. Soudjani and A. Abate",
year = "2016",
month = "5",
day = "31",
language = "English",
pages = "1--36",
journal = "arXiv.org, e-Print Archive, Physics",

}

Verification of general Markov decision processes by approximate similarity relations and policy refinement. / Haesaert, S.; Esmaeil, S.; Soudjani, S. ; Abate, A.

In: arXiv.org, e-Print Archive, Physics, 31.05.2016, blz. 1-36.

Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademic

TY - JOUR

T1 - Verification of general Markov decision processes by approximate similarity relations and policy refinement

AU - Haesaert,S.

AU - Esmaeil,S.

AU - Soudjani,S.

AU - Abate,A.

PY - 2016/5/31

Y1 - 2016/5/31

N2 - In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed with uncountably-infinite state spaces and metric output (or observation) spaces. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states, and distances between model outputs. We show that the new probabilistic similarity relations, inspired by a notion of simulation developed for finite-state models, can be effectively employed over general Markov decision processes for verification purposes, and specifically for control refinement from abstract models.

AB - In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed with uncountably-infinite state spaces and metric output (or observation) spaces. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states, and distances between model outputs. We show that the new probabilistic similarity relations, inspired by a notion of simulation developed for finite-state models, can be effectively employed over general Markov decision processes for verification purposes, and specifically for control refinement from abstract models.

KW - cs.SY

M3 - Article

SP - 1

EP - 36

JO - arXiv.org, e-Print Archive, Physics

T2 - arXiv.org, e-Print Archive, Physics

JF - arXiv.org, e-Print Archive, Physics

M1 - 1605.09557v1

ER -