Temporal logic control of POMDPs via label-based stochastic simulation relations

S. Haesaert, P. Nilsson, C. I. Vasile, R. Thakker, A. Agha-mohammadi, A. D. Ames, R. M. Murray

Research output: Contribution to journalArticleAcademicpeer-review

3 Citations (Scopus)
2 Downloads (Pure)

Abstract

The synthesis of controllers guaranteeing linear temporal logic specifications on partially observable Markov decision processes (POMDP) via their belief models causes computational issues due to the continuous spaces. In this work, we construct a finite-state abstraction on which a control policy is synthesized and refined back to the original belief model. We introduce a new notion of label-based approximate stochastic simulation to quantify the deviation between belief models. We develop a robust synthesis methodology that yields a lower bound on the satisfaction probability, by compensating for deviations a priori, and that utilizes a less conservative control refinement.

Original languageEnglish
Pages (from-to)271-276
Number of pages6
JournalIFAC-PapersOnLine
Volume51
Issue number16
DOIs
Publication statusPublished - 1 Jan 2018

Fingerprint

Temporal logic
Labels
Specifications
Controllers

Keywords

  • control synthesis
  • Markov decision processes
  • partially observable
  • Temporal properties

Cite this

Haesaert, S., Nilsson, P., Vasile, C. I., Thakker, R., Agha-mohammadi, A., Ames, A. D., & Murray, R. M. (2018). Temporal logic control of POMDPs via label-based stochastic simulation relations. IFAC-PapersOnLine, 51(16), 271-276. https://doi.org/10.1016/j.ifacol.2018.08.046
Haesaert, S. ; Nilsson, P. ; Vasile, C. I. ; Thakker, R. ; Agha-mohammadi, A. ; Ames, A. D. ; Murray, R. M. / Temporal logic control of POMDPs via label-based stochastic simulation relations. In: IFAC-PapersOnLine. 2018 ; Vol. 51, No. 16. pp. 271-276.
@article{528d3c9e1cbf4e309da171217e9649e6,
title = "Temporal logic control of POMDPs via label-based stochastic simulation relations",
abstract = "The synthesis of controllers guaranteeing linear temporal logic specifications on partially observable Markov decision processes (POMDP) via their belief models causes computational issues due to the continuous spaces. In this work, we construct a finite-state abstraction on which a control policy is synthesized and refined back to the original belief model. We introduce a new notion of label-based approximate stochastic simulation to quantify the deviation between belief models. We develop a robust synthesis methodology that yields a lower bound on the satisfaction probability, by compensating for deviations a priori, and that utilizes a less conservative control refinement.",
keywords = "control synthesis, Markov decision processes, partially observable, Temporal properties",
author = "S. Haesaert and P. Nilsson and Vasile, {C. I.} and R. Thakker and A. Agha-mohammadi and Ames, {A. D.} and Murray, {R. M.}",
year = "2018",
month = "1",
day = "1",
doi = "10.1016/j.ifacol.2018.08.046",
language = "English",
volume = "51",
pages = "271--276",
journal = "IFAC-PapersOnLine",
issn = "2405-8963",
publisher = "Elsevier",
number = "16",

}

Haesaert, S, Nilsson, P, Vasile, CI, Thakker, R, Agha-mohammadi, A, Ames, AD & Murray, RM 2018, 'Temporal logic control of POMDPs via label-based stochastic simulation relations', IFAC-PapersOnLine, vol. 51, no. 16, pp. 271-276. https://doi.org/10.1016/j.ifacol.2018.08.046

Temporal logic control of POMDPs via label-based stochastic simulation relations. / Haesaert, S.; Nilsson, P.; Vasile, C. I.; Thakker, R.; Agha-mohammadi, A.; Ames, A. D.; Murray, R. M.

In: IFAC-PapersOnLine, Vol. 51, No. 16, 01.01.2018, p. 271-276.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Temporal logic control of POMDPs via label-based stochastic simulation relations

AU - Haesaert, S.

AU - Nilsson, P.

AU - Vasile, C. I.

AU - Thakker, R.

AU - Agha-mohammadi, A.

AU - Ames, A. D.

AU - Murray, R. M.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The synthesis of controllers guaranteeing linear temporal logic specifications on partially observable Markov decision processes (POMDP) via their belief models causes computational issues due to the continuous spaces. In this work, we construct a finite-state abstraction on which a control policy is synthesized and refined back to the original belief model. We introduce a new notion of label-based approximate stochastic simulation to quantify the deviation between belief models. We develop a robust synthesis methodology that yields a lower bound on the satisfaction probability, by compensating for deviations a priori, and that utilizes a less conservative control refinement.

AB - The synthesis of controllers guaranteeing linear temporal logic specifications on partially observable Markov decision processes (POMDP) via their belief models causes computational issues due to the continuous spaces. In this work, we construct a finite-state abstraction on which a control policy is synthesized and refined back to the original belief model. We introduce a new notion of label-based approximate stochastic simulation to quantify the deviation between belief models. We develop a robust synthesis methodology that yields a lower bound on the satisfaction probability, by compensating for deviations a priori, and that utilizes a less conservative control refinement.

KW - control synthesis

KW - Markov decision processes

KW - partially observable

KW - Temporal properties

UR - http://www.scopus.com/inward/record.url?scp=85052662704&partnerID=8YFLogxK

U2 - 10.1016/j.ifacol.2018.08.046

DO - 10.1016/j.ifacol.2018.08.046

M3 - Article

AN - SCOPUS:85052662704

VL - 51

SP - 271

EP - 276

JO - IFAC-PapersOnLine

JF - IFAC-PapersOnLine

SN - 2405-8963

IS - 16

ER -