ETAPS 2019
Sat 6 - Thu 11 April 2019 Prague, Czech Republic
Tue 9 Apr 2019 14:00 - 14:30 at JUPITER - Machine Learning Chair(s): Bernhard Steffen

We provide the first solution for model-free reinforcement learning of omega-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of omega-regular objectives to an almost-sure reachability problem and extend this technique to learning how to control an unknown model so that the chance of satisfying the objective is maximized. A key feature of our technique is the compilation of omega-regular properties into limit-deterministic Buechi automata instead of the traditional Rabin automata; this choice sidesteps difficulties that have marred previous proposals. Our approach allows us to apply model-free, off-the-shelf reinforcement learning algorithms to compute optimal strategies from the observations of the MDP. We present an experimental evaluation of our technique on benchmark learning problems.

Tue 9 Apr

14:00 - 15:00: TACAS 2019 - Machine Learning at JUPITER
Chair(s): Bernhard SteffenTechnical University Dortmund
tacas-2019-papers14:00 - 14:30
Ernst Moritz HahnQueen's University Belfast, Mateo Perez, Sven ScheweUniversity of Liverpool, Fabio Somenzi, Ashutosh Trivedi, Dominik Wojtczak
Link to publication
tacas-2019-papers14:30 - 15:00
Nathan FultonMIT-IBM Watson AI Lab, André PlatzerCarnegie Mellon University
Link to publication