SLE 2018
Sun 4 - Fri 9 November 2018 Boston, Massachusetts, United States
co-located with SPLASH 2018
Mon 5 Nov 2018 11:30 - 12:00 at Studio 1 - Parsing Chair(s): Tijs van der Storm

The POSIX shell language defies conventional wisdom of compiler construction on several levels: The shell language was not designed for static parsing, but with an intertwining of syntactic analysis and execution by expansion in mind. Token recognition cannot be specified by regular expressions, lexical analysis depends on the parsing context and the evaluation context, and the shell grammar given in the specification is ambiguous. Besides, the unorthodox design choices of the shell language fit badly in the usual specification languages used to describe other programming languages. This makes the standard usage of Lex and Yacc as a pipeline inadequate for the implementation of a parser for POSIX shell.

The existing implementations of shell parsers are complex and use low-level character-level parsing code which is difficult to relate to the POSIX specification. We find it hard to trust such parsers, for instance when using them for writing automatic verification tools for shell scripts.

This paper offers an overview of the technical difficulties related to the syntactic analysis of the POSIX shell language. It also describes how we have resolved these difficulties using advanced parsing techniques (namely speculative parsing, parser state introspection, context-dependent lexical analysis and longest-prefix parsing) while keeping the implementation at a sufficiently high level of abstraction so that experts can check that the POSIX standard is respected.

The resulting tool, called Morbig, is an open-source static parser for a well-defined and realistic subset of the POSIX shell language. Its implementation crucially relies on the purity and incrementality of LR(1) parsers generated by Menhir.

Mon 5 Nov
Times are displayed in time zone: (GMT-05:00) Guadalajara, Mexico City, Monterrey change

10:30 - 12:00: SLE 2018 - Parsing at Studio 1
Chair(s): Tijs van der StormCWI & University of Groningen
papers10:30 - 11:00
Luis Eduardo de Souza AmorimDelft University of Technology, Netherlands, Michael J. SteindorferDelft University of Technology, Sebastian ErdwegTU Delft, Eelco VisserDelft University of Technology
Link to publication DOI
papers11:00 - 11:30
L. Thomas van BinsbergenRoyal Holloway University of London, Elizabeth ScottRoyal Holloway University of London, Adrian Johnstone
File Attached
papers11:30 - 12:00
Yann Régis-GianasIRIF, University Paris Diderot and CNRS, France / INRIA PI.R2, Nicolas JeannerodIRIF, University Paris Diderot and CNRS, France, Ralf TreinenIRIF