Visibly Pushdown Languages

1. Visibly Pushdown Languages Philippe Giabbanelli CMPT 894 – Spring 2008

2. This presentation is about a subclass of context-free languages. First of all, we will see what we use this subclass for, or in other words why we have the need for a definition of a particuliar subclass. Once we define it, we present the usual theoretical tools associated with a class (closures, determinization, characterisation). Why a special subclass? Formal definition of the class and what it can model How to characterize the class (automaton, grammar) Can we have some more general? 1

3. Subset of Context-Free Formalism Characterizations Extensions • How can we describe a non-concurrent programming language like C? sequential computations + nested (sometimes recursive) calls Could be described by a regular language. Involves the stack. An automaton with a stack is a pushdown automaton, and it models context free languages. • Pushdown automata are used in program analysis, compiler optimization and model checking when the question can be modelled as a decision problem over pushdown automata. Regular properties ∙ Dead code (i.e. code that will never get executed) ∙ Accessing unitialized variables Context-free property ∙ Bounds on stack size • Checking context-free properties for pushdown automata is undecidable. 2

4. Subset of Context-Free Formalism Characterizations Extensions So what? It’s undecidable but we still do it? Not exactly. It’s undecidable in the general case, but some things are duable. Why we need to check bounds on stack size: interrupt-driven software • In real time applications, there are external events and we must be fast to answer (i.e. to handle them). It’s mine! It came to me! Let see why bounds on stack size is duable. No! Mine! • Each source of event will emit an interrupt and a handler will take care of it, if you are in the interrupt-processing mode (i.e. if we are ready to take care of external events). Liar! Give it to me! No! Mine! • When we call a handler, we place the return address on the stack so that we can resume normal execution. CRASH • If the interrupt handlers are interrupting each other indefinitively, then we have an unbounded stack (overflow) thus we crash. • Checking context-free properties for pushdown automata is undecidable. 3

5. Subset of Context-Free Formalism Characterizations Extensions • Checking context-free properties for pushdown automata is undecidable. Let see why bounds on stack size is duable. The problems of bounds on stack size are the followings: ∙ Given a program p and a constant K, will the stack never exceed a size of K? (i.e. never grows beyond K). Stack Boundedness Problem. ∙ Given a program p, what is the highest integer K so that the stack will never exceed its size? If there is an execution where the stack can be unbounded, we get K = ∞. Exact maximum stack size problem. Depending on which class of program we look at, we can have polynomial algorithms for those problems. 4

6. Subset of Context-Free Formalism Characterizations Extensions • In general, checking a context-free property on a pushdown automata is undecidable. However, there are cases under some restrictions where we can do it. So, what class of properties can be checked on pushdown models? The whole regular class. A bit of context-free… A class « in between » has been defined: visibly pushdown languages (VPL). In other words, VPL is a subset of deterministic context-free languages. Regular languages are closed under union and intersection. VPL are closed under union and intersection. Context-Free Languages are closed under union but not intersection. 5

7. Subset of Context-Free Formalism Characterizations Extensions • A comparable work has been done on the language of parenthesis: a production of the context-free grammar produces a pair. • If we think of this pair of parentheses from the point of view of modeling a program, it can stand for a function call ( and a return ). • Knuth showed that this language is closed under union and intersection. • This idea of parenthesis was developped by balanced grammars that allow a rule to produce different parantheses, and regular languages. • Both balanced grammars and parenthesis languages are a subset of visibly pushdown languages (VPL). • One difference with VPL is that a function call might not return, hence we allow unmatched paranthesis. 6

8. Subset of Context-Free Formalism Characterizations Extensions Definition of a Visibly Pushdown Automaton • To define it, think of the way C uses the stacks (in low level like MIPS). • If you read a call, you push something on the stack. If you return, you pop the stack. Otherwise, you are neither calling or returning: it’s just a local action that does not need to use the stack. • The stack has a bottom ┴. • We have an alphabet Σc for the calls, Σr for return, Σl for local actions. • The automatons over words of <Σc, Σr, Σl> has: ∙ Set of states Q, initial states Qin and final states QF (q, a, q’) : read a, transfer control to q’ (nothing with the stack) (q, a, q’, γ) : read a, pop γ, transfer control to q’ (q, a, q’, γ) : read a, push γ, transfer control to q’ ∙ Γ the stack alphabet with the special symbol ┴ Q x Σc x Q x (Γ \ ┴) ∙ Transitions of 3 types: Q x Σr x Q x Γ Q x Σl x Q 7

9. Subset of Context-Free Formalism Characterizations Extensions Definition of a Visibly Pushdown Automaton • An execution of a Visibly Pushdown Automaton (VPA) over a sequence of words is called a run. It is represented by (q0, σ0), …, (qk, σk) with the states q and the top of the stack σ. • A word is accepted if there is a run on it leading to a final state. (note that we are just asking to end up on a final state: there might still be things on the stack, and we do not care to have it empty or not) • A language is a visibly pushdown language (VPL) if its words are accepted by a VPA. 8

10. Subset of Context-Free Formalism Characterizations Extensions Modeling formal verification problems • Let say that we have a program where variables have finite types (for example booleans…) with calls of procedures. • We create a VPA where a symbol is associated to all transitions of the program, mapping the calls to Σc, the returns to Σr and otherwise to Σl. ≤ K Pj Pi Property p Property q • P is a generator for the langage L(P). The specification is given by the VPL S over the same alphabet and is satisfied if L(P) is a subset of S. • All regular properties can be verified this way, plus some non-regular: ∙ Partial Correctness. If we had the property p before a call and if we return, then we have q. ∙ Total Correctness. If we had the property p before a call and we will return, then we have q. ∙ Access control. A call is only allowed if we are in a special function (i.e. Pi can be invoked iff Pj is in the current stack). ∙ Interrupt Stack Limit. We have property p if bounded stack. 9

11. Subset of Context-Free Formalism Characterizations Extensions Closure Properties • Let L1 and L2 be visibly pushdown languages on the same alphabet. • For the union, we assume that they are disjoint. We can take the union of states and transitions. • For the intersection, the states are created by product. When one automaton pushes γ1 and the other pushes γ2, then we push (γ1, γ2). • For the renaiming (i.e. changing each word by a function f), we can apply f to all transitions. Thus, a transition on a becomes on f(a). • For the concatenation, we guess the split in a non deterministic way. • For the Kleene star, whenever on a final state, we guess to go back to an initial state and we treat the stack as empty (through renaiming). • A non deterministic automaton can be determinized with O(2^n²) states.

12. Subset of Context-Free Formalism Characterizations Extensions Decision Problems • Inclusion problem: Given two visibly pushdown automata, is the language generated by one a subset of the language generated by the other? → Take the complement of an automaton, its intersection with the other one and check for emptiness. • Universality problem: Given a visibly pushdown automaton M, does it accept all strings? → Let’s take a simple visibly pushdown automaton with one step, accepting all strings. We test if its language is included in the one generated by M. • Those problems were not decidable for Pushdown Automata. Here, we ‘can’ do them but it is EXPTIME-complete. 11

13. Subset of Context-FreeFormalism Characterizations Extensions Regular Tree Languages • We have already met trees when speaking of Context Free Languages. Simply by deriving (i.e. looking at the grammar). Here, we will define formally the trees so that we can create them on a string and determined by the language itself rather than the grammar. 12

14. Subset of Context-FreeFormalism Characterizations Extensions Regular Tree Languages • An Σ-labelled binary tree is given by T = (V, λ) where V is a set of 0 and 1’s identifying a node and λ : V → Σ a labelling function. ε Root of the tree (ε, 0) (ε, 1) 0 The identity of the sons is taken from the one of the father and adding 0 or 1 1 (0, 00) (0, 01) 00 01 • TΣ is the set of all possible Σ-labeled trees. • We will define a function η that, given a word, produces a tree. 13

15. Subset of Context-FreeFormalism Characterizations Extensions Regular Tree Languages • An Σ-labelled binary tree is given by T = (V, λ) where V is a set of 0 and 1’s identifying a node and λ : V → Σ a labelling function. ∙ For the empty word, we produce the empty tree. ∙ If the word begins by a call: Otherwise, let the word be cw’. If there is a matching return, let the word be cw1rw2. c c η(w’) η(w1) η(rw2) ∙ If the word begins by a local action of a return: a • We will define a function η that, given a word, produces a tree. η(w’) 14

16. Subset of Context-FreeFormalism Characterizations Extensions Regular Tree Languages • An Σ-labelled binary tree is given by T = (V, λ) where V is a set of 0 and 1’s identifying a node and λ : V → Σ a labelling function. • We will define a function η that, given a word, produces a tree. ∙ As you notice, a call and its matching return are next to each other. ∙ The tree encoding helps keeping track of the stack. ∙ The trees produced by η are called stack-trees. ∙ The set of stack-trees is the set of trees produced by η on any non-empty word, thus STree = η(Σ+). • Those stack-trees can be used as automata. 15

17. Subset of Context-FreeFormalism Characterizations Extensions Regular Tree Languages • An Σ-labelled binary tree is given by T = (V, λ) where V is a set of 0 and 1’s identifying a node and λ : V → Σ a labelling function. • We will define a function η that, given a word, produces a tree. • Those stack-trees can be used as automata. ∙ A set Q of states and Qin of initial states. ∙ Four types of transitions corresponding to four different nodes: Δ01 if a node has both left (0) and right children (1) Δ0 if a node has only a left children Δ1 if a node has only a right children ΔØ if a node has no children (i.e. is a leaf) • Those automata capture the class of Visibly Pushdown Languages. 16

18. Subset of Context-FreeFormalism Characterizations Extensions Characterizations by a grammar • We have expressed formally a Visibly Pushdown Language and we showed that it could be described by a stack-tree automaton. • When we deal with a context-free language, after defining it formally and showing a pushdown automaton, we characterize it by its grammar. V → w non-terminal whatever you want • A Visibly Pushdown Grammar is restricted on the right hand side by the following three possibilities: ∙ X → ε • The non-terminals are in two sets V0 when a call is matched by a return ∙ X → aY V1 when a call is unmatched ∙ X → aYbZ 17

19. Subset of Context-FreeFormalism Characterizations Extensions Characterizations by a grammar • Given the grammar, how do we build the automaton? ∙ The non-terminals are the states. ∙ For each production X → aY we have a transition (X, a, Y) if a is a local action, (X, a, Y, $) if a is a call and (X, a, ┴, Y) + (X, a, $, Y) if a is a return. ∙ For each production X → aYbZ we have a transition (X, a, Y, (b, Z)). • A Visibly Pushdown Grammar is restricted on the right hand side by the following three possibilities: ∙ X → ε ∙ X → aY ∙ X → aYbZ 18

20. Subset of Context-FreeFormalism Characterizations Extensions Extensions to Infinite Words • When a word is infinite, how do we say that we accept it? ∙ An execution (or run) is defined in the same manner by a succession of states and top of the stack. We denote it by ρ. ∙ inf(p) is the states that occur through an infinite execution. ∙ Büchi acceptance condition for a word would be that the final states have to be met at some point : inf(p) ∩ F ≠ Ø. ∙ Muller acceptance condition for a word is that we have to meet only a subset of the final states. If F is the set of possible subets, then inf(p) must belong to F. • We have the same closure properties, but a non-deterministic ω-VPA cannot be determinized (proof by showing a counter-example). • For infinite words, we use ω. Hence we have the class ω-VPL, and a ω-VPA. Depending to which acceptance condition we use, we can speak of a Muller ω-VPA. • A ω-VPL is characterized by infinite stack-trees. 19

21. Applications • In XML we have < >. That looks pretty much like balanced parenthesis, or calls and returns! • Many things are represented with a XML format. We want to know which documents support some queries. We can use a Visibly Pushdown Automaton to check that! • Games are useful to model to model a system and its specifications. Most models use finite graphs, but when we take into consideration the calls then we need to keep track of the stack. It is decidable to check if a visibly pushdown game graph G satisfies requirements expressed in VPL. 20

22. Articles used for this presentation Visibly Pushdown Languages (Rajeev Alur and P. Madhusudan, 36th ACM Symposium on Theory of Computing, 2004 ) Stack Size Analysis for Interrupt-Driven Programs (Chatterjee & Ma & Majumdar & Zhao & Henzinger & Palsberg, Elsevier 2004) Visibly Pushdown Games (C. Löding, P. Madhusudan, and O. Serre. FSTTCS 2004: Foundations of Software Technology and Theoretical Computer Science, 24th International Conference, Springer, 2004.) D U E T O P O P U L A R D E M A N D S , I D O N O T T H A N K Y O U