The Chomsky Hierarchy: Basics of Language and Grammar Classes

1. The chomsky hierarchy Module 03.3COP4020 – Programming Language Concepts Dr. Manuel E. Bermudez

2. Language and Grammar classes • We’ll Cover • Grammar Conventions • Sample Grammars • The Chomsky Hierarchy • Regular Languages • Context-free Languages • Context-sensitive Languages • Recursively Enumerable Languages

3. Upper case letter – nonterminal Lower case letter – terminal Lower case greek letter – strings in V* Left part of the first production: start symbol. If same as previous, omit left part. Example: S → aSb → ε Grammar conventions

4. Definition: The language generated by a grammar G, is the set L(G) = { Σ* | S =>* } Definition: A sentential form generated by a grammar G is any string  such that S =>* . Definition: A sentence generated by a grammar G is any sentential form  such that  Σ*. Lemma: L(G) = { |  is a sentence} Proof: Trivial. Language and Grammar classes

5. Example: Grammar for identifiers. Identifier → Letter → Identifier Letter → Identifier Digit Letter → ‘a’ → ‘A’ → ‘b’ → ‘B’ . . . → ‘z’ → ‘Z’ Digit → ‘0’ → ‘1’ . . . → ‘9’ A regular grammar Language pattern: {an | n > 1}

6. Grammar: S → aSb S → sentential forms S => aSb => aaSbb => aaaSbbb => aaaaSbbbb =>… ε ab aabb aaabbb aaaabbbb sentences A context-free grammar Language pattern: {anbn| n > 1} => => => => =>

7. A context-sensitive Grammar A => aABC => aaABCBC => … aBC aaBCBC aaaBCBCBC abC aabCBC aaaBBCBCC abc aabBCC aaaBBBCCC aabbCC aaabBBCCC aabbcC aaabbbCCC aabbcc aaabbbcCC aaabbbccc A → aABC → aBC CB → BC aB → ab bB → bb bC → bc cC → cc => => => => => => => => => => => => => => => => => Language pattern: {anbncn | n > 1}

8. A hierarchy of grammars, the languages they generate, and the machines the accept those languages. Developed by Noam Chomsky (MIT) in 1957. Linguist, philosopher, cognitive scientist, historian, logician. Social critic, political activist. Four language levels: Regular languages: {an | n > 1} Context-free languages: {an bn/ n > 0} Context-sensitive languages: {an bncn| n > 1} Unrestricted: (no pattern) The chomsky hierarchy

9. Impractical The chomskyhierarchy Practical

10. The chomskyhierarchy 0: Recursively Enumerable Languages Type 3: appropriate for lexical analysis (scanning). Type 2: appropriate for Syntax analysis (parsing) 1: Context-Sensitive Languages 2: Context-free Languages 3: Regular Languages {an | n > 0} {anbn | n>0} {anbncn | n>0} English?

11. summary • Grammar Conventions • The Chomsky Hierarchy • Regular Languages (good for scanning) • Context-free Languages (good for parsing) • Context-sensitive Languages • Recursively Enumerable Languages