Embedding DSLs in Newspeak: NewShell, EBNF and Hopscotch

1. Embedding DSLs in Newspeak:NewShell, EBNF and Hopscotch Gilad Bracha Ministry of Truth

2. Three DSLs • Shell Scripting • Parsing • UI

3. Newspeak Syntax for Java Programmers

4. Newspeak Syntax for Java Programmers

5. Newspeak Syntax for C# Programmers

6. Syntax: Unary Expressions x foo means x.foo()

7. Syntax: Unary Expressions x foo means x.foo()

8. Binary Expressions 6 + 3 factorial evaluates to 12. In javanese, we’d write 6.+(3.factorial()) or, really 6.plus(3.factorial())

9. Keyword Expressions Windows version: 7 + 0i . is equivalent to all these: Windows version: (7 + 0i). Windows version: (7 + (0 i)). Windows.version(7 + 0.i()) ;

10. Keyword Expressions add: 1 to: Cobol in javanese would be add(1, Cobol); Big advantage - no arity errors

11. Three DSLs • Shell Scripting • Parsing • UI

12. NewShell class NewShell = ( ls value: ‘-l’. ls - ‘lt’. echo value: ‘Hello World’. svn value: ‘ls’ value: ‘svn://myDir’. (ls value findTokens: {Character cr}) collect: ls. )()

13. Three DSLs • Shell Scripting • Parsing • UI

14. Parser Combinators BNF id = letter (letter | digit) *

15. Parser Combinators BNF id = letter (letter | digit) * Newspeak id = letter, (letter | digit) star.

16. Parser Combinators BNF id = letter (letter | digit) * Newspeak id = letter, (letter | digit) star. Javanese id = letter().seq(letter().or(digit()).star());

17. How it Works id = letter, (letter | digit) star.

18. How it Works id = letter, (letter | digit) star.

19. How it Works id = letter, (letter | digit) star.

20. How it Works id = letter, (letter | digit) star.

21. How it Works id = letter, (letter | digit) star. letter

22. How it Works id = letter, (letter | digit) star. letter

23. How it Works id = letter, (letter | digit) star. letter

24. How it Works id = letter, (letter | digit) star. letter

25. How it Works id = letter, (letter | digit)star. letter

26. How it Works id = letter, (letter | digit) star. letter

27. How it Works id = letter, (letter | digit) star. letter letter

28. How it Works id = letter, (letter| digit) star. letter letter

29. How it Works id = letter, (letter | digit) star. letter letter digit

30. How it Works id = letter, (letter | digit) star. letter | letter digit

31. How it Works id = letter, (letter | digit) star. letter star | letter digit

32. How it Works , id = letter, (letter | digit) star. letter star | letter digit

33. Why is this Pretty? id = letter, (letter | digit) star.

34. Why is this Ugly? id = letter().seq(letter().or(digit()).star());

35. Why is this Ugly? id = letter().seq(letter().or(digit()).star()); vs. id = letter (letter | digit) * vs. id = letter, (letter | digit) star.

36. Why is this Ugly? id = letter().seq(letter().or(digit()).star()); vs. id = letter (letter | digit) * vs. id = letter, (letter | digit) star.

37. Why is it Ugly? A programming language is low level when its programs require attention to the irrelevant - Alan Perlis

38. A Complete Grammar class ExampleGrammar1 = ExecutableGrammar ( | digit = charBetween: ‘0’ and: ‘9’. letter = (charBetween: ‘a’ and:’z’) | (charBetween: ‘A’ and:’Z’). id = letter, (letter | digit) star. identifier = tokenFor: id. hat = tokenFromChar: ‘ˆ’. expression = identifier. returnStatement = hat, expression. | )()

39. Building an AST returnStatement = hat, expression wrapper:[:r : e | ReturnStatAST on: e ]

40. Building an AST returnStatement = hat, expression wrapper:[:r : e | ReturnStatAST on: e ] ^ wrapper: , expression

41. Grammar returnStatement = hat, expression wrapper:[:r : e | ReturnStatAST on: e ]

42. Semantic Action returnStatement = hat, expression wrapper:[:r : e | ReturnStatAST on: e ]

43. Factor out Grammar Superclass (pure grammar specification): returnStatement = hat, expression Subclass (AST builder): returnStatement = ( super returnStatement wrapper:[:r : e | ReturnStatAST on: e] )

44. Modular Parser class ExampleParser1= ExampleGrammar1 () ( id = ( ˆsuper id wrapper:[:fst :snd | fst asString, (String withAll: snd)] ) identifier = ( ˆsuper identifier wrapper:[:v | VariableAST new name: v token; start: v start; end: v end]. ) r

45. Modular Parser … returnStatement = ( ˆsuper returnStatement wrapper:[:r :e | ReturnStatAST new expr:e; start: r start; end: e end]. ) )

46. Extending a Grammar class ExampleGrammar2 = ExampleGrammar1 ( | if = tokenFromSymbol:#if. then = tokenFromSymbol:#then. else = tokenFromSymbol:#else. ifStatement = if, expression, then, statement, else, statement. statement = ifStatement | returnStatement. | )()

47. Mutual Recursion returnStat IfStmt if then else | stmt

48. Mutual Recursion digit nil letter nil nil id identifier nil hat nil expression nil returnStatement nil if nil then nil else nil ifStatement nil statement nil

49. Mutual Recursion digit a parser letter nil nil id identifier nil hat nil expression nil returnStatement nil if nil then nil else nil ifStatement nil statement nil

50. Mutual Recursion digit a parser letter a parser nil id identifier nil hat nil expression nil returnStatement nil if nil then nil else nil ifStatement nil statement nil