1 / 93

Multi-Paradigm Programming in Oz for Natural Language Processing

Multi-Paradigm Programming in Oz for Natural Language Processing. Torbjörn Lager, 2003 Acknowledgement: Some of the slides are due to van Roy, Haridi and Schulte. This Course. English or Swedish? The course as a whole Mostly self-studying Exercises Projects Information Course page

chaka
Download Presentation

Multi-Paradigm Programming in Oz for Natural Language Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multi-Paradigm Programming in Oz for Natural Language Processing Torbjörn Lager, 2003 Acknowledgement: Some of the slides are due to van Roy, Haridi and Schulte

  2. This Course • English or Swedish? • The course as a whole • Mostly self-studying • Exercises • Projects • Information • Course page • Mailing list • Why these lectures? • To get you started • To whet your appetite • To point to stumble blocks • Difficulties • Different backgrounds -> Multiple entry points to Oz • Different level of expertise • Highlights • Denys Duchier NLP in Oz

  3. Functional Programming in Oz

  4. Defining a Function • fun {Square X} X*X end • % or equivalently • Square = fun {$ X} X*X end • x.x*x Lambda expression Anonymous function NLP in Oz

  5. Calling a Function • X = {Square 3} % binds X to 9 • X = {Square {Square 3}} % binds X to 81 NLP in Oz

  6. Functional Programming • A program consists of functions (much like ordinary mathematical functions) • The main program itself is written as a function which receives the program's input as its arguments and delivers the program's output as its result • Typically the main function is defined in terms of other functions, which in turn are defined in terms of still more functions, until at the bottom level the functions are language primitives (built-in functions). NLP in Oz

  7. Example: Word Counter (WC) • fun {CountWords S} • {Count {String.tokens 32 S}} • end Tokenizer Counter NLP in Oz

  8. Lists • [a b c] % a list with three elements • [a] % a list with one element • nil % the empty list • a|b|c|nil % a different way of writing the first list • a|b|X % a list with an unbound tail (stream) % also, a pattern NLP in Oz

  9. Strings • Strings are lists whose elements correspond to character codes. For example: • "OZ 2.0” • is the list • [79 90 32 50 46 48] • or equivalently • [&O &Z &  &2 &. &0] • There is also a ByteString datatype - a more economical representation for textual data NLP in Oz

  10. The Unification Operator '=' • The simplest cases are bindings to values, e.g., X=[a b c], and variable-variable bindings, e.g., X=Y. • Unification is symmetric. For example, [a b c]=X means the same as X=[a b c]. • Any two partial values can be unified. For example, unifying the two lists [a X] and [Y b] binds X to b and Y to a. • If the values are already equal, then unification does nothing. For example, [a b]=[a b] does nothing. • If the partial values are incompatible then they cannot be unified. For example, an attempt to unify the two lists [a b] and [a c] will raise a failure exception. • Unification can create cyclic structures, i.e., structures that refer to themselves. For example, the unification X=a|b|X. This creates a cyclic list. NLP in Oz

  11. Pattern Matching • case <Value> • of <Pattern> then … • [] <Pattern> then … • [] <Pattern> then … • else … • end • Can be used as a statement or as an expression NLP in Oz

  12. If-Then-Else • if <Value> then … end • if <Value> then … else … end • if <Value> then … • elseif … then … • elseif … then … • else … end • Can be used as a statement or as an expression NLP in Oz

  13. Example: Functional Append • fun {Append Xs Ys} • case Xs • of nil then Ys • [] X|Xs1 then X|{Append Xs1 Ys} • end • end • X = {Append [a b] [c d]} NLP in Oz

  14. Example: Naive Reverse • fun {Reverse L} • case L • of nil then nil • [] X|Xs then {Append {Reverse Xs} [X]} • end • end • X = {Reverse [a b c]} NLP in Oz

  15. Example: A Faster Reverse • local • fun {RevAux L Acc} • case L • of nil then Acc • [] X|Xs then {RevAux Xs X|Acc} • end • end • in • fun {Reverse L} {RevAux L nil} end • end • X = {Reverse [a b c]} NLP in Oz

  16. Example: Member • fun {Member X Xs} • case Xs • of nil then false • [] Head|Tail then • Head == X orelse {Member X Tail} • end • end • X = {Member c [a b c d]} NLP in Oz

  17. Higher-Order Programming • A programming language supports higher-order programming if: • Functions can be passed as arguments to other functions • The result of applying a function to its arguments can be another function • Functions can be put in data structures NLP in Oz

  18. Examples: SquareList and DoubleList • fun {SquareList Xs} • case Xs • of nil then nil • [] X|Xs1 then X*X |{SquareList Xs1} • end • end • fun {DoubleList Xs} • case Xs • of nil then nil • [] X|Xs1 then 2*X |{DoubleList Xs1} • end • end NLP in Oz

  19. Example: Map • fun {Map Xs F} • case Xs • of nil then nil • [] X|Xs1 then {F X}|{Map Xs1 F} • end • end • X = {Map [1 2 3] Square} % binds X to [1 4 9] • X = {Map [1 2 3] fun {$ X} X*X end} % same • X = {Map [1 2 3] fun {$ X} 2*X end} % binds X to [2 4 6] NLP in Oz

  20. Examples: Sum and Product • fun {Sum Xs} • case Xs • of nil then 0 • [] X|Xs1 then X + {Sum Xs1} • end • end • fun {Product Xs} • case Xs • of nil then 1 • [] X|Xs1 then X * {Sum Xs1} • end • end NLP in Oz

  21. FoldR • fun {FoldR Xs F Y} • case Xs • of nil then Y • [] X|Xs1 then {F X {FoldR Xs1 F Y}} • end • end • X = {FoldR [1 2 3] Number.'+' 0} • X = {FoldR [1 2 3] Number.'*' 1} NLP in Oz

  22. Some Operations in the List Module • X = {Append [a b c] [d e]} % X = [a b c d e] • X = {Reverse [a b c]} % X = [c b a] • X = {Member b [a b c]} % X = true • X = {Last [a b c]} % X = c • X = {Length [a b c]} % X = 3 • X = {List.subtract [a b c] b} % X = [a c] • X = {List.flatten [a [a b] c]} % X = [a a b c] • X = {Sort [c d b d a] Value.'<'} % X = [a b c d d] • X = {Map [12 13 1] IntToFloat} % X = [12.0 13.0 1.0] • X = {Filter [1 a 2 x] IsInt} % X = [1 2] • .... NLP in Oz

  23. Records Label • Record • r1(numb:sg pers:1) • Unifying records • A = r1(numb:_ B = r1(pers:_ • pers:1) numb:sg) • {Inspect A=B} % Inspector shows r1(numb:sg pers:1) • {Inspect A.numb} % Inspector shows ’sg’ Feature Value NLP in Oz

  24. Example: A Simple Lexicon • local • Lexicon = lexicon(’Sandy':[pn] • ’Kim':[pn] • loves:[vb] • sees:[vb] • ) • in • fun {LexLookup Word} Word#(Lexicon.Word) end • end • X = {Map [’Sandy’ loves ’Kim’] LexLookup} • % X = [’Sandy'#[pn] loves#[vb] ’Kim'#[pn]] NLP in Oz

  25. Open Records and Record Constraints • Open records • A = r1(numb:_ B = r1(pers:_ • pers:1 ...) 'case':nom • numb:sg ...) • {Inspect A=B} % r1('case':nom numb:sg pers:1 ...) • Record constraints • R^cat = n • R^agr^numb = sg • R^agr^pers = 1 • {Inspect R} % _(agr:_(numb:sg pers:1 ...) cat:n ...) NLP in Oz

  26. Tuples • T1 = t(a b c) • T1 = t(1:a 2:b 3:c) % succeeds • T1.3 = c % succeeds • T2 = a#b#c % equivalent to ’#’(a b c) NLP in Oz

  27. Example: Compositional Semantics • John = j • Smiles = fun {$ X} smiles(X) end • `John Smiles` = {Smiles John} • John = fun {$ P} {P j} end • Smiles = fun {$ X} smiles(X) end • `John Smiles` = {John Smiles} • `John Smiles` = {fun {$ P} {P j} end fun {$ X} smiles(X) end} • % In all of these cases, `John Smiles` gets % bound to smiles(j) NLP in Oz

  28. A Bird’s View on All This

  29. The Kernel Language Approach • A kernel language • Abstractions based on the kernel language and/or other abstractions. Examples: functions, loops, classes, ports, etc. • Linguistic abstraction/Syntactic sugar. Examples: function expressions, loop expressions, etc. • “Contrary to many (possibly most) languages, we always designed Oz starting from the semantics. The kernel language is carefully given formal semantics. For the more high-level aspects, we always asked the question: can we explain this novel thing of the language in terms of the base language. That's how we managed to erect the full system: by building up on sound semantic foundations; level by level.” NLP in Oz

  30. The Kernel Language Approach Full language • Kernel languages have a small number of programmer-significant elements • Their purpose is to understand programming from the programmer’s viewpoint • They are given a semantics which allows the practicing programmer to reason about correctness and complexity at a high level of abstraction Kernel language Foundational calculus Virtual machine For programmers For mathematicians For implementors NLP in Oz

  31. The Kernel Language Approach fun {Sq X} X*X end Z = {Sq {Sq X}} Full language proc {Sq X Y} {Number.’*’ X X Y} end local X Y Z in {Sq X Y} {Sq Y Z} end Kernel language NLP in Oz

  32. Functional Append -> Procedural Append • fun {Append Xs Ys} • case Xs • of nil then Ys • [] X|Xs1 then X|{Append Xs1 Ys} • end • end • proc {Append Xs Ys Zs} • case Xs • of nil then Zs = Ys • [] X|Xs1 then • Ys1 in Zs = X|Ys1 % or: X|Ys1 = Zs in • {Append Xs1 Ys Ys1} • end • end NLP in Oz

  33. Kernel Language (almost complete) <s> ::= Empty statement Variable-variable binding Variable-value binding Sequential composition Variable creation Conditional Pattern matching Procedure invocation Thread creation Trigger creation Name creation Exception context Raise exception Choice Failure Cell creation Cell exchange Encapsulated search skip <x>1=<x>2 <x>=<v> <s>1 <s>2 local <x> in <s> end if <x> then <s>1else <s>2 end case <x> of <p> then <s>1else <s>2end {<x> <y>1 … <y>n} thread <s> end {ByNeed <x>1 <x>2} {NewName <x>} try <s>1catch <x> then <s>2end raise <x> end choice <s>1[] ... [] <s>2end fail {NewCell <x>1 <x>2} {Exchange <x>1 <x>2 <x>3} <space>

  34. Primary Data Types NLP in Oz

  35. Computation Models concurrent OOP (active object style, e.g., Erlang) (shared state style, e.g., Java) + computation spaces constraint programming Declarative model strict functional programming, e.g., Scheme deterministic logic programming + concurrency + by-need synchronization declarative concurrency lazy functional programming, e.g., Haskell + nondeterministic choice concurrent logic programming + exception handling + encapsulated state object-oriented programming + search nondeterministic LP, e.g., Prolog • We show some of the relationships between the different models • Each model has its own kernel language, its own reasoning techniques, and its own programming techniques • The kernel languages are closely related, e.g., the declarative model is a subset of all of them

  36. Computation Models and NLP concurrent OOP (active object style, e.g., Erlang) (shared state style, e.g., Java) + computation spaces constraint programming Declarative model strict functional programming, e.g., Scheme deterministic logic programming + concurrency + by-need synchronization declarative concurrency lazy functional programming, e.g., Haskell + nondeterministic choice concurrent logic programming + exception handling + encapsulated state object-oriented programming + search nondeterministic LP, e.g., Prolog • Viterbi statistical tagger • Chart parser • Word frequency lister • Compositional logical semantics interpreter • Finite-state tools

  37. Example: The Viterbi Algorithm • Pure functional programming. Is this the optimal paradigm for this application? NLP in Oz

  38. The Mozart Programming Environment • Based on Emacs editor • + GUI tools • Inspector/Browser (visualize values) • Debugger • Profiler • Panel (resource usage) • Compiler Panel (compiler settings and environment), • Distribution Panel (distribution) • Explorer (interactive resolution of constraint problems) • Documentation • van Roy & Haridi: Appendix A • Manual NLP in Oz

  39. Concurrency Oriented Programming

  40. The World is Concurrent! • Concurrent programs • Several activities execute simultaneously • A lot of software in use is concurrent • Operating systems, user interfaces, web servers, etc. • The need for NLP and concurrency? NLP in Oz

  41. Sequential x := 0 x := x + 1 x := x + 1 print x What happens here? X0 = 0 thread X1 = X0 + 1 end thread X2 = X1 + 1 end {Inspect S} What happens here? x := 0 thread x := x + 1 end thread x := x + 1 end print x What happens here? declare Xs S in thread Xs={List.number 0 100 1} end thread S={Map Xs fun {$ X} X*X end} end {Inspect S} Concurrency Can Be Difficult, but Does Not Have to Be… NLP in Oz

  42. Threads • Thread creation: • thread … end • Can be used as a statement or as an expression • Threads in Oz are dataflow threads, i.e., they suspend on availability of data • Excellent for synchronization! • Threads in Oz a lightweight NLP in Oz

  43. Concurrent Map • declare F Xs Ys Zs • fun {CMap Xs F} • case Xs • of nil then nil • [] X|Xr then • thread {F X} end|{CMap Xr F} • end • end • {Inspect thread {CMap Xs F} end} % Inspector displays _ • Xs = 1|2|Ys % Inspector displays _|_|_ • fun {F X} X*X end % Inspector displays 1|4|_ • Ys = 3|Zs % Inspector displays 1|4|9|_ • Zs = nil % Inspector displays [1 4 9] NLP in Oz

  44. Fibonacchi and the Panel • A demo to show how lightweight Oz threads are NLP in Oz

  45. Streams • A stream is a potentially unbounded list of messages, i.e., it is a list whose tail is an unbound dataflow variable. • declare Xs Xs3 • {Inspect Xs} • Xs=0|1|2|3|4|Xs2 % Inspector shows 0|1|2|3|4|_ • Xs2=5|6|7|Xs3 % Inspector shows 0|1|2|3|4|5|6|7|_ • Important container! NLP in Oz

  46. Declarative Concurrency • Producer-consumer with dataflow fun {Prod N Max} if N<Max then N|{Prod N+1 Max} else nil end end fun {Cons Xs A} case Xs of X|Xr then {Cons Xr A+X} [] nil then A end end localXs S in threadXs={Prod 0 1000} end threadS={Cons Xs 0} end end • Prod and Cons threads share list Xs • Dataflow behavior of case statement (synchronizing on data availability) gives stream communication • No other concurrency control needed Xs Prod Cons NLP in Oz

  47. Declarative Concurrency • Let us compare the sequential and concurrent versions • The result of the calculation is the same in both cases • So what is different? • Sequential version: • Results are produced in batch: the whole calculation is done and then all results are given at once • Concurrent version: • Results are produced incrementally, element by element local Xs S in Xs={Prod 0 1000} S={Cons Xs 0} end local Xs S in thread Xs={Prod 0 1000} end thread S={Cons Xs 0} end end NLP in Oz

  48. Transformation-Based Tagging: Small Part-of-Speech Tagging Example rules pos:NN>VB <- pos:TO@[-1] opos:VB>NN <- pos:DT@[-1] o.... input She decided to table her data lexicon data:NN decided:VB her:PN she:PN table:NN to:TO NP VB TO NN PN NN NP VB TO NN PN NN NP VB TO VB PN NN NLP in Oz

  49. Example: Incremental Brill Tagging • declare Wd T1 T2 T3 • Lex = lex(the:dt light:vb cloud:nn 'is':vb) • {Inspect T3} • proc {Tag N} • T1^N = Lex.(Wd^N) % lexical lookup • if T1^N == vb andthen T1^(N-1) == dt then % vb>nn <- dt@[-1] • T2^N = nn • else T2^N = T1^N end • if T2^N == nn andthen T2^(N+1) == nn then % nn>jj <- nn@[-1] • T3^N = jj • else T3^N = T2^N end • end • /* • Wd^1 = the thread {Tag 1} end % _(1:dt ...) • Wd^2 = light thread {Tag 2} end % _(1:dt ...) • Wd^3 = cloud thread {Tag 3} end % _(1:dt 2:jj ...) • Wd^4 = is thread {Tag 4} end % _(1:dt 2:jj 3:nn 4:vb ...) • Wd^5 = the thread {Tag 5} end % _(1:dt 2:jj 3:nn 4:vb 5:dt) • */ NLP in Oz

  50. Programming with Explicit State

More Related