Type Inference

1. Type Inference Edited version of notes by Dave Walker@ Princeton who in turn edited notes by Bob Harper@CMU Both Dave and Bob were grad students at Cornell, so I decided it was OK to borrow their work…

2. Criticisms of Typed Languages • Types overly constrain functions & data • polymorphism makes typed constructs useful in more contexts • universal polymorphism => code reuse • existential polymorphism => rep. independence • subtyping => coming soon • Types clutter programs and slow down programmer productivity • type inference • uninformative annotations may be omitted

3. Type Inference • Today • overview • generation of type constraints from unannotated simply-typed programs • solving type constraints

4. Type Schemes • A type scheme contains type variables that may be filled in during type inference • s ::= a | int | bool | s1 -> s2 • A term scheme is a term that contains type schemes rather than proper types • e ::= ... | fun f (x:s1) : s2 is e end

5. Example fun map (f, l) is if null (l) then nil else cons (f (hd l), map (f, tl l))) end

6. Step 1: Add Type Schemes fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l), map (f, tl l))) end type schemes on functions

7. Step 2: Generate Constraints fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l), map (f, tl l))) end b = b’ list

8. Step 2: Generate Constraints constraints b = b’ list fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l), map (f, tl l))) end : d list

9. Step 2: Generate Constraints constraints b = b’ list fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l), map (f, tl l))) end : d list b = b’’’ list b = b’’ list

10. Step 2: Generate Constraints constraints b = b’ list b = b’’ list b = b’’’ list fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l : b’’), map (f, tl l : b’’’ list))) end : d list a = a b = b’’’ list

11. Step 2: Generate Constraints constraints b = b’ list b = b’’ list b = b’’’ list a = a b = b’’’ list fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l : b’’) : a’, map (f, tl l): c)) end : d list a = b’’ -> a’

12. Step 2: Generate Constraints constraints b = b’ list b = b’’ list b = b’’’ list a = a b = b’’’ list a = b’’ -> a’ fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l : b’’) : a’, map (f, tl l): c)) : c’ list end : d list c = c’ list a’ = c’

13. Step 2: Generate Constraints constraints b = b’ list b = b’’ list b = b’’’ list a = a b = b’’’ list a = b’’ -> a’ c = c’ list a’ = c’ fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l : b’’) : a’, map (f, tl l): c)) : c’ list end : d list d list = c’ list

14. Step 2: Generate Constraints constraints b = b’ list b = b’’ list b = b’’’ list a = a b = b’’’ list a = b’’ -> a’ c = c’ list a’ = c’ d list = c’ list fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l : b’’) : a’, map (f, tl l) : c)) : c’ list : d list end : d list d list = c

15. Step 2: Generate Constraints final constraints b = b’ list b = b’’ list b = b’’’ list a = a b = b’’’ list a = b’’ -> a’ c = c’ list a’ = c’ d list = c’ list d list = c fun map (f : a, l : b) : c is if null (l) then nil else cons (f (hd l), map (f, tl l))) end

16. Step 3: Solve Constraints • Constraint solution provides all possible solutions to type scheme annotations on terms final constraints b = b’ list b = b’’ list b = b’’’ list a = a ... map (f : a -> b x : a list) : b list is ... end solution a = b’ -> c’ b = b’ list c = c’ list

17. Step 4: Generate types • Generate types from type schemes • Option 1: pick an instance of the most general type when we have completed type inference on the entire program • map : (int -> int) -> int list -> int list • Option 2: generate polymorphic types for program parts and continue (polymorphic) type inference • map : All (a,b,c) (a -> b) -> a list -> b list

18. Type Inference Details • Type constraints are sets of equations between type schemes • q ::= {s11 = s12, ..., sn1 = sn2} • eg: {b = b’ list, a = b -> c}

19. Constraint Generation • Syntax-directed constraint generation • our algorithm crawls over abstract syntax of untyped expressions and generates • a term scheme • a set of constraints • Algorithm defined as set of inference rules (as always) • G |-- u => e : t, q

20. Constraint Generation • Simple rules: • G |-- x ==> x : s, { } (if G(x) = s) • G |-- 3 ==> 3 : int, { } (same for other ints) • G |-- true ==> true : bool, { } • G |-- false ==> false : bool, { }

21. Operators G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 ------------------------------------------------------------------------ G |-- u1 + u2 ==> e1 + e2 : int, q1 U q2 U {t1 = int, t2 = int} G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 ------------------------------------------------------------------------ G |-- u1 < u2 ==> e1 + e2 : bool, q1 U q2 U {t1 = int, t2 = int}

22. If statements G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 G |-- u3 ==> e3 : t3, q3 ---------------------------------------------------------------- G |-- if u1 then u2 else u3 ==> if e1 then e2 else e3 : a, q1 U q2 U q3 U {t1 = bool, a = t2, a = t3}

23. Function Application G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 ---------------------------------------------------------------- G |-- u1 u2==> e1 e2 : a, q1 U q2 U {t1 = t2 -> a}

24. Function Declaration G, f : a -> b, x : a |-- u ==> e : t, q ---------------------------------------------------------------- G |-- fun f(x) is u end ==> fun f (x : a) : b is e end : a -> b, q U {t = b} (a,b are fresh type variables; not in G, t, q)

25. Solving Constraints • A solution to a system of type constraints is a substitution S • a function from type variables to types • because some things get simpler, we assume substitutions are defined on all type variables: • S(a) = a (for almost all variables a) • S(a) = s (for some type scheme s) • dom(S) = set of variables s.t. S(a)  a

26. Substitutions • Given a substitution S, we can define a function S* from type schemes (as opposed to type variables) to type schemes: • S*(int) = int • S*(s1 -> s2) = S*(s1) -> S*(s2) • S*(a) = S(a) • Due to laziness, I will write S(s) instead of S*(s)

27. Composition of Substitutions • Composition (U o S) applies the substitution S and then applies the substitution U: • (U o S)(a) = U(S(a)) • We will need to compare substitutions • T <= S if T is “more specific” than S • T <= S if T is “less general” than S • Formally: T <= S if and only if T = U o S for some U

28. Composition of Substitutions • Examples: • example 1: any substitution is less general than the identity substitution I: • S <= I because S = S o I • example 2: • S(a) = int, S(b) = c -> c • T(a) = int, T(b) = int -> int, T(c) = int • we conclude: T <= S • if T(a) = int, T(b) = int -> bool then T is unrelated to S (neither more nor less general)

29. Solving a Constraint • S |= q if S is a solution to the constraints q S(s1) = S(s2) S |= q ----------------------------------- S |= {s1 = s2} U q ---------- S |= { } a solution to an equation is a substitution that makes left and right sides equal any substitution is a solution for the empty set of constraints

30. Most General Solutions • S is the principal (most general) solution of a constraint q if • S |= q (it is a solution) • if T |= q then T <= S (it is the most general one) • Lemma: If q has a solution, then it has a most general one • We care about principal solutions since they will give us the most general types for terms

31. Examples • Example 1 • q = {a=int, b=a} • principal solution S:

32. Examples • Example 1 • q = {a=int, b=a} • principal solution S: • S(a) = S(b) = int • S(c) = c (for all c other than a,b)

33. Examples • Example 2 • q = {a=int, b=a, b=bool} • principal solution S:

34. Examples • Example 2 • q = {a=int, b=a, b=bool} • principal solution S: • does not exist (there is no solution to q)

35. principal solutions • principal solutions give rise to most general reconstruction of typing information for a term: • fun f(x:a):a = x • is a most general reconstruction • fun f(x:int):int = x • is not

36. Unification • Unification: An algorithm that provides the principal solution to a set of constraints (if one exists) • If one exists, it will be principal

37. Unification • Unification: Unification systematically simplifies a set of constraints, yielding a substitution • during simplification, we maintain (S,q) • S is the solution so far • q are the constraints left to simplify • Starting state of unification process: (I,q) • Final state of unification process: (S, { })

38. Unification Machine • We can specify unification as a transition system: • (S,q) -> (S’,q’) • Base types & simple variables: -------------------------------- (S,{int=int} U q) -> (S, q) ------------------------------------ (S,{bool=bool} U q) -> (S, q) ----------------------------- (S,{a=a} U q) -> (S, q)

39. Unification Machine • Functions: • Variable definitions ---------------------------------------------- (S,{s11 -> s12= s21 -> s22} U q) -> (S, {s11 = s21, s12 = s22} U q) --------------------------------------------- (a not in FV(s)) (S,{a=s} U q) -> ([a=s] o S, [s/a]q) -------------------------------------------- (a not in FV(s)) (S,{s=a} U q) -> ([a=s] o S, [s/a]q)

40. Occurs Check • What is the solution to {a = a -> a}?

41. Occurs Check • What is the solution to {a = a -> a}? • There is none! (Remember your homework) • The occurs check detects this situation -------------------------------------------- (a not in FV(s)) (S,{s=a} U q) -> ([a=s] o S, [s/a]q) occurs check

42. Irreducible States • Recall: final states have the form (S, { }) • Stuck states (S,q) are such that every equation in q has the form: • int = bool • s1 -> s2 = s (s not function type) • a = s (s contains a) • or is symmetric to one of the above • Stuck states arise when constraints are unsolvable

43. Termination • We want unification to terminate (to give us a type reconstruction algorithm) • In other words, we want to show that there is no infinite sequence of states • (S1,q1) -> (S2,q2) -> ...

44. Termination • We associate an ordering with constraints • q < q’ if and only if • q contains fewer variables than q’ • q contains the same number of variables as q’ but fewer type constructors (ie: fewer occurrences of int, bool, or “->”) • This is a lexicographic ordering • There is no infinite decreasing sequence of constraints • To prove termination, we must demonstrate that every step of the algorithm reduces the size of q according to this ordering

45. Termination • Lemma: Every step reduces the size of q • Proof: By cases (ie: induction) on the definition of the reduction relation. -------------------------------- (S,{int=int} U q) -> (S, q) ------------------------------------ (S,{bool=bool} U q) -> (S, q) ---------------------------------------------- (S,{s11 -> s12= s21 -> s22} U q) -> (S, {s11 = s21, s12 = s22} U q) ------------------------ (a not in FV(s)) (S,{a=s} U q) -> ([a=s] o S, [s/a]q) ----------------------------- (S,{a=a} U q) -> (S, q)

46. Complete Solutions • A complete solution for (S,q) is a substitution T such that • T <= S • T |= q • intuition: T extends S and solves q • A principal solution T for (S,q) is complete for (S,q) and • for all T’ such that 1. and 2. hold, T’ <= T

47. Properties of Solutions • Lemma 1: • Every final state (S, { }) has a complete solution. • It is S: • S <= S • S |= { }

48. Properties of Solutions • Lemma 2 • No stuck state has a complete solution (or any solution at all) • it is impossible for a substitution to make the necessary equations equal • int  bool • int t1 -> t2 • ...

49. Properties of Solutions • Lemma 3 • If (S,q) -> (S’,q’) then • T is complete for (S,q) iff T is complete for (S’,q’) • T is principal for (S,q) iff T is principal for (S’,q’) • in the forward direction, this is the preservation theorem for the unification machine!

50. Summary: Unification • By termination, (I,q) ->* (S,q’) where (S,q’) is irreducible. Moreover: • If q’ = { } then • (S,q’) is final (by definition) • S is a principal solution for q • Consider any T such that T is a solution to q. • Now notice, S is principal for (S,q’) (by lemma 1) • S is principal for (I,q) (by lemma 3) • Since S is principal for (I,q), we know T <= S and therefore S is a principal solution for q.