A Type System for Higher-Order Modules

A Type System forHigher-Order Modules Derek Dreyer (joint work with Karl Crary and Bob Harper) Fall 2002

SML Generativity • SML functors are generative: • If result signature has abstract types, each application of the functor generates new types • At the level of type theory (HL), generativity is very easy to enforce: • Restrict types to only be projected from module paths (or, more generally, values)

Higher-Order Functors signature S = sig type t val x : t end structure SomeX : S = ... functor ApplyToSomeX(F : S -> S) = F(SomeX) functor Id(X : S) = X structure Res1 = Id(SomeX) structure Res2 = ApplyToSomeX(Id) • Res1.t = SomeX.t • Due to generativity of SML functors, Res2.t is a new abstract type!

Applicative Functors • Would like to express dependency of result of ApplyToSomeX on argument F: ApplyToSomeX : (F : S -> S) -> S where type t = F(SomeX).t • Then, Res2.t = ApplyToSomeX(Id).t =Id(SomeX).t = SomeX.t, as desired.

Applicative Functors • This is an “applicative” interpretation of functors: • F(X) outputs the same “t” component every time F is applied to X. • Applicativity in O’Caml an easy extension of path restriction (Leroy 95): • Extend paths with named functor applications

Problems with O’Caml • Say structure Y = X. • According to phase separation,F(Y).t = F(X).t since X and Y have the same type components. • Not so in O’Caml: • Types compared syntactically

Problems with O’Caml • In the presence of effects, generativity is often critical: signature SYMBOL_TABLE = sig type symbol val string_to_symbol : string -> symbol val symbol_to_string : symbol -> string val eq : symbol * symbol -> bool end

Problems with O’Caml functor SymbolTableFun() :> SYMBOL_TABLE = type symbol = int (* index into table *) val table = Array.array(initial_size,NONE) ... fun symbol_to_string n = (case Array.sub(table,n) of SOME x => x | NONE => raise (Fail “bad symbol”)) ... end

Importance of Generativity • Two dynamic instances of SymbolTableFun ought to generate distinct symbol types. • With applicative functors, however, the symbol type provided by any instance of SymbolTableFun will = SymbolTableFun().t

Phase Separation • Why are applicative functors sensible? • Why is F(X).t well-determined? • Principle of phase separation: • Modules are second-class • Type components of a module can only depend on other type components,not on run-time conditions.

Key Question • When are two types t1 and t2 equivalent? • If t1 = M1.t and t2 = M2.t, the question is: When are M1 and M2 equivalent? • Most liberal answer: Static Equivalence • G` M1@ M2 : s when their type components are equal

Fully Transparent Modules • Start by ignoring sealing (M :>s). • Relax Harper-Lillibridge to allow projecting types from any module: • F(X).t no big deal ) functors are applicative • I.e. all modules are determinate

Abstraction • Now we add opaque sealing: M ::s • Problem: Say A = M ::s and B = M ::s. • A.t and B.t are distinct abstract types. • But by reflexivity of equivalence,A.t = (M ::s).t = (M ::s).t = B.t • So sealed modules must be indeterminate.

Abstraction as an Effect • Irreflexivity of sealed modules reminiscent of irreflexivity in languages with effects: • Call a module pure if it is free of sealing,and impure otherwise. • Grant M certain privileges only if it is pure: • Can project types from M • Can compare M for equivalence

Abstraction is not Generativity • Functors still behave applicatively: • F(X).t is perfectly valid, assuming F and X are module variables, since variables are pure. • Sealing allowed in applicative functors • Semantics similar to O’Caml • If we want generativity, we need to distinguish it from abstraction.

Generativity • Generativity = generation of abstract types at run-time • Violation of phase separation! • Not really, but we’ll pretend... • Track generativity like a “real” effect • Abstract types tied (notionally) to run-time state of module defining them

Static and Dynamic Effects • Abstraction (from before) is a static effect • Generativity is a dynamic effect • Weak sealing (M ::s) is abstract • Induces just a static effect • Strong sealing (M :>s) is generative • Induces a static and a dynamic effect

Generative Functor Signatures • Need to know from a functor’s signature whether its body is impure (generative): • Distinguish total vs. partial functor signatures, i.e. applicative (as before) vs. generative • To deserve an applicative signature, the functor’s body must be dynamically pure.

Type System Overview • Translucency modeled by singletons: [T] ¼ “sig type t end” S(M) ¼ “sig type t = M.t end” • Static equivalence defined almost exactly the same as for singleton kind calculus • Because pure module language very close to type constructor language

Type System Overview • Module typing judgment: G`k M : s, where k is purity level drawn from lattice • Mostly standard dependent typing rules

Syntax

Atomic Typing Rules

S Typing Rules • M must be pure in p2M in order for the signature s’’[p1M/s] to be well-formed

P Typing Rules • M must be pure in F(M) in order for the signature s’’[M/s] to be well-formed

Singleton Typing Rules

Let and Subsumption • Let has signature annotation, necessary to dodge the avoidance problem: • Subsumption can give a module a weaker signature or purity level:

Base Equivalence Rules • Equivalence of atomic type modules = equivalence of the types they contain • Need these rules to observe equivalences like Typ [t] ´t:

Unitary Equivalence • Call a signature unitary if it is of the form: • Modules of unitary signature have no type components, so equivalence is trivial:

Decidability • Principal signature synthesis • G`k M )s. • Algorithm similar to principal kind synthesis, also synthesizes minimal purity level • Type system is decidable • As with kinds, boils down to decidability of type/module equivalence • Proof nearly identical to Stone-Harper

Existential Signatures • Say we want un-annotated lets, and the ability to write F(M) and p2M, where M is impure... • Introduce existential signature 9 s:s1.s2.

Elaboration • Not so easy to introduce existentials... • Use elaboration (same as Harper-Stone): • Existential signatures model hidden modules • 9 s:s1.s2 is really S s:s1.s2. • But when you see an existential, immediately project out the second component... • 9 s:s1.s2 is like [1Bs:s1.2*:s2] in HS

Modules as First-Class Values • Existential types are a known way of encoding modules as first-class values • Encodable in the system via Church encoding • If s phase-splits to [a:k.t], then ¼9a:kappa.t.

A Primitive Extension • But we can get open-scope unpacking with a primitive extension:

Importance of Generativity • Dynamic effects are critical here • Types can now actually depend on run-time conditions • Example: • F must be generative, or else X1@ X2.

Related Work: Russo’s Thesis • Defines a higher-order module language with only applicative functors • Existential signatures used in a way very similar to Shao’s • Same power as DCH before the addition of generativity and dynamic effects

Related Work: Moscow ML • Moscow ML combines Russo’s higher-order modules with Standard ML • Allowed to write anything in the body of an applicative functor • Including generative functor applications • Generativity doesn’t work • Can eta-expand a generative functor into an applicative one • Moreover, language is unsound

Related Work: Shao • Shao has both applicative and generative • Applicative = Transparent • Generative = Opaque • Precludes one from having opaque substructures in an applicative functor • But this is useful, e.g. for datatype’s • Shao can be encoded as a subsystem of DCH that lacks weak sealing

Conclusion • Static equivalence (HMM-style) is as liberal as possible for second-class • Restrict type equivalence by treating abstraction and generativity as effects: • Abstraction is a static effect • Generativity is a dynamic effect, captured by (generative) functors • Scales to handle real run-time type generation in first-class modules

A Type System for Higher-Order Modules