Extending Type Systems in a Library

1. Extending Type Systems in a Library Yuriy Solodkyy, Jaakko Järvi, EsamMlaih Department of Computer Science and Engineering Texas A&M University March 23, 2010 http:://parasol.tamu.edu/~yuriys/

2. Motivation: Trends • Programs grow in size and complexity • Use multiple libraries • Depend on third party components (APIs, protocols) • Have to account for OS, hardware & compiler differences • Are written by many people with different skill sets • Languages grow in abstraction level • Crave for more features • Have to deal with backward compatibility • Provide larger standard libraries • Domain specialization

3. Motivation: Bugs • Cost U.S. economy $60 billion each year • users incurred 64% of the cost • developers and vendors – 36% • Improvements in testing • can reduce it by about a third ($23 billion) • won't eliminate all software errors Collection of Software Bugs by Thomas Huckle

4. Motivation: Bugs Ariane 5 Explosion narrowing conversion 6/4/1996, $500 million NASA Mars Climate Orbiter mixing metric and imperial units 8/23/1999, $125 million LA Air-Traffic Control counter underflow 9/14/2004 Patriot Missile Failure float imprecision rounding 02/25/1991, 28 dead 100 injured Zune's New Year Freeze infinite loop 12/31/2008 Pentium FDIV bug incomplete entries in a look-up-table 1994, $400 million Collection of Software Bugs by Thomas Huckle

5. Motivation: Solutions • Testing is not a panacea • Does not prove absence of errors • Specific to a project and cannot be reused • Has to be maintained in sync with evolution of project • Improvements in it will reduce the cost of bugs by about a third, but won't eliminate all software errors • Many errors can be detected without running the program through the use of: • Type system • Static analysis • Sometimes absence of specific run-time errors can be proven

6. The problem with Type System • Interesting ones are domain-specific • e.g units, qualifiers • Impossible to account for all interesting ones in general-purpose programming language • covariance typing, full static typing and subtype substitutability: pick two • can be designed and approached differently • Composability of multiple type systems in a single program is hard to achieve • e.g. units for measurements and regular expression types for patterns

7. XTL: eXtensible Typing Library • Type systems • prevent certain kinds of bugs from happening • not easily extensible • domain specific • Observation • many interesting domain specific type-systems can be implemented as libraries • Objective • explore how far a pure library solution suffices in extending a type system Solodkyy et al. LCSD’06

8. Use Cases • Tracking physical quantities/units • converting between compatible units • rejecting operations that do not agree on units • Tracking semantic properties • nullability, sign, oddity of a number • security vulnerability to format strings • usage of user pointers in kernel space • deadlocks and data races • Regular Expression Types • typing XML documents • ordering patterns • Variant Parametric Types • defining relations between instances of a parameterized type based on relations of its argument types • units • kg = lbs – ok, convert • kg = km/h – error • semantic properties • even+odd=odd • sprintf(buf,…); • T* krnl = usr;: error • regular expression types • T*U* <: (T|U)* • variant parametric types • vector<B*> <: vector<D*> Solodkyy et al. LCSD’06

9. XTL Contributions • We report on implementing simple type qualifiers as a C++ library • We implement regular expression types (in a limited form) to check XML data • We provide a framework to help others in extending the C++ type system for their abstractions Solodkyy et al. LCSD’06

10. Example returns only positive numbers // a is a positive value double a = current_height(); // b is a positive value double b = max_allowed_height(); // b-a may be negative though! Result is always positive double c = std::sqrt(b-a); // b+a is assumed to be positive. Unless there is bug! double d = std::sqrt(b+a); upper bound for values from previous call difference is positive, but not for type system argument and result are always positive

11. Refining Type Systems Objective: emulate domain-specific type system as a library Solodkyy et al. LCSD’06

12. Type System 1: Type qualifiers Objective: providing support for type qualifiers in programs • Allow tracking of semantic properties like: • immutability of a certain value (const) • sign of a number (pos, neg) • assumptions about pointers (optional, nonnull) • trustworthiness of a certain value (tainted, untainted) • oddity of a number (odd, even) • origin of a pointer (user, kernel) • Easily defined in terms of: • direction (positive/negative qualifier) • abstract operations on qualifiers • But • cannot handle flow-sensitive qualifiers • cannot handle arbitrary reference qualifiers Solodkyy et al. LCSD’06

13. Example: Qualifiers’ Hello World Declare few qualifiers: Q is positive if T <: Q T Q is negative if Q T <: T Define how different operations transfer properties #include <xtl/qualdecl.hpp> DECLARE_NEGATIVE_QUALIFIER(pos); DECLARE_POSITIVE_QUALIFIER(tainted); // ... Other qualifiers ... namespace xtl { template <> struct minus<pos, neg> { typedef qual<pos> type; }; template <> struct minus<neg, pos> { typedef qual<neg> type; }; template <> struct mul<pos, pos> { typedef qual<pos> type; }; template <> struct mul<pos, neg> { typedef qual<neg> type; }; template <> struct mul<nonnull, nonnull>{ typedef qual<nonnull> type;}; template <> struct div<nonnull, nonnull>{ typedef qual<nonnull> type;}; } // of namespace xtl int main() { untainted<nonnull<neg<int> > >a(-42); pos<untainted<nonnull<int> > >b(7); neg<nonnull<long> >c = a * b; // OK: drop negative qualifier untainted //nonnull<pos<double> > d = b - a; // Error: nonnull isn’t carried by - pos<tainted<double> >e = b + a*c; // OK to add positive qualifier //pos<double> f = e; // Error: ... but not OK to drop it! } Subtraction does not carry nonnull Positive qualifiers can be added to result type... Multiplication carries nonnull & negativeness on pos & neg arguments Declare your variables with appropriate properties ... but not dropped once they are there! Solodkyy et al. LCSD’06

14. Example function descend accepts only positive numbers we achieve this by restricting its argument type to be a subtype of pos<double> we convert value of a subtype into a value of a supertype // Example definition that accepts only positive doubles template<class U> typename enable_if< typename is_subtype<U, pos<double> >::type, void >::type descend(const U& altitude) { pos<double>a = subtype_cast<pos<double> >(altitude); //... }; // Data coming from measurements is marked untainted extern pos<untainted<int> >get_corridor_height(); untainted<pos<int> >a = get_corridor_height(); descend(a); // no negative altitudes here! returns a value that is both: positive and untainted a can hold positive, untainted values. order of qualifiers is not important! no negative altitudes can appear here!

15. Type System 2: XML Typing Objective: providing support for typing XML snippets in programs • Types can describe XML elements with certain structure • sequences, alternatives, elements etc. • Subtyping describes structurally more powerful types • types that can hold all the values of their subtype • Compile-time assurance that only valid XML documents are produced • when document schema changes, are we backward compatible with the old one? • Value-preserving type conversions • logically the same entities can be represented by different XML elements Solodkyy et al. LCSD’06

16. typedef element<name, string> XMLname; typedef element<email,string> XMLemail; typedef element<icq, int>XMLicq; typedef element<contact, boost::variant< XMLemail, XMLicq > > XMLcontact; typedef element<person, fusion::tuple< XMLname, XMLcontact > > XMLperson; <xsd:elementname="name"type="xsd:string"/> <xsd:elementname="email"type="xsd:string"/> <xsd:elementname="icq"type="xsd:decimal"/> <xsd:elementname="contact"> <xsd:complexType> <xsd:choice> <xsd:elementref="email"/> <xsd:elementref="icq"/> </xsd:choice> </xsd:complexType> </xsd:element> <xsd:elementname="person"> <xsd:complexType> <xsd:sequence> <xsd:elementref="name"/> <xsd:elementref="contact"/> </xsd:sequence> </xsd:complexType> </xsd:element> Example we use a dedicated type element to represent XML elements we use a dedicated type element to represent XML elements for each tag we create a dedicated tag-type for each tag we create a dedicated tag-type we map XML data types into C++ types we map XML data types into C++ types back references are mapped to previous typedefs back references are mapped to previous typedefs back references are mapped to previous typedefs XML Schema’s choice is mapped to Boost variant XML Schema’s choice is mapped to Boost variant XML Schema C++ XML Schema’s sequence is mapped to Fusion’s tuple XML Schema’s sequence is mapped to Fusion’s tuple

17. Tel <: AnyContact ICQ <: AnyContact Name, Tel, ICQ <: Name, AnyContact, AnyContact Person <: PersonEx Example Instantiate an XML snippet that corresponds to Person type // ... typedef variant<Email,Tel,ICQ> AnyContact; typedef element<person, tuple<Name, Tel, ICQ> > Person; typedef element<person, tuple<Name, AnyContact, AnyContact> > PersonEx; int main() { Person p(make_tuple(Name("Yuriy"),Tel("555-4321"),ICQ(1234))); PersonEx x = p; // OK: Subtyping conversion // p = x; // ERROR: Not in subtyping relation ifstream xml("old-person.xml"); xml >> x; // read data from XML file. assumes file exist cout << x << endl; // show XML source on the screen } Person <: PersonEx Assignment involves subtype conversion PersonEx is not a subtype of Person Parses only XML files that correspond to PersonEx schema Produces XML source on the screen

18. XDuce • Type • set of sequences over a certain domain • Regular Expression Types • concatenation : A,B • alternation : A|B • repetition : A* • optional : A? • type construction : l[A] • recursion : X = A,X | ø • Subtyping • inclusion between the sets defined by types

19. C++ • Type • set of sequences over a certain domain • Regular Expression Types • concatenation : A,B tuple<A,B> • alternation : A|B variant<A,B> • repetition : A* vector<A> • optional : A? optional<A> • type construction: l[A] element<l,A> • recursion : X = A,X | ø– • Subtyping • is_subtype and subtype_cast Objective: similar representation and the same semantics as in XDuce

20. XTL’s Strengths • Simplicity • reasonably powerful type systems can be built without creating a language tool • Genericity • common interface for defining custom subtyping relation • common interface for defining conversion from a subtype to a supertype • Reusability • ready definitions to be used in other type systems • subtyping of array types • subtyping of function types • subtyping of sequences and union types • subtyping of qualified types Solodkyy et al. LCSD’06

21. XTL’s Limitations • Unable to take information about control-flow into account • if (x>0) … does not make the type of x - pos<int> • Slows down compilation on complex type systems • e.g. XML types • Scaling problems on complex type systems • XML typing is exponential • No implicit transitivity of subtyping relation • library has no access to all available types • Meta-function join may return an arbitrary upper bound • same reason – no global information on all types Solodkyy et al. LCSD’06

22. THANK YOU! Abstract Questions deserve Abstract Answers

23. Compilation Times XML Typing Type Qualifiers