Program Synthesis for Automating Education

1. Program Synthesis for Automating Education Sumit Gulwani (sumitg@microsoft.com) Microsoft Research, Redmond

2. Program Synthesis: Why Today? Program Synthesis = Synthesis of executable code from user intent expressed using some constraints. • Enabling Technology is now available • Better search/logical reasoning techniques • SAT/SMT solvers • Faster machines (Good application for multi-cores) • A natural goal to enable people to easily leverage computational power.

3. Program Synthesis: Recent Success Our techniques can synthesize a wide variety of algorithms/programs from logic and examples. • Undergraduate book algorithms (e.g., sorting, dynamic prog) • [Srivastava/Gulwani/Foster, POPL 2010] • Program Inverses (e.g, deserializers from serializers) • [Srivastava/Gulwani/Chaudhuri/Foster, MSR-TR-2010-34] • Graph Algorithms (e.g., bi-partitenesscheck) • [Itzhaky/Gulwani/Immerman/Sagiv, OOPSLA 2010] • String Manipulating Programs(e.g. ”Viktor Kuncak”-> “Kuncak, V.”) • [Gulwani, POPL 2011] • Bit-vector algorithms (e.g., turn-off rightmost one bit) • [Jha/Gulwani/Seshia/Tiwari, ICSE 2010]

4. Bitvector Algorithms • Straight-line programs that use • Arithmetic Operators: +,-,*,/ • Logical Operators: Bitwise and/or/not, Shift left/right • Challenge: Combination of arithmetic + logical operators leads to unintuitive algorithms • Application: Provides most-efficient way to accomplish a given task on a given architecture

5. Examples of Bitvector Algorithms 1 0 1 0 1 1 0 0 Z Z & (Z-1) 1 0 1 0 1 0 0 0 Z 1 0 1 0 1 1 0 0 Z-1 & 1 0 1 0 1 0 1 1 Z & (Z-1) 1 0 1 0 1 0 0 0 Turn-off rightmost 1-bit

6. Examples of Bitvector Algorithms P25: Higher order half of product of x and y o1 := and(x,0xFFFF); o2 := shr(x,16); o3 := and(y,0xFFFF); o4 := shr(y,16); o5 := mul(o1,o3); o6 := mul(o2,o3); o7 := mul(o1,o4); o8 := mul(o2,o4); o9 := shr(o5,16); o10 := add(o6,o9); o11 := and(o10,0xFFFF); o12 := shr(o10,16); o13 := add(o7,o11); o14 := shr(o13,16); o15 := add(o14,o12); res := add(o15,o8); P24: Round up to next highest power of 2 o1 := sub(x,1); o2 := shr(o1,1); o3 := or(o1,o2); o4 := shr(o3,2); o5 := or(o3,o4); o6 := shr(o5,4); o7 := or(o5,o6); o8 := shr(o7,8); o9 := or(o7,o8); o10 := shr(o9,16); o11 := or(o9,o10); res := add(o10,1); [ICSE 2010] Joint work with Susmit Jha, SanjitSeshia (UC-Berkeley), Ashish Tiwari (SRI) and Venkie (MSR Redmond)

7. Pyramid of Technology Users Algorithm Designers Software Developers End-Users Most Transformational Target Students and Teachers Consumers of Program Synthesis Technology

8. Automating Education Make education interactive and fun • Automated problem solving (for students) • Provide hints • Point out mistakes and suggest fixes • Creation of teaching material (for teachers) • Authoring tools • Problem construction • Group interaction (for teachers/students) • Ask questions • Share annotations Domains: Geometry, Algebra, Probability, Mechanics, Electrical Circuits, etc.

9. Geometry Constructions Domain What is the role of PL + Logic + Synthesis? • Programming Language for Geometry • Objects: Point, Line, Circle • Constructors • Ruler(Point, Point) -> Line • Compass(Point, Point) -> Circle • Intersect(Circle, Circle) -> Pair of Points • Intersect(Line, Circle) -> Pair of Points • Intersect(Line, Line) -> Point • Logic for Geometry • Inequality predicates over arithmetic expressions • Distance(Point, Point), Angle(Line, Line), … • Automated Problem Solving • Given pre/postcondition, synthesize a straight-line program

10. Geometry Domain: Automated Problem Solving Automated Problem Solving Given pre/postcondition, synthesize a straight-line program Example: Draw a line L’ perpendicular to a given line L. Precondition: true Postcondition: Angle(L’,L) = 90 Program Step 1: P1, P2 = ChoosePoint(L); Step 2: C1 = Circle(P1,P2); Step 3: C2 = Circle(P2,P1); Step 4: <P3, P4> = Intersect(C1,C2); Step 5: L’ = Line(P3,P4);

11. Constructing line L’ perpendicular to given line L Step 1: P1, P2 = ChoosePoint(L); Step 2: C1 = Circle(P1,P2); Step 3: C2 = Circle(P2,P1); Step 4: <P3, P4> = Intersect(C1,C2); Step 5: L’ = Line(P3,P4); L’ P3 C2 C1 P1 L P2 P4

12. Examples of Geometry Constructions Bisect a given line. Bisect an angle. Copy an angle. Draw a line parallel to a given line. Draw an equilateral triangle given two points. Draw a regular hexagon given a side. Given 4 points, draw a square with each of the sides passing through a different point. Other Applications: New approximate geometric constructions 2D/3D planning problems

13. Synthesis Algorithm for Geometry Constructions Precondition: True Postcondition: Angle(L,L’) = 90 Step 1: P1, P2 = ChoosePoint(L); Step 2: C1 = Circle(P1,P2); Step 3: C2 = Circle(P2,P1); Step 4: <P3, P4> = Intersect(C1,C2); Step 5: L’ = Line(P3,P4); • Synthesis, in general, is harder than verification. • Synthesis Problem: Given pre/postcondition, synthesize a straight-line program • Verification Problem: Given pre/postcondition, and a straight-line program, determine whether the Hoare triple holds. • Decision procedures for verification of geometry constructions are known, but are complex. • Because of symbolic reasoning.

14. A simpler strategy for verification of Constructions Symbolic reasoning based decision procedures are complex. How about property testing? Theorem: A construction that works (i.e.,satisfies the postcondition) for a randomly chosen model of precondition also works for all models (w.h.p.). Proof: Objects constructed using ruler/compass can be described using polynomial ops (+,-,*), square-root& division operator. The randomized polynomial identity testing algorithm lifts to square-root and division operators as well !

15. Randomized Polynomial Identity Testing • Problem: Given two polynomials P1 and P2, determine whether they are equivalent. • The naïve deterministic algorithm of expanding polynomials to compare them term-wise is exponential. • A simple randomized test is probabilistically sufficient: • Choose random values r for polynomial variables x • If P1(r) ≠ P2(r), then P1 is not equivalent to P2. • Otherwise P1 is equivalent to P2 with high probability,

16. Synthesis Algorithm for Geometry Constructions Problem: Symbolic reasoning is hard. Idea #1: Leverage Property Testing to reduce symbolic reasoning to concrete reasoning. • Construct a random input-output example (I,O) for the problem and find a construction that can generate O from I. • Example: Construct incenter of a triangle. • If I chose my input triangle to be an equilateral one, then the circumcenter construction also appears to work! • Since incenter = circumcenter for an equilateral traingle. • But what are the chances of choosing an random triangle to be an equilateral one?

17. Synthesis Algorithm for Geometry Constructions Exhuastive Search Strategy: Given input objects I and desired objects O, keep constructing new objects from I using ruler and compass until objects O are obtained. Problem: Search blows up, i.e., too many (useless) objects get constructed. • Example: n points lead to O(n^2) lines, which leads to O(n^4) points, and so on…

18. Synthesis Algorithm for Geometry Constructions Problem: Search space is huge. • Idea #2: Perform goal-directed reasoning. • Example: If an operation leads to construction of a line L that passes through a desired output point, it is worthwhile constructing line L. • Mimics human intelligence. • For this to be effective, we need solutions with small depth. • Idea #3: Work with a richer library of primitives. • Common constructions picked up from chapters of text-books. • A search space of (small width, large depth) is converted into one of (large width, small depth). • Mimics human experience/knowledge.

19. Search space Exploration: With/without goal-directness

20. Problem Solving Engine with Natural Interfaces Problem Description inEnglish Natural Language Processing Problem Description asLogical Relation Synthesis Engine Solution asFunctional Program Paraphrasing Solution in English Joint work with: Kalika Bali, Monojit Chaudhuri (MSR Bangalore) Vijay Korthikanti (UIUC), Ashish Tiwari (SRI)

21. Useful modules powered by problem solving engine The next step is to architect several useful modules on top of the problem-solving architecture such as: • Interactive feedback to students • Provide hints • Point out mistakes and suggest fixes • Creation of teaching material (for teachers) • Problem construction • Authoring tools

22. Other Domains What domains should we prioritize for automation? • Mathematics • Algebra • Probability • Physics • Mechanics • Electrical Circuits • Optics • Chemistry • Quantitative Chemistry • Organic Chemistry

23. Electrical Circuits: Concept-specific solutions Joint work with: Swarat Chaudhuri (Penn State University) • Consider the problem of computing effective resistance between two nodes in a graph of resistances. • MATLAB implements Kirchoff’s law based decision procedure • Algebraic sum of the currents at any circuit junction = 0 • Sum of changes in potential in any complete loop = 0

24. Electrical Circuits: Concept-specific solutions Joint work with: Swarat Chaudhuri (Penn State University) • Consider the problem of computing effective resistance between two nodes in a graph of resistances. • Kirchoff’slaw based decision procedure is not useful for students who are expected to know only simpler concepts. • Solutions need be parameterized by specific concepts such as: • Series/Parallel composition of resistances • Symmetry Reduction • Wheatstone Bridge

25. Resistance Reduction Concepts Series Combination Parallel Combination Wheat-stone Bridge If R3/R1 = R4/R2, then VD = VB

26. Pyramid of Technology Users Algorithm Designers Software Developers End-Users Most Transformational Target Students and Teachers Consumers of Program Synthesis Technology

27. Automating Education: Long-term Goals Ultra-intelligent computer Model of human mind Inter-stellar travel 

28. References • Tutorial at FMCAD 2010 (Oct 20, Wednesday, 9 am) • “Dimensions in Program Synthesis” • (More) applications and methodologies/techniques for synthesis • A cool demo  • Technical report • “Synthesizing Geometry Constructions” by Sumit Gulwani, Vijay Korthikanti, Ashish Tiwari • Send email to sumitg@microsoft for copy of the draft.