Selective Open Recursion Modular Reasoning about Components and Inheritance

1. Selective Open RecursionModular Reasoning aboutComponents and Inheritance Jonathan Aldrich Carnegie Mellon University Kevin Donnelly Boston University

2. Outline • The Fragile Base Class Problem • Selective Open Recursion • Implementation & Analysis • Discussion

3. Inheritance and Information Hiding • Parnas’ advice: • Modularize a system to hide information that may change

4. Inheritance and Information Hiding • Parnas’ advice: • Modularize a system to hide information that may change • Research question • How to hide information in component-based systems with inheritance?

5. A Challenge: Open Recursion • Recursion • Objects can make self-calls to their own methods

6. A Challenge: Open Recursion • Recursion • Objects can make self-calls to their own methods • Open Recursion • Self-calls are dynamically dispatched • I.e., target of call is “open” and can be overridden • Beneficial in “template method” design patterns

7. class Window { void draw() { // draw background this.drawForeground(); } void drawForeground() { … } } class MyWindow { void draw() { super.draw(); } void drawForeground() { super.drawForeground(); ... } ... } Open Recursion Window draw drawFore

8. class Window { void draw() { // draw background this.drawForeground(); } void drawForeground() { … } } class MyWindow { void draw() { super.draw(); } void drawForeground() { super.drawForeground(); ... } ... } Open Recursion Window draw drawFore Window MyWindow draw draw drawFore drawFore

9. Hook Methods • Hook methods • Called by this when some event occurs • Intended as extension point for subclasses • Open recursion is necessary

10. Hook Methods • Hook methods • Called by this when some event occurs • Intended as extension point for subclasses • Open recursion is necessary • Other methods • Perform some task • May be overridden, but has no “event” semantics • Open recursion unnecessary • At best, minor convenience • Has potential to cause the Fragile Base Class Problem

11. class CountingSet extends Set { private int count; void add(Object elem) { super.add(elem); count++; } void addAll(Collection c){ super.addAll(c); count+=c.size(); } int size() { return count; } // other functions } The Fragile Base Class Problem Set CountingSet addAll addAll add add

12. class CountingSet extends Set { private int count; void add(Object elem) { super.add(elem); count++; } void addAll(Collection c){ super.addAll(c); count+=c.size(); } int size() { return count; } // other functions } The Fragile Base Class Problem Set CountingSet addAll addAll add add Implicit assumption: Set.addAll does not call Set.add If this assumption is violated (or changed), CountingSet breaks

13. The Fragile Base Class Problem • Definition (for this instance of FBC) • A class may depend on the calling patterns of a superclass, and break if these are changed

14. Two Solutions to the FBC • Document open recursion [Kiczales & Lamping, Steyaert et al., Ruby & Leavens] • Exposes information, rather than hiding it • Prohibits natural changes to superclass Use forwarding to avoid open recursion [Bloch,Szyperski] Gives up benefits of open recursion Can we get the benefits of open recursion without the reasoning costs?

15. Two Solutions to the FBC • Document open recursion [Kiczales & Lamping, Steyaert et al., Ruby & Leavens] • Exposes information, rather than hiding it • Prohibits natural changes to superclass • Use forwarding to avoid open recursion [Bloch,Szyperski] • Gives up benefits of open recursion Can we get the benefits of open recursion without the reasoning costs?

16. Two Solutions to the FBC • Document open recursion [Kiczales & Lamping, Steyaert et al., Ruby & Leavens] • Exposes information, rather than hiding it • Prohibits natural changes to superclass • Use forwarding to avoid open recursion [Bloch,Szyperski] • Gives up benefits of open recursion • Can we get the benefits of open recursion without the reasoning costs?

17. Outline • The Fragile Base Class Problem • Selective Open Recursion • Implementation & Analysis • Discussion

18. Selective Open Recursion Use open recursion only where necessary • Hook methods • Use open modifier on method • Expresses “hook method” intent • All calls to open methods dispatched dynamically • Just as in Java today

19. Selective Open Recursion Use open recursion only where necessary • Hook methods • Use open modifier on method • Expresses “hook method” intent • All calls to open methods dispatched dynamically • Just as in Java today • Other methods • Calls to non-open methods on this dispatched statically • The only change vs. Java • Hides internal calling patterns from subclasses • Other calls dispatched dynamically • open≠ virtual

20. class Set { void add(Object o) { // adds an element } void addAll(Collection c){ foreach (Object o in c) this.add(o); } } add is not open, so subclass cannot intercept and depend on the call. Set can change without affecting subclasses. Selective Open Recursion Examples

21. class Set { void add(Object o) { // adds an element } void addAll(Collection c){ foreach (Object o in c) this.add(o); } } add is not open, so subclass cannot intercept and depend on the call. Set can change without affecting subclasses. class Set { // invoked for each add op. open void add(Object o) { // adds an element } void addAll(Collection c){ foreach (Object o in c) this.add(o); } } add is open, indicating the developer’s intent to expose this method as a semantic event to subclasses. Any changes to class Set must preserve these semantics. Selective Open Recursion Examples

22. Design Principles • Non-open as default • Only use open recursion when explicitly intended

23. Design Principles • Non-open as default • Only use open recursion when explicitly intended • Annotate the method, not the call • Design intent is attached to operation

24. Do you have to change the language? • Coding guidelines • Never call a public method on this • Hook methods should be protected • Non-hook, protected methods should be final • Not our idea • Suggested by Ruby & Leavens • Used extensively in JDK libraries

25. Do you have to change the language? • Coding guidelines • Never call a public method on this • Hook methods should be protected • Non-hook, protected methods should be final • Not our idea • Suggested by Ruby & Leavens • Used extensively in JDK libraries • However— • This solution relies on programmer discipline • Language integration provides automated checking

26. Outline • The Fragile Base Class Problem • Selective Open Recursion • Implementation & Analysis • Future work and Conclusion

27. Implementation in Java • Extension to Barat compiler • Bokowski & Spiegel • Rewrite calls to non-open methods on this • Call a final method with the implementation for the current class • Available at http://www.archjava.org/

28. Open Recursion Inference • Analysis to compute open annotations • Method m in class C must be open if: • Method m’ in class C’ <= C calls this.m • Class C’’ <= C’ overrides m • Class C’’ inherits or super-calls C’.m’

29. Open Recursion Inference • Analysis to compute open annotations • Method m in class C must be open if: • Method m’ in class C’ <= C calls this.m • Class C’’ <= C’ overrides m • Class C’’ inherits or super-calls C’.m’ • Results may be imperfect • Assumes whole program is available • Misses open methods that are not overridden • May mark methods open that should be refactored to be non-open

30. Experiments • Ran open recursion inference • java.* packages in JDK 1.4.2 • Except java.nio, java.sql (see paper)

31. Experiments • Ran open recursion inference • java.* packages in JDK 1.4.2 • Except java.nio, java.sql (see paper) • Hypotheses • Open recursion is rarely needed • Selective open recursion enables more optimization

32. Frequency of Open Recursion • 9897 methods in our portion of stdlib • 246 (2.5%) of methods were inferred open

33. Frequency of Open Recursion • 9897 methods in our portion of stdlib • 246 (2.5%) of methods were inferred open • Maybe the stdlib doesn’t use inheritance • 1394 of these methods are overridden • Only 18% of these are open Open recursion is rarely needed Thus making it selective may enable substantial information hiding

34. Frequency of Open Recursion • 9897 methods in our portion of stdlib • 246 (2.5%) of methods were inferred open • Maybe the stdlib doesn’t use inheritance • 1394 of these methods are overridden • Only 18% of these are open • Open recursion is rarely needed • Thus making it selective may enable substantial information hiding

35. Optimization Potential • 22339 calls in our portion of stdlib • 6852 self-calls • 716 to open methods • Must be dynamically dispatched • 6136 to non-open methods • Can be inlined in our proposal (27% of total calls) • Inlining in Java would require private, final, or whole-program analysis

36. Outline • The Fragile Base Class Problem • Selective Open Recursion • Implementation & Analysis • Future Work and Conclusion

37. Future Work: Application to Formal Reasoning • Formalizing as Featherweight Java + module system

38. Future Work: Application to Formal Reasoning • Formalizing as Featherweight Java + module system • Goal: bisimulation-based proof technique for showing contextual equivalence of modules in the presence of inheritance

39. Future Work: Application to Formal Reasoning • Formalizing as Featherweight Java + module system • Goal: bisimulation-based proof technique for showing contextual equivalence of modules in the presence of inheritance • Use selective open recursion and other techniques to provide more information hiding • i.e. prove more programs equivalent

40. Conclusion • Open Recursion complicates reasoning • rarely used in practice • Selective open recursion • retains benefits of open recursion where needed • avoids fragile base class problem • can be efficiently inferred • may allow more optimization and reasoning

42. class Set { void add(Object elem) { // adds an element } void addAll(Collection c) { foreach (Object o in c) this.add(o) } ... class CountingSet extends Set { private int count; void add(Object elem) { super.add(elem); count++; } void addAll(Collection c){ super.addAll(c); count+=c.size(); } int size() { return count; } // other functions } Open Recursion Calls to this are dispatched to subclass Thus subclass can tell exactly when each method is called