Testing Atomicity of Composed Concurrent Operations

1. Testing Atomicity of Composed Concurrent Operations Ohad Shacham Nathan Bronson Alex Aiken Mooly Sagiv Martin Vechev Eran Yahav Tel Aviv University Stanford University Stanford University Tel Aviv University ETH & IBM Research Technion

2. Concurrent Data Structures • Writing highly concurrent data structures is complicated • Modern programming languages provide efficient concurrent collections with atomic semantics … … … … … … . . … …

3. Challenge Testing the atomicity of composed operations … … … … … … … …

4. TOMCAT Motivating Example TOMCAT 5.* TOMCAT 6.* attr = new HashMap(); attr = new ConcurrentHashMap(); … … Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val; } } Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val; Invariant: removeAttribute(name) returns the value it removes from attr or null

5. removeAttribute(“A”) { Attribute val = null; attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); attr.remove(“A”); } return val; o

6. Linearizability removeAttribute(“A”) { Attribute val = null; null attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); o attr.remove(“A”); attr.remove(“A”); } return val; o attr.put(“A”, o); null attr.put(“A”, o); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; attr.put(“A”, o); null removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.remove(“A”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; o null null attr.remove(“A”); null o o attr.remove(“A”); null

7. Challenge Testing the linearizabiliy of composed operations … … … … … … … …

8. removeAttribute(“A”) { Attribute val = null; attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); attr.remove(“A”); } return val; • Reconstruction in TOMCAT extremely challenging – Large traces – Large number of traces – Bugs occur in rare cases with specific key values

9. Our Solution Generates simple traces Modularity Enables Env control Base linearizability Restrict generated traces Influence Restrict generated traces

10. Modular Checking removeAttribute(“A”) { Attribute val = null; attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); attr.remove(“A”); } return val;

11. Challenge Testing Linearizability of composed operation in a modular fashion … … … … … … … …

12. removeAttribute(“P”) { removeAttribute(“O”) { removeAttribute(“LL”) { removeAttribute(“L”) { removeAttribute(“G”) { removeAttribute(“R”) { removeAttribute(“B”) { Attribute val = null; found = attr.containsKey(“B”) ; if (found) { found = attr.containsKey(“GGG”) found = attr.containsKey(“GGG”) if (found) { if (found) { val = attr.get(“K”); attr.remove(“K”); } } } } } } } attr.remove(“R”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; return val; if (found) { return val; val = attr.get(“GGG”); attr.remove(“GGG”); } return val; return val; return val; return val; return val; return val; return val; return val; } return val; removeAttribute(“GGG”) { Attribute val = null; Attribute val = null; Attribute val = null; Attribute val = null; found = attr.containsKey(“K”) found = attr.containsKey(“P”) if (found) { val = attr.get(“P”); attr.remove(“P”); attr.remove(“P”); attr.remove(“O”); attr.remove(“LL”); attr.remove(“L”); attr.remove(“G”); removeAttribute(“GGG”) { removeAttribute(“K”) { removeAttribute(“K”) { removeAttribute(“P”) { Attribute val = null; Attribute val = null; Attribute val = null; Attribute val = null; Attribute val = null; Attribute val = null; Attribute val = null; attr.put(“A”, o); attr.put(“GGG”, o); attr.put(“K”, o’); attr.put(“P”, o’); attr.put(“P”, o’); attr.put(“O”, o’); attr.put(“LL”, o’); attr.put(“L”, o’); attr.put(“G”, o’); attr.put(“R”, o’); found = attr.containsKey(“O”) found = attr.containsKey(“LL”) found = attr.containsKey(“L”) found = attr.containsKey(“G”) found = attr.containsKey(“R”) found = attr.containsKey(“K”) if (found) { return val; val = attr.get(“P”); val = attr.get(“O”); val = attr.get(“LL”); val = attr.get(“L”); val = attr.get(“G”); val = attr.get(“R”); found = attr.containsKey(“P”) if (found) { if (found) { if (found) { if (found) { if (found) { if (found) { attr.put(“K”, o’); attr.remove(“R”);

13. Influence Base Environment removeAttribute(“A”) { Attribute val = null; attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); attr.remove(“A”); } return val;

14. Running Example Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }

15. removeAttribute(“A”) { Attribute val = null; attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); attr.remove(“A”); } return val; Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }

16. removeAttribute(“A”) { Attribute val = null; null attr.put(“A”, o); found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); o attr.remove(“A”); attr.remove(“A”); } return val; o attr.put(“A”, o); null attr.put(“A”, o); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; attr.put(“A”, o); null removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.remove(“A”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; o null null attr.remove(“A”); o null o attr.remove(“A”); null

17. COLT program CO extractor library spec Timeout Non-Lin candidate COs CO instrument linearizability checking Execution key/value driver influence driver

18. Benchmark • Used simple static analysis to extract composed operations – 19% needed manual modification • Extracted 112 composed operations from 55 applications – Apache Tomcat, Cassandra, MyFaces – Trinidad, etc… • Extracted all composed operations per application • We did not find additional composed operations – Using Google Code and Koders

19. 112 Unknown

20. 59 Non 53 Unknown Linearizable

21. 17 Open Non Linearizable 53 Unknown 42 Non Linearizable

22. 17 27 Open Non Linearizable Linearizable 42 Non 26 Globals Linearizable

23. Globals … … … … … … … … … m.put(k,v) … … … … … … … … … …

24. 27 Linearizable 85 Non-linearizable

25. Easy Detection Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }

26. Current Work • Define a class of data-independent composed operations • Use small model reduction to prove linearizability of data- independent composed operations • Empirical study for the ratio of real life data-independent composed operations

27. Summary • Writing concurrent data structures is hard • Employing atomic library operations is error prone • Modular linearizability checking • Leverage influence • Sweet spot – Identify many important bugs together with a traces showing and explaining the violations – Hard to find – Simple and efficient technique