Testing Atomicity of Composed Concurrent Operations

1. Testing Atomicity of Composed Concurrent Operations Ohad ShachamTel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University Mooly Sagiv Tel Aviv University Martin Vechev ETH & IBM Research Eran Yahav Technion

2. Concurrent Data Structures Writing highly concurrent data structures is complicated Modern programming languages provide efficient concurrent collections with atomic semantics … … … … … … .. … …

3. Challenge Testing the atomicity of composed operations … … … … … … … …

4. TOMCAT Motivating Example TOMCAT 5.* TOMCAT 6.* attr = new HashMap(); … Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val; } attr = new ConcurrentHashMap(); … Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val; } Invariant: removeAttribute(name) returns the value it removes from attr or null

5. removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”); o

6. removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.put(“A”, o); attr.remove(“A”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; attr.remove(“A”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; attr.put(“A”, o); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”); attr.remove(“A”); Linearizability null o o null null o null null null o o null

7. Challenge Testing the linearizabiliy of composed operations … … … … … … … …

8. removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”); • Reconstruction in TOMCAT extremely challenging • Large traces • Large number of traces • Bugs occur in rare cases with specific key values

9. Our Solution Generates simple traces Modularity Enables Env control Base linearizability Restrict generated traces Influence Restrict generated traces

10. Modular Checking removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”);

11. Challenge Testing Linearizability of composed operation in a modular fashion … … … … … … … …

12. removeAttribute(“O”) { • Attribute val = null; • found = attr.containsKey(“O”) • if (found) { • val = attr.get(“O”); • attr.remove(“O”); • } • return val; removeAttribute(“GGG”) { Attribute val = null; found = attr.containsKey(“GGG”) if (found) { return val; • removeAttribute(“GGG”) { • Attribute val = null; • found = attr.containsKey(“GGG”) • if (found) { • val = attr.get(“GGG”); • attr.remove(“GGG”); • } • return val; • removeAttribute(“K”) { • Attribute val = null; • found = attr.containsKey(“K”) • if (found) { • val = attr.get(“K”); • attr.remove(“K”); • } • return val; • removeAttribute(“P”) { • Attribute val = null; • found = attr.containsKey(“P”) • if (found) { • val = attr.get(“P”); • attr.remove(“P”); • } • return val; removeAttribute(“K”) { Attribute val = null; found = attr.containsKey(“K”) if (found) { return val; removeAttribute(“B”) { Attribute val = null; found = attr.containsKey(“B”) ; if (found) { return val; • removeAttribute(“LL”) { • Attribute val = null; • found = attr.containsKey(“LL”) • if (found) { • val = attr.get(“LL”); • attr.remove(“LL”); • } • return val; • removeAttribute(“L”) { • Attribute val = null; • found = attr.containsKey(“L”) • if (found) { • val = attr.get(“L”); • attr.remove(“L”); • } • return val; • removeAttribute(“G”) { • Attribute val = null; • found = attr.containsKey(“G”) • if (found) { • val = attr.get(“G”); • attr.remove(“G”); • } • return val; • removeAttribute(“R”) { • Attribute val = null; • found = attr.containsKey(“R”) • if (found) { • val = attr.get(“R”); • attr.remove(“R”); • } • return val; removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; • removeAttribute(“P”) { • Attribute val = null; • found = attr.containsKey(“P”) • if (found) { • val = attr.get(“P”); • attr.remove(“P”); • } • return val; attr.put(“O”, o’); attr.put(“K”, o’); attr.put(“G”, o’); attr.put(“L”, o’); attr.put(“A”, o); attr.put(“R”, o’); attr.put(“GGG”, o); attr.put(“P”, o’); attr.put(“P”, o’); attr.put(“LL”, o’); attr.put(“K”, o’); attr.remove(“R”);

13. Influence Base Environment removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”);

14. Running Example Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }

15. removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”); Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }

16. attr.put(“A”, o); attr.remove(“A”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { return val; attr.put(“A”, o); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; attr.put(“A”, o); attr.remove(“A”); attr.remove(“A”); removeAttribute(“A”) { Attribute val = null; found = attr.containsKey(“A”) ; if (found) { val = attr.get(“A”); attr.remove(“A”); } return val; null attr.put(“A”, o); o attr.remove(“A”); o null null o null null o null o null

17. program COLT CO extractor library spec Timeout Non-Lin candidateCOs CO instrument linearizability checking Execution key/value driver influence driver

18. Benchmark • Used simple static analysis to extract composed operations • 19% needed manual modification • Extracted 112 composed operations from 55 applications • Apache Tomcat, Cassandra, MyFaces – Trinidad, etc… • Extracted all composed operations per application • We did not find additional composed operations • Using Google Code and Koders

19. 112Unknown

20. 59Non Linearizable 53Unknown

21. 17Open Non Linearizable 53Unknown 42Non Linearizable

22. 17Open Non Linearizable 27Linearizable 42Non Linearizable 26Globals

23. Globals … … … … … … … … m.put(k,v) … … … … … … … … … … …

24. 27Linearizable 85Non-linearizable

25. Easy Detection Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val; }

26. Current Work • Define a class of data-independent composed operations • Use small model reduction to prove linearizability of data-independent composed operations • Empirical study for the ratio of real life data-independent composed operations

27. Summary Writing concurrent data structures is hard Employing atomic library operations is error prone Modular linearizability checking Leverage influence Sweet spot Identify many important bugs together with a traces showing and explaining the violations Hard to find Simple and efficient technique