Conditional Correlation Analysis for Safe Region-based Memory Management

1. Conditional Correlation AnalysisforSafe Region-based Memory Management Xi Wang, Zhilei Xu, Xuezheng Liu, Zhenyu Guo, Xiaoge Wang, Zheng Zhang Microsoft Research Asia, Tsinghua University PLDI, June 9th2008, Tucson

2. Region-based Memory Management • Memory Region (a.k.a. Pool) is widely used • Apache web server • SVN(subversion) version control system • RC compiler • …

3. Region Usage • Allocate objects in regions • A region Owns objects • Destroy a region, delete all objects it owns r = region_create(); a = region_alloc(r); b = region_alloc(r); c = region_alloc(r); region_destroy(r); r Ownership relation a b c

4. Subregion • One region can be Subregion of its parent • Detroy a parent, destroy all its subregions • Parent lives longer than sub x Subregion relation y = region_create(x); z = region_create(x); w = region_create(y); region_destroy(x); y z Subregion relation w

5. Dangling pointer between regions • Object can Access object in another region • When pointee’s region get destroyed earlier, pointer become dangling make(r, table) { iterator = region_alloc(r); iterator.f = table; } iterator parent foo(parent) { sub = region_create(parent); table = region_alloc(sub); iterater = make(parent, table); region_destroy(sub); } Dangling pointer Access relation table sub

6. Harmful & Really exists (in svn…) • Further Deref. -- crash • No Deref. -- Pointer lives longer than necessary & cause memory waste make(r, table) { iterator = region_alloc(r); iterator.f = table; } iterator parent foo(parent) { sub = region_create(parent); table = region_alloc(sub); iterater = make(parent, table); region_destroy(sub); } Dangling pointer table sub

7. The problem is not easy • Correlated Relations • Ownership : Object - Region • Subregion : Region - Region • Access : Object - Object • Existing solution to safe region usage • reference counting • …

8. Solution: RegionWiz • To verify Object P mayaccess object Q, P’s owner region must be descendant of Q’s owner region. (consistent conditional correlation) • Static analysis to infer the ownership, subregion & access relations • Verify correlation, find dangling pointers • Context-sensitive, heap cloning

9. Highlights • Conditional correlation analysis framework for safe region usage • Context-sensitive analysis with heap cloning • Found bugs in real-world applications written in C (100+KLOC) • 12 dangling pointers in 6 software packages • 13 false alarms • Case study & experience

10. Framework program info into DB tables Source Code Context Clone Compiler plug-in (Phoenix in VC) Context-sensitive program info Ownership Subregion Access relations Relation Computation (datalog) Correlation Analysis (datalog) Dangling pointer Report Post processing

11. Extract Program Information program info into DB tables Source Code Context Clone Compiler plug-in (Phoenix in VC) Context-sensitive program info Ownership Subregion Access relations Relation Computation (datalog) Correlation Analysis (datalog) Dangling pointer Report Post processing

12. Extract Program Information • Extract program information into DB tables • Following analysis can be datalog inference • Example • Call(foo, make) • Call(make , region_alloc) foo( parent ) { make(…) } make( r , table ) { region_alloc(…) }

13. Context Cloning program info into DB tables Source Code Context Clone Compiler plug-in (Phoenix in VC) Context-sensitive program info Ownership Subregion Access relations Relation Computation (datalog) Correlation Analysis (datalog) Dangling pointer Report Post processing

14. Context Cloning make(r, table) { iterator = region_alloc(r); iterator.f = table; } foo bar foo(parent) { sub = region_create(parent); table = region_alloc(sub); iterater = make(parent, table); region_destroy(sub); } make bar(parent) { table = region_alloc(parent); iterater = make(parent, table); } ralloc

15. Context Cloning make(r, table) { iterator = region_alloc(r); iterator.f = table; } foo (1) bar (1) foo(parent) { sub = region_create(parent); table = region_alloc(sub); iterater = make(parent, table); region_destroy(sub); } make (1) make make make (2) bar(parent) { table = region_alloc(parent); iterater = make(parent, table); } ralloc (1) ralloc (2) ralloc ralloc ralloc ralloc ralloc (3) ralloc (4) Heap Cloning(Specialization)

16. Context Cloning make(r, table) { iterator = region_alloc(r); iterator.f = table; } foo (1) bar (1) make (1) make (2) foo(parent) { sub = region_create(parent); table = region_alloc(sub); iterater = make(parent, table); region_destroy(sub); } ralloc (1) ralloc (2) ralloc (3) ralloc (4) iterator(1) iterator (2) bar(parent) { table = region_alloc(parent); iterater = make(parent, table); } table (1) table (2)

17. Program Information -- Cloned • From now on program information tables are decorated with context information • Example • Call(bar - 1, make - 2) • Call(bar - 1 , region_alloc - 4) • Call(make - 2 , region_alloc - 3) bar (1) make (2) ralloc (3) ralloc (4)

18. Relation Computation program info into DB tables Source Code Context Clone Compiler plug-in (Phoenix in VC) Context-sensitive program info Ownership Subregion Access relations Relation Computation (datalog) Correlation Analysis (datalog) Dangling pointer Report Post processing

19. Relation computation using datalog • Datalogrules • How new relations can be computed from existing relations • Basic Rules for regions NewRegion( context ) :- Call( _ , “region_alloc” – context) NewObject( context ) :- Call( _ , “region_alloc” – context) • Context- & Field- sensitive Points-to analysis in ~20 rules

20. Compute the needed relations • Access(P , Q) :- Object P points-to Q through whatever field. • Own(Rgn , Obj) :- region_alloc(r) called, return value assigned to v, r points-to Rgn, v points-to Obj. • Subregion(Sub , Parent) :- region_create(u) called, return value assigned to v, v points-to Sub, u points-to Parent. Descendant(Des , Ans) :- Des=Ans , NewRegion(Des), NewRegion(Ans). Descendant(Des , Ans) :- Descendant(Des , r), Subregion(r, Ans).

21. Framework program info into DB tables Source Code Context Clone Compiler plug-in (Phoenix in VC) Context-sensitive program info Ownership Subregion Access relations Relation Computation (datalog) Correlation Analysis (datalog) Dangling pointer Report Post processing

22. Detect unsafe usage • Rule for detecting dangling pointer: PaccessQ; region1ownP; region2ownQ; not ( region1 is a descendant of region2 ).  Warning(P , Q) • Report Warning(P , Q) when there is some, or report that the correlation is consistent • Heuristics to filter out very unlikely warnings, reduce the total warnings

23. Framework program info into DB tables Source Code Context Clone Compiler plug-in (Phoenix in VC) Context-sensitive program info Ownership Subregion Access relations Relation Computation (datalog) Correlation Analysis (datalog) Dangling pointer Report Post processing

24. Experiment • Conducted on Intel Xeon 2.0GHz / 32G RAM • Latest stable version applications tested • RCC RC Compiler • Apache 2.2.6 HTTP web server & utilities • freeswitch 1.0b1 Telephony platform shell • jxta-c 2.5.2 P2P framework shell • lklftpd FTP server • SVN (subversion) 1.4.5 Version control system

25. Bugs found use APR ~ 200KLOC not count in

26. Time consumption & relation sizes • Current context: full call-path • Future work: better context definition They blow up because of Context cloning! Unacceptable! <1h acceptable

27. Case study – 1 (svn) • Region structure should be consistent with program logic • Iterator vs. Hash table • Request vs. Connection • RegionWiz can effectively find this kind of bugs iterator parent Dangling pointer table sub

28. Case study – 2 (rcc) • r1 and r2 are totally independent • Destroying r2 earlier will cause dangling pointer config r1 name r2

29. Case study – 2 (rcc) • r1 and r2 are totally independent • Destroying r2 earlier will cause dangling pointer • To use immutable string, it’s better to make a private copy in pointer’s own region config r1 copy of name name r2

30. Case study – 3 (svn) • Temporary unsafe usage • Often involves branches • Path-sensitivity • Dangerous as code evolves • Re-organize code to avoid even temporary unsafe usage svn_do_open(……) { lock = region_alloc(parent); if (Conditon) { hash = region_alloc(sub); lock.f = hash; } …… if (Condition) { lock.f = NULL ; } region_destroy(sub); }

31. Related work • Language support for regions • Reap [OOPSLA '2002], Cyclone [PLDI '2002], RC [PLDI '2001], Ownership types[PLDI '2003] • Correlation Analysis • Locksmith [PLDI '2006], Chord [PLDI '2006, POPL '2007] • Context-sensitive Analysis • bddbddb [PLDI '2004, PODS '2005]

32. Conclusion • Use memory regions safely is not trivial • RegionWiz can detect dangling pointers between regions through static conditional correlation analysis • RegionWiz is efficient to find real bugs in real applications & improve safety of region-based memory management • We believe the correlation analysis framework can solve other problems

33. Thank you! Q/A

34. Heuristics • For Warning(Pointer, Pointee) • Pointer’s type mismatch with pointee’s type – less possible • Pointer & pointee never allocated from the same region under some context – more possible • Examined 205 lower-ranked warnings, all but one are really false alarms

35. Limitations • Function pointer – standard inter-procedural propagation of function pointer values, but only propagate through variables & parameters, not through heap objects • Pointer arithmetic (variable as array index) not supported, just ignored • Limited thread support, no support for Asynchronous event, Callbacks, etc. • Heuristics bring unsoundness