Bug Hunting

1. Bug Hunting Simon Polkowske Senior Seminar 2/26/07

2. Security Bugs • No one likes them • Their cost is calculated in the damage that is done • Its big news, not only exploiting bugs but just telling people they exist. • Has spawned “Responsible Disclosure”

3. Responsible Disclosure • Companies have asked bug hunters to wait a reasonable amount of time before publishing bugs • Releasing bugs before a patch is out can put users at risk • OR gives companies more time to waste while problem is not fixed says bug hunters

4. Open VS ProprietaryFIGHT! • Often debated over which leads to better quality of code. • Open source thought of as better. • Study done by Coverity showed • On average Open had less bugs per LOC • Top of the line Proprietary was more than 5 times better than best Open source

5. Proprietary Cheats • Open source enthusiast say this was not a fair test. • They argue that comparing mission critical software to software that is not critical is bad comparison • Comparing like software is better comparison • Ex: MS Internet Explorer vs Mozilla Firefox

6. Bug Hunter Unreality • “Only very bright, knowledgeable people can find security bugs.” • Helps to have experience, time is more important. • “There is always a shady motivation behind the search for security bugs.” • As many reasons as there are people • “Person that finds a security bug knows everything there is to know about it” • The full scale of security bugs is often missed.

7. Getting Your Bug • Some things to keep in mind • How many people you have • What their experience and technical skills are • How much time you have • What kind of tools at your disposal

8. Approaches • Depending on the resources there are a few different bug finding approaches • Lone Ranger Mode • Time-Constrained Peer Audit • Assembly-Line Teamwork • Tournament • Rotating Teams

9. Lone Ranger Mode • One person or more • Their sole responsibility is to find bugs • Best for long term searching

10. Time-Constrained Peer Audit • Two to three people • Looking where bugs are suspected • Similar to Extreme Programming techniques • Good for short time periods and when the group has similar skills and experience

11. Assembly-Line Teamwork • Two or more people • Good for groups with varying skills and time is short • Puts people on one specific area that they are good at

12. Tournament • Multiple Groups • Give the same thing to check to each group, first to find bugs (or the most bugs) wins • Good way to find numerous bugs in a short period • Can also rules in tournament that help establish a process • Fun way to keep moral up

13. Rotating Teams • Several Teams • Cycling the teams into the bug hunting hat • Ongoing process • Hard for bugs to hide from all those eyes • Doesn’t let people get bored doing just one thing

14. Criteria of Bug Hunting • A clear definition of the target technology • Know exactly what the software will be on • Example: a default installation of Windows XP Pro • Process Documentation • Everyone must understand what is being done • Important if there are people at different amounts of expertise or even different locations

15. Criteria of Bug Hunting (cont.) • Results Documentation • Helps people coming in after you • Diversification • Use a mixture of techniques and a team of people to hunt bugs

16. Techniques • Source Code Audit • Reverse Engineering: Debug & Disassembly • Reverse Engineering: Network Traffic • Black Box Testing • Brute Force • Top-Down Analysis • Information Gathering

17. Source Code Audit • RTFS – “Read the fine source” • Reading the source code looking for bugs and poor programming • Requires little knowledge of the system • Good to have experience in bug hunting • Also good to have the development team on hand

18. Reverse Engineering: Debug & Disassembly • Debugging • Team actively monitors the execution and inputs for bugs • Source code not necessary • Disassembly • Looks at the source code in assembly • The tool must know the underling technology • Only good on small and simple software

19. Reverse Engineering: Network Traffic • Needs to have network components • Good when the project is large and complex • Requires a tool to capture packets • Try to spot possible flaws • Usually requires a lot of experience

20. Black Box Testing • Manipulating the environment and inputs to produce bugs • Looking for usual bugs in type of software • Requires more experience than technical skills • Doesn’t have knowledge of how it works or access to source code

21. Brute Force • Black Box approach using tools. • Tries to test every possible input • Requires little to no expertise • Needs good tools

22. Top-Down Analysis • A review of the high-level area and will look deeper if something is suspect • Quick to yield results • Can lead to many dead ends

23. Information Gathering • “Quick and Lazy” • Searching for information on bugs • Helps to find bugs to verify

24. It Works • October 2000, MySQL authentication bug • Used a time-constrained source code audit and assembly-line teamwork • Problem was in their cryptography • June 1998, SSH CRC-32 insertion attack • Top-down overview and peer audit • Weak integrity checks allow for multiple attack attempts

25. Tools • Help fix programming styles and identify where bugs could occur • for C: • Lint (Sun Microsystems) • LCLint/Splint (University of Virginia) • for C++: • PC-Lint (Gimpel Software) • CodeWizard/C++Test (Parasoft) • for C#: • FxCop (Microsoft) • ClockSharp (TIOBE) • for Java: • CheckStyle (SourceForge project) • JCSC (SourceForge project) • JTest/CodeWizard (Parasoft)

26. Lint • A library that you can include in program • It will do • Analysis structure and flow of the source program • Analysis of control flow and data flow • Analysis of data types usage • Constant propagations and constant expression evaluations

27. Squashed Bug • No one bug hunting approach or technique is best • Mixing different approaches, techniques, and tools will better the chances of finding more bugs

28. Work Cited • Arce, Ivan. Bug Hunting: the Seven Ways of the Security Samurai. Core Security Technologies, 2002. 1-5. • Chelf, Ben. "Insecurity in Open Source." Viewpoint (2006). 25 Feb. 2007 <http://www.businessweek.com/technology/content/oct2006/tc20061006_394140.htm?campaign_id=bier_tco.g3a.rss1007>. • Cmot. "October 2006 Archives." Raw Matter. 25 Feb. 2007 <http://fortytwo.ch/blog/archives/2006-10.html#e2006-10-06T09_04_13.txt>. • Evers, Joris, and Marguerite Reardon. "Bug Hunters, Software Firms in Uneasy Alliance." CNET News.Com (2005). 25 Feb. 2007 <http://news.com.com/Bug+hunters,+software+firms+in+uneasy+alliance/2100-1002_3-5846019.html>. • Lemos, Robert. "Flaw Finders Go Their Own Way." CNET News.Com (2005): 1-2. 25 Feb. 2007 <http://news.com.com/Flaw+finders+go+their+own+way/2100-1002_3-5550430.html>.

29. Work Cited • "Ling Source Code Checker." Sun Microsystems. 25 Feb. 2007 <http://docs.sun.com/source/806-3567/lint.html>. • "Proof-of-Concept Code Increases Risk to Computer Users." Microsoft.Com. Microsoft. 25 Feb. 2007 <http://www.microsoft.com/security/incident/im_info.mspx>. • Rasch, Mark. "'Responsible Disclosure' Draft Could Have Legal Muscle." SecurityFocus (2002). 25 Feb. 2007 <http://www.securityfocus.com/columnists/66>. • Robers, Paul. "Do Bug-Hunting Security Firms Put Users At Risk?" IDG News Service (2002). 25 Feb. 2007 <http://www.pcworld.com/article/id,106517-page,1/article.html>.