I Can Slack Off, My Code’s Compiling

1. I Can Slack Off, My Code’s Compiling Comic and title care of XKCD.com

2. Who am I? Jeff Ward Lead Architect Orbus Gameworks IGDA Tools SIG Chair Author: The Toolsmiths www.thetoolsmiths.com

3. What this talk is about

4. The Big Question How much time and money is lost just waiting for a build?

5. How long is your build? To answer this question, you need to know:

6. How long is your build? • On average? • For a complete rebuild?

7. Our build(includes unit tests) • CPP Libraries (vs2003, 2005, 2008): 00:36:24 • C# Libraries: 00:00:27 • Java Libraries: 00:00:04 • Python Libraries: 00:00:01

8. Now that you know that… • How long is your average incremental build? • How many times, per day, is your programmer going to perform that build? • By how much can you improve that? • The difference is wasted time!!!

9. Why know that? It’s all about the Benjamins.

10. The Money Assumptions • A 52 week year • A 5 day work week • A $62k average salary (~$30/hr) • This does not include other overhead for a programmer.

11. The Programmer Burn Chart

12. This means, that over a 2 year project… …you can lose approximately $468,000, the cost of 2 lead programmers, to build times.

13. You Programmer Burn (cont.) • Two way to look at these numbers: • Money Lost (in programmer productivity time) • Money Saved (by increasing productivity time)

14. Saving Money, Saving Time • Faster builds save more than just build time • Keeps programmers from task switching • Keeps you in the groove • Faster builds encourage quick iteration • Which encourages more testing

15. Fixing Programmer Burn

16. Fixing Programmer Burn (Easy) • Parallelize your build (Incredibuild, distcc etc) • Very Little investment, can see huge returns • Little to no programmer involvement (unless you’re switching build systems)

17. Fixing Programmer Burn (Easy) • Dedicate more hardware • 2 programmers wasting 30 minutes per day pays for a beefy server. • Little to no programmer involvement, instant returns • 2 programmers wasting 30 minutes per day costs you $7800, which buys one beefy server

18. Story Time

19. Fixing Programmer Burn (Medium) • Fix dependencies, modularize code • Modularized code is faster to compile and easier to test • Reduce dependencies between modules. • Minimize unnecessary includes, use forward declarations. • Make your inheritance chains shallow.

20. Fixing Programmer Burn (Medium) • Use Precompiled Headers (when you can) • Don’t put every header into precompiled headers. It’s silly and it annoys programmers more than anything else • Only place header that are significantly large, used very frequently, and unlikely to change • Put in guards to avoid PCH dependence

21. Fix Programmer Burn (Hard) • Use a fast compiling or interpreted language in high level code, one that doesn’t require includes or linking. • C#, Lua, Python, JavaScript, Scheme, UnrealScript?

22. Fix Programmer Burn (Hard) • Refactor code into individually testable libraries • Makes much of your code reusable • Allows testing of single library units • Can shorten iteration time on that library / unit • Going too crazy with this can increase link times

23. Fix Programmer Burn (Hard) • Move libraries that don’t change out of your build process • Goes with modularizing your libraries. • Allows testing of individual modules to be issolated. • Beware this suggestion. Frequently teams that do this mistakenly think “rarely edited” means “never edited”

24. You’re so vain… Bad builds aren’t just about lost programmer time

25. Deployed Builds • Broken Builds vs. Bad Builds • Important to consider when they affect the productivity of the team • If a team can’t work because of a because the build is bad, this is all lost time.

26. Your Team Burn Chart

27. How do we prevent this?

28. Anyone’s Jeff’s Tenets of Good Builds • Be fast • Be automated • Keep the team informed • Leave little room for human error • Prevent “bad builds”

29. These points are things to strive for … for some of them, you’ll never get there.

30. Be fast • Use continuous integration to build on each check-in, unit test all builds, and analyze code. • Create up to date and optimized assets on each check-in • Build assets incrementally, use source control to trigger only specific builds in assets • Use dependency trees to figure out what needs to be rebuilt as a result of asset changes.

31. Be Automated UNLIKELY! • Have check-ins occur at the lowest possible point, automate the rest. • This means max, maya, psd. • Automate informing the team of changes • Automate testing • Unit, functional, approval, monkey • Automate approval and deployment of good builds • Where it makes sense

32. Corollary: Remove things that are slow • Integration tests • CPP: 00:09:14 • CS: 00:00:13 • Generator: 00:16:26 • Java: 00:15:44 • Smoke Tests • Developer Approval

33. Keep the Team Informed • How long until the next build? • What changes does the next build include? • Do I need this build? • The team should know builds are bad BEFORE they get it.

34. Human Error • Automate export and build. • This reduces human interaction, and prevents build inconsistencies. • Isolate mistakes from the rest of the team • We all make mistakes, we all check in bad code or bad assets • Keep these changes from dispersing in the wild, which would cause hours of wasted time.

35. Prevent Bad Builds

36. Create a Pipeline • A build is not one step! • Get from source control • Build • Automated Unit Tests • Automated Integration Tests • Deploy to team • QA Smoke Tests • User Tests • Deploy to testers • Deploy to beta • Deploy to production

37. The Pipeline

38. Per Project vs. Per Company • Most code optimizations are project based. • Pipelines can be reused

39. No nono, nothing about this adds up at all!

40. Your Programmer Burn • Creating your build system takes time. • Time saved > Time spent • This makes sense only on large projects losing lots of time to builds • Don’t slack

41. Build Complete http://www.jeffongames.com http://www.thetoolsmiths.com jeff@fuzzybinary.com