1 / 82

540 -> 624 Members

540 -> 624 Members. m eetup.com/ achieverstech achievers.com/tech. Profiling & Tuning a Web Application. Kaelen Proctor. A bout Me. Joined A chievers in May 2010 Senior Software Developer @ A chievers Professional Experience: Java web applications w/Spring + Hibernate

jgonzales
Download Presentation

540 -> 624 Members

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 540 -> 624 Members meetup.com/achieverstech achievers.com/tech

  2. Profiling & Tuning a Web Application Kaelen Proctor

  3. About Me • Joined Achievers in May 2010 • Senior Software Developer @ Achievers • Professional Experience: • Java web applications w/Spring + Hibernate • JavaScript frameworks… without jQuery! • PHP with CodeIgniter • MySQL, MySQL, MySQL

  4. Profiling your Web Application

  5. So what if my site isn’t the fastest? • Response time directly relates to your business • In 2007 Amazon determined that a 100ms increase in load time would cause a 1% drop in sales • In 2009 Shopzilla decreased page load time from 6 to 1.2 seconds, which netted a 7-12% conversion rate increase! • The slower you can serve up pages the more frustrated your customers become

  6. What exactly is profiling? Profiling is a dynamic analysis of the time complexity, frequency/duration of function calls, or memory allocation of a program A profiling tool runs this analysis by instrumenting either the source or executable, through a variety of techniques including event hooks or dynamic recompilation

  7. Goals To show off some tools of the profiling trade To demonstrate how to use them effectively and identify the biggest “bang for you buck” bottlenecks To impress upon you the need to integrate profiling early and continuously into your development cycle

  8. Agenda • Profiling your application • What a single request looks like • The database (MySQL) • Application code (PHP) • In the Browser • Maintaining performance at scale • Load testing • Production monitoring

  9. Before you embark What is your performance goal? Where is that relative to today? What processes are necessary to maintain the goal?

  10. Profiling

  11. Profiling for a Web Application • Web applications are all about speed; how quickly a response can be sent and usable • On the app server, that means understanding the queries ran, 3rd party libraries, web APIs, and application code • Simple can sometimes be best • We use CodeIgniter (CI) here and its built-in request profiler is easy to use and extremely helpful

  12. Enabling the CI profiler • Drop this line anywhere before the controller ends: • $this->output->enable_profiler(true); • The output code injects the profiling content at the end of the <body> tag • Lets see what it looks like site: Special K’s Video Rentals

  13. CI profiler output

  14. How does the profile details help? • Great overview of what is occurring in the request • Queries executed is the most important aspect of a profile • Identification of long-running or duplicate queries • Adding timing benchmarks can give a lot of insight • Especially if you leverage a lot of 3rd party libraries or web services

  15. It’s more of a guideline • Most likely, you’re profiling on your dev machine with test data • No idea how the request will scale • No competition for resources (i.e. database) • Have you profiled all possible configurations? • Is anyone profiling or even paying attention to the details?

  16. … In my experience, they aren’t No matter how much documentation you write on how to profile and which tools to use, it will get dropped in crunch time Most developers didn’t even turn the profiler on

  17. Achievers Performance Header ™ Leveraging CI’s profiler, we tie the profiling summary with our performance targets Text is colour-coded on a linear scale from green to red as the further the request is from our targets Expanding the header shows the summarized performance details

  18. Shoving in their faces Now once any target’s threshold is passed, the header defaults to the expanded view

  19. Knowing is half the battle • Finding issues early => more time to fix • Always profiling => instant detection of a performance-killing change • But there is balance • “Premature optimization is the root of all evil” • Wait until a feature is working before making it work fast

  20. The Database (MYSQL)

  21. Database performance is critical It is the biggest shared resource your application contains Really slow queries will affect the speed of the entire database Scaling out your DB is not a simple task, so ensuring it isn’t bogged down is critical

  22. Finding the stragglers • First you need to identify the slow queries, so you can: • Manually review each query in your code • Profile every request and review each executed query • Let MySQL do the work with its slow query log • Let’s go with option #3

  23. Slow query log • When on, MySQL logs any query that runs longer than a threshold # of seconds • The log contains the total query time, lock time, and rows examined/sent • To enable, add to the MySQL config file: log-slow-queries=/var/log/mysql/slow-query.log long_query_time=0.1

  24. In the log

  25. pt-query-digest http://www.percona.com/doc/percona-toolkit/2.2/pt-query-digest.html Reads the slow query log and groups queries by their structure Outputs aggregated statistics on the whole log as well as for each query

  26. Digesting • All the Percona tools are Perl scripts, so execution is fairly straightforward • Usage (on unix/linux): • pt-query-digest /var/log/mysql/slow-query.log > digest.out • Options for specifying a date range, filter queries, writing the results to a DB, etc.

  27. Digest output

  28. Digest output

  29. Digest output

  30. Digest output

  31. Well, now what? • Now you have a great starting point for finding bottlenecks in your DB • Slow queries - run MySQL EXPLAIN • Refer to the tech talk by Dr. Aris Zakinthinos • Most likely it is missing indexes • De-normalization may be necessary • Protip: Use your biggest data sets when running explain

  32. Regurgitation • Running pt-query-digest once won’t solve all your database issues • Tuning your query performance is a never-ending process • Teach developers how to use EXPLAIN and optimize queries • Weekly reports using pt-query-digest to give visibility into DB performance

  33. Application code (PHP)

  34. The Devil is in the details Callgrind is a language agnostic command line tool that profiles the function calls in your application (through emulation) It generates callgrind files which can contain the entire call stack, and can be read to summarize what your app code is doing

  35. Grind output

  36. XDebug profiling to the rescue • Awesomely, XDebug writes callgrind files when profiling is enabled • This makes generating the grind files trivial in PHP • Just add to your php.ini: xdebug.profiler_enable=1 xdebug.profiler_output_dir = "/tmp/xdebug"

  37. WebGrind

  38. Other Grind Visualizers KCacheGrind was the original visualizer, which was ported as Windows as WinCacheGrind Regardless, all three aggregate the function calls into total # of calls and total cost(s) WebGrind is limited to a summary table, whereas the other two can display the full call tree

  39. Installing WebGrind • Installation: • Prerequisite: Install XDebug • Download zip from WebGrind’sGithub • Extract zip to folder accessible by webserver • Setup virtual host for WebGrind • WebGrind will read from the XDebug profiler output directory automatically • Open in browser and voila!

  40. Are we screwed? No! We can fix it, otherwise this wouldn’t be a good demo =) We ran into these performance issues with the OWASP library late in our security release

  41. Output encoding • Output encoding is a very expensive task • Simply put, the OWASP library encodes any non-alphanumeric character • It makes no assumptions on the incoming data, so ends up doing a lot of encoding detection and normalization before any real output encoding

  42. Digging into OWASP • How did we make it more efficient? • First, we installed WebGrind and started looking at exactly what OWASP was doing • We identified the functions that were taking too long or being called too often, and then dove into the code • A little elbow grease and trial/error later, we had it optimized and running smoothly

  43. Opcode caching PHP is an interpreted language, so with every request, the code is read from disk, parsed, and compiled into opcode before executing An opcode cache stores the compiled opcode so the first three steps are skipped Speeds up your application by 2-5 times! Options: APC, XCache, Zend Optimizer+

  44. Setting up APC • http://pecl.php.net/package/APC • Linux: • Install w/ PECL: pecl install apc-3.1.9 • Compile the extension yourself • Windows: Download pre-compiled binary from http://downloads.php.net/pierre/ • Enable by adding the extension in php.ini • Sit back and enjoy the performance boost

  45. Keep on Grindin’ Use WebGrind to summarize what your app code is doing; find the functions bottlenecking your application Make it second nature to profile your application code with WebGrind But for a quick boost, start using an opcode cache now and never look back!

  46. In the browser

  47. Wealth of information Improving browser performance is a well-documented subject A simple Google search will return thousands of results for how to optimize HTML, JavaScript, CSS, Images, HTTP requests, server settings, etc. We’ll focus on a couple easy-to-use tools that tell you exactly what you should do

  48. Developer tools Firebug + webkit developer tools The most important aspect of these tools to performance is the network/timeline tab Shows you all resource requests and their timings including blocking, waiting, receiving, and more Displays when the DOMContent and Load events are fired

  49. Yahoo’s YSlow and Google PageSpeed Browser extensions for Chrome + Firefox (sorry, IE) Analyzes a page request/response and offers best practices about how to improve performance Yahoo and Google know what they are talking about; follow the tools advice for the biggest wins in the shortest timeframe

More Related