1 / 29

How to Design and Implement Load Testing for Sakai

How to Design and Implement Load Testing for Sakai. The Michigan Experience. Why Load Test?. Measure end-user response time How long does a specified operation take? Currently a minor part of our testing Define optimal hardware configuration Enables pre-tuning of Oracle

channing-melendez
Download Presentation

How to Design and Implement Load Testing for Sakai

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How to Design and Implement Load Testing for Sakai The Michigan Experience

  2. Why Load Test? • Measure end-user response time • How long does a specified operation take? • Currently a minor part of our testing • Define optimal hardware configuration • Enables pre-tuning of Oracle • Testing various JVM settings • Check reliability • How long or hard can system work without producing errors (failures)? lmp@umich.edu

  3. Why Load Test.... • Check hardware & software upgrades • Does new host or software perform as well as previously tested versions? • Evaluate new products • Used to evaluate hardware for database replacement • Measure system capacity • How much can the system handle before performance degradation occurs? • Identify bottlenecks • What part of the environment causes performance degradation? lmp@umich.edu

  4. Test Environment • Performance Testing Tool • Infrastructure • Courses & Users • Data lmp@umich.edu

  5. Performance Testing Tool • Mercury Interactive LoadRunner • VuGen (virtual user scripting) • Controller (scenario and load generation) • Analyzer (analysis and reports) • Approach appropriate to any tool • Presentation will not focus on tool lmp@umich.edu

  6. Infrastructure • As close to Production as possible • Hardware • Software • Same configuration unless testing something specific • Authentication • Use application rather than CoSign • UM-developed, open-source single web login service • UMIAC • Warehouse for needed administrative data • UM-developed, PERL scripts accessing Peoplesoft system • Database servers not always identical • Where less “powerful,” larger host will likely perform better than observed in tests • AFS resource storage lmp@umich.edu

  7. Courses & Users • UMIACload provides course names, instructor owners, and rosters of students • 11,000 courses • 11,000 “instructors” • 43,000 “students” • Why so many? • To permit high level of activity without excessive concurrency lmp@umich.edu

  8. Data • Testing requires that all courses be pre-populated with materials to be used by students • Both AFS and Oracle data used • Saved versions of “base” data maintained to enable rollback to “clean” data • Always start with data refresh when adding new tools to test suite lmp@umich.edu

  9. Testing Process • Scripts • Scenarios • Analysis • Results lmp@umich.edu

  10. Script Setup • Analyze tool to be scripted • Understand tool usage • Value of user profiling • Design test plan • Goals • Single or multiple scripts? • Student or Instructor activity? • Are course materials required? • Design data files • Used by Virtual Users (VU) to change input parameters lmp@umich.edu

  11. Script Setup... • Create course materials • Populate courses with materials • Must be done sequentially to ensure all courses get materials • Multiple resources can be added to same course with same script • Need more efficient method • Record activity to be tested lmp@umich.edu

  12. Script Parameterization • Scripts recorded as single user • Load iterations need to change users each iteration • Data files used to manage user & input changes • All object-oriented elements in Sakai receive associated ID numbers that must change whenever a course or user changes • Recording rules used to manage ID number changes • Generic parameter identifiers changed to make analysis easier lmp@umich.edu

  13. Script Pseudocode - Chat • Login as “u10001” • Moves to “My Workspace” • Select course tab • Use “LOADTEST 102 001” • Select “Chat” tool from left-hand navigation column • Moves to Chat tool • Enter message into Chat text entry box • Return to “My Workspace” • Logout lmp@umich.edu

  14. Script Walkthrough - Chat Chat() { web_set_max_html_param_len("1024"); web_url("xlogin", "URL=https://ctoolsload.ds.itd.umich.edu/portal/xlogin", "Resource=0", "RecContentType=text/html", "Referer=", "Snapshot=t1.inf", "Mode=HTML", EXTRARES, "Url=../ctlib/skin/ctools/images/logo_inst.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/tool_base.css", ENDITEM, "Url=../ctlib/skin/ctools/images/topleft-p-title.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/tool.css", ENDITEM, "Url=../ctlib/skin/ctools/images/help.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/tool.css", ENDITEM, "Url=../ctlib/skin/ctools/images/topright-p-title.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/tool.css", ENDITEM, "Url=../ctlib/skin/ctools/images/reload_h.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/tool.css", ENDITEM, "Url=../ctlib/skin/ctools/images/chatpresicon.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/tool.css", ENDITEM, LAST);

  15. // [WCSPARAM WCSParam_UserID 36 b2de7f47-7c74-40d5-0024-98d191450a5d] Parameter {WCSParam_UserID} created by Correlation Studio web_reg_save_param( "WCSParam_UserID", "LB=site/", "RB=\"", "Ord=2", "Search=Body", "RelFrameId=1.1", LAST ); web_submit_form("xlogin_2", "Snapshot=t2.inf", ITEMDATA, "Name=eid", "Value={CTlogin}", ENDITEM, "Name=pw", "Value={CTpwd}", ENDITEM, "Name=submit", "Value=Login", ENDITEM, EXTRARES, "Url=../ctlib/skin/ctools/images/topright-tab.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/portal.css", ENDITEM, "Url=../ctlib/skin/ctools/images/topright-curtab.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/portal.css", ENDITEM, "Url=../ctlib/skin/ctools/images/topright-tab_hov.gif", "Referer=https://ctoolsload.ds.itd.umich.edu/ctlib/skin/ctools/portal.css", ENDITEM, LAST);

  16. web_link("LOADTEST {Course2} 001 W06", "Text=LOADTEST {Course2} 001 W06", "Snapshot=t3.inf", LAST); lr_think_time(5); // [WCSPARAM WCSParam_ChatID 36 194d7683-e9fc-41d3-0012-f0ac6fea0d60] Parameter {WCSParam_ChatID} created by Correlation Studio web_reg_save_param( "WCSParam_ChatID", "LB=title/", "RB=?", "Ord=1", "Search=Body", "RelFrameId=1.3", LAST ); web_link("Chat Room", "Text=Chat Room", "Snapshot=t4.inf", EXTRARES, "Url=/courier/194d7683-e9fc-41d3-0012-f0ac6fea0d60/1145294730027-0.2702248719455739?auto=true", "Referer=", ENDITEM, "Url=/courier/194d7683-e9fc-41d3-0012-f0ac6fea0d60/1145294740215-0.4220853133253527?auto=true", "Referer=", ENDITEM, "Url=/courier/194d7683-e9fc-41d3-0012-f0ac6fea0d60/1145294750293-0.2283065980084248?auto=true", "Referer=", ENDITEM, "Url=/courier/{WCSParam_UserID}-presence/1145294759434-0.23544053982281854?auto=true", "Referer=", ENDITEM, LAST);

  17. web_url("{WCSParam_UserID}", "URL=https://ctoolsload.ds.itd.umich.edu/portal/presence/{WCSParam_UserID}", "Resource=0", "RecContentType=text/html", "Referer=https://ctoolsload.ds.itd.umich.edu/portal/presence/{WCSParam_UserID}", "Snapshot=t5.inf", "Mode=HTML", EXTRARES, "Url=/courier/{WCSParam_ChatID}/1145294760356-0.31310668668084407?auto=true", "Referer=", ENDITEM, LAST);

  18. web_submit_data("{WCSParam_ChatID}", "Action=https://ctoolsload.ds.itd.umich.edu/portal/tool/{WCSParam_ChatID}?panel=Control", "Method=POST", "RecContentType=text/html", "Referer=https://ctoolsload.ds.itd.umich.edu/portal/tool/{WCSParam_ChatID}?panel=Control", "Snapshot=t6.inf", "Mode=HTML", ITEMDATA, "Name=message", "Value=Who me? Wadya want, meathead? sheesh.....", ENDITEM, "Name=eventSubmit_doSend", "Value=x", ENDITEM, "Name=send", "Value=Add message", ENDITEM, EXTRARES, "Url=/courier/{WCSParam_ChatID}/1145294761621-0.647618403375133?auto=true", "Referer=", ENDITEM, LAST); web_url("{WCSParam_ChatID}_2", "URL=https://ctoolsload.ds.itd.umich.edu/portal/tool/{WCSParam_ChatID}?panel=List", "Resource=0", "RecContentType=text/html", "Referer=https://ctoolsload.ds.itd.umich.edu/portal/tool/{WCSParam_ChatID}?panel=Main", "Snapshot=t7.inf", "Mode=HTML", LAST);

  19. lr_think_time(6); web_url("%7E{CTlogin}", "URL=https://ctoolsload.ds.itd.umich.edu/portal/site/%7E{CTlogin}", "Resource=0", "RecContentType=text/html", "Referer=https://ctoolsload.ds.itd.umich.edu/portal/site_tabs/{WCSParam_UserID}", "Snapshot=t8.inf", "Mode=HTML", LAST); return 0; }

  20. Scenarios • One or more scripts to be run in load test • Number of virtual users running script • Number of iterations per virtual user • Pacing (how much delay between iterations) • Think time (emulates user variability) • Browser emulation • Network bandwidth emulation • Results detail (step or full transaction level) lmp@umich.edu

  21. Instructor CreateAnnounce CreateAssign ReviseResource ScheduleEvent UploadLgFile UploadSmFile Student Chat CreateProject DownloadSmFile DownloadLgFile NewDiscussion ReadBrowse SubmitAssign 13 “Core” Tests lmp@umich.edu

  22. Instructor AddSoMod AddVzMod EditSyllabus EditWiki3L SubmitGrades Student ReadTxtMod ReadSoMod ReadVzMod ReadSyllabus ReadWiki3L ViewGrade “New” Tool Tests lmp@umich.edu

  23. Scenario Preparation • Single script runs • 1 VU • 100 iterations • No wait time between iterations • No think time • Showing response times for each step of transaction • Identify serious script problems • Take average response time for scenario lmp@umich.edu

  24. Scenario Iteration Timing (125%) lmp@umich.edu

  25. Analysis & Results • Scenario runs for 1 hour plus ramp up and down time • Upon completion of run, use LoadRunner Analyzer to crunch data • Compare resulting graphs with previous runs • Developers, DBA, SysAdmins look for performance bottlenecks • Fixes proposed and test rerun lmp@umich.edu

  26. CTload Scenario Results:2.0_r3636w8app (100%) lmp@umich.edu

  27. CTload Scenario Results:2.2.1_r14159w6app (125%) lmp@umich.edu

  28. Sharing the Burden? • Multiple campuses participate • Ideally a few dedicated partners • Localized hardware • Localized test tool choice • Centralized resources • Test design • Pseudocode for scripts • Course data files • Results • Support and discussion lmp@umich.edu

  29. Community Testing? • Share common core tests • Baseline for comparison • Focus on new tools of interest to campus • Contribute resources back to central repository • Contribute results to community • Greater stress on performance in development lmp@umich.edu

More Related