1 / 15

Teaching and Training Developer-Testing Techniques and Tool Support

Teaching and Training Developer-Testing Techniques and Tool Support. Tao Xie (North Carolina State University) Peli de Halleux , Nikolai Tillmann , Wolfram Schulte (Microsoft Research). The Recipe of Unit Testing. Three essential ingredients (e.g., testing List.Add ( int )): Data

truly
Download Presentation

Teaching and Training Developer-Testing Techniques and Tool Support

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Teaching and Training Developer-Testing Techniques and Tool Support • Tao Xie (North Carolina State University) • Peli de Halleux , Nikolai Tillmann, Wolfram Schulte (Microsoft Research)

  2. The Recipe of Unit Testing • Three essential ingredients (e.g., testing List.Add(int)): • Data • Method Sequence • Assertions void TestAdd() { int item = 3; var list = new List(); list.Add(item); Assert.AreEqual(1, list.Count); }

  3. Parameterized Unit Testing [Tillmann&Schulte ESEC/FSE 05] • Parameterized Unit Test = Unit Test with Parameters • Separation of concerns • Data is generated by a tool (e.g., Pex) • Developer can focus on functional (algebraic) spec void TestAdd(List list, int item) { Assume.IsTrue(list != null); var count = list.Count; list.Add(item); Assert.AreEqual(count + 1, list.Count); }

  4. Test Generalization void TestAdd() { int item = 3; var list = new List(); list.Add(item); Assert.AreEqual(1, list.Count); } Required skill set Abstraction Generalization, Experimentation, Comprehension, … void TestAdd(List list, int item) { Assume.IsTrue(list != null); var count = list.Count; list.Add(item); Assert.AreEqual(count + 1, list.Count); }

  5. Teaching in University Settings • Graduate software testing course (CSC 712) at North Carolina State University • 2008, 2009, 2010 Fall semesters (20, 18, and 27 students) • Lectures: two 75-minute lectures per week • Mons: foundation/coverage criteria instructed based on “Introduction to Software Testing” by Ammann and Offutt • Weds: practical/hands-on testing techniques/tools, with Pex, Code Contracts, and Nmodel • Assignments/projects • Test generalization + writing new PUTs http://research.csc.ncsu.edu/ase/courses/csc712/

  6. Tools/Technologies used in Project Using industrial-strength tools and technologies (e.g., Pex) • reduce the “debugging” overhead imposed on both the students and teaching staff • give students experiences that they could immediately benefit from when they took on their industrial jobs Over using academic research prototypes • lack support for dealing with various types of real-world code features • encounter many faults in prototypes • lack timely technical support or fixing of reported faults

  7. Training in Industrial Settings • One/half-day tutorials (10-25 practitioners) • inside MS, e.g., internal training of MS developers • outside MS, e.g., invited tutorials at .NET user groups • Inside Microsoft (in a training lab at Microsoft) • Attendees using lab computers installed w/ Pex • Slides + demos + intensive hands-on exercises • Outside Microsoft (in a meeting room) • Attendees often not having computers installed w/ Pex • Slides + demos http://research.microsoft.com/pex/documentation.aspx

  8. Lessons Learned from Training in Industrial Settings • Set realistic expectations right away • Tools/technologies have limitations, not “panacea” • Need teach developers about limitations • Don’t try to change deeply ingrained beliefs • … at least not all at once • Example: Believers in TDD (Test-Driven-Development) • Emphasize how new techniques relates to existing ones, how they complement

  9. Comparison btw University and Industrial Settings • Trainees with different incentives of learning (inducing different emphasis on technology adoption) • Univ: earn good course grades, learning various skills • Industry: “come and watch”, learning what is going on • Training with different durations • Univ: one semester • Industry: full/half day

  10. Pex for Fun (Web-based Learning) http://pexforfun.com/ http://pexforfun.com/TheSocialClassroom

  11. Pex for Fun – Takeaway • Accessibility • Visual Studio vs. Browser • Active learning • Doing with feedback • Contributing back • Social aspects • Ranking • Livefeed • Competitive environment

  12. PexVisual Studio Power Tool • http://msdn.microsoft.com/en-us/vstudio/bb980963.aspx

  13. Thank you http://research.microsoft.com/pex http://codeplex.com/Pex https://sites.google.com/site/asergrp/

  14. Thank you http://research.microsoft.com/pex http://codeplex.com/Pex https://sites.google.com/site/asergrp/

  15. Desirable Project Characteristics • Testing real-world code (not just “toy” code examples) • Different projects for different teams • Allow learning from each other in real-world complications • Effort focused on testing, not code development • Fully exploit limited course period time • Student team selects a real open-source project to test • Equipped with existing conventional unit tests • Project milestones • Midterm: test generalization; final: new PUT writing • Drawback • Not “real” develop testing!

More Related