ITB/ITN751 Games Production Lecture 6 Quality Assurance

ITB/ITN751 Games Production Lecture 6Quality Assurance Ross Brown

Lecture Contents • Game Programming • Guest Lecture – Scott Cameron, Halfbrick

Readings • Textbook Chapters: • Chapter 11 • Websites: • http://www.igda.org/qa/index.php - IGDA QA Special Interest Group • http://www.sloperama.com/advice/lesson5.htm - some practical advice on game testing employment • http://www.gamasutra.com/php-bin/news_index.php?story=13338 – some thoughts on QA

Quality and Quantity • “Those who speak most of progress measure it by quantity and not by quality” – George Santayana

Infinity Ward Studio TourQuality Assurance http://youtube.com/watch?v=2e2xPRDQRB8

What is Quality Assurance? • The step of the game development life cycle that ensures a game is: • Bug free • Functional • Usable • Fun • Is similar to the usability testing utilised within other ICT industry sectors – testing cycles

Why so important? • To make sure the game is bug free • Improves acceptance of game • Makes sure that game concept is going to sell – compare to screenings for movies

And… • Builds rapport with your future audience via the use of bug testing groups • Games are very complex pieces of software, that will be run on many forms of hardware • While having a word processor crashing as a piece of software is bad

No Flow • Having a game crash is simply destructive of the immersion experience • Thus giving your game players a bad experience of the game

Piracy • Prevents piracy…most cracking requires a bug exploit • Prevents cheating – again, unless the codes are easter eggs, you can prevent cheats from misusing your game… • Check: http://videogames.lovetoknow.com/wiki/Category:Video_Game_Cheat_Codes http://en.wikipedia.org/wiki/Image:Jolly-roger.svg

Processes • Alpha Testing • Focus Group Testing • Beta Testing • Ongoing Bug Reports • Competitive Analysis • Publisher Testing

Pre-Alpha Testing • Pre-alpha code is not feature complete and is often full of bugs • Thus no testing pre-alpha by Q/A as pointless • Pre-alpha bug fixing is handled by development team as matter of course

Alpha Testing • Alpha is typically [Textbook] defined as code that is content and feature complete, but may have bugs • The bugs in this case may crash the system • This testing is done in house by Q/A team and ad hoc development meetings • Part of the overall development process defined by the timetable of the producer

Closed Beta Testing • Closed Beta – has all content features and assets complete • No crashing bugs • Usually used in focus groups with players, but not released to public

Focus Group Testing • Gather a group of people to play a beta of the game • Concentrates on the fun aspects of the game • May video the players expressions to correlate gameplay with emotional state of players

Open Beta • Halo 3 Anyone? • Beta is released to closed subscribers for play testing • Can report on discovered bugs • Can track information about the gameplay via network links to the home server http://www.clubskill.com/Game_News/3779

Bugs vs Gameplay • It should be noted here that the testing processes is divided up into bug discovery in code, and gameplay issues. • A program may be bug free and not crash • But it may not be an effective game

Software Bug Testing • Looking for software bugs that cause a number of effects: • Write specific test plans to look for bugs • Use regression testing to keep changes within test limits • Can use automated testing for these purposes, ie. little test programs which use the new code

Software Bug Testing • Seek to replicate circumstances the cause game to crash, and or crash the system. • Usual technical issues of memory problems, bad code logic, OBOEs – Out By One Errors • Game Mechanics Visual and Audio Effects – game runs fine but the appearance is affected, eg. Holes in objects, textures destroyed, passing through walls etc.

Gameplay Testing • Designers/Balancers and other development team members have interests in getting the gameplay right • Thus Q/A is also about assessing the Quality of the gameplay as well as the Correctness of software code • Game Development Team is usually interested in a number of gameplay factors

Some Gameplay Factors Assessed • Mission – uptake and completion • Content Usage – reaction to, amongst other things, rewards, weapons, economy • Player vs Player – balancing factors, are the players able to fight fairly • Difficulty – levels and puzzles balance of challenge and ease of completion

Some Gameplay Factors Assessed • Social Factors – chat usage, social locations • Emergent Gameplay – social hangouts, novel use of weapons • User Interface – How players configure their games

Publisher/Manufacturer Testing • Console companies have VERY stringent testing regimes regarding the release of games • Final CD/DVD of game is submitted to publisher/manufacturer to test • May not pass on first round, and requires resubmission

Technology • Typical collaboration tools: wikis, spreadsheets, or web tools for communication of bug reports • Bug tracking software • Integrated into engine/game build processes within the development teams – use of live and development versions of games

Technology • Video camera for player focus testing • Benchmarking tools – to test the game to a predefined standard of frame rates on defined hardware profiles • Integrated Version Control Systems • version control systems (eg. Subversion, Alienware) allow for the tracking of versions of documents, especially content and software • also known as Digital Asset Management Systems

Technology • Debug builds of game engines for logging of data • Bug reporting tools as spreadsheets or web-based tools for reporting bugs to development team • Visualisation software for test data to see patterns

Skills • Is often the entrance point for people into the game industry • Game producers often come from a QA background • Does not require specific training - we don’t have a QA minor in this degree ;-) • Design/Culture/Business units in Games Degree are probably the best point to gain knowledge • Also play games and volunteer for game testing focus groups

Skills • Enthusiasm – checking holes in walls is boring after all  • Deep understanding of gameplay and what is fun • Thorough approach to playing and checking game environments – playing and checking are two different things

Skills • People skills for communicating bugs to other development team members – polite positive criticism • People skills for receiving criticism from focus groups and beta testers – some criticism is not going to be nice

Roles • Lead QA – manages the complete testing team and testing/QA processes under supervision of the producer and is part of the development team • Assistant QA – Manages testing of components of game • Testers – who do the wall crawling etc. • Producer/Designer/Lead Developer – interface to development team

Roles and then some… • Roles change depending on at what level the testing is performed • With the latest SCRUM and Agile software methodologies, there is an emphasis on unit testing as a way of ensuring quality code • Each unit of code of only accepted after its accessed as being complete

Assignment 2: Sim Game Studio • Sim Game Studio • Team Assignment • Gain an understanding of setting up and running a new studio for three years

Assignment 2: Sim Game Studio • Team worker set up • You will lose marks if not in team by • IMPORTANT: Form teams NOW and have regular meetings – like right this minute during the break!

Guest Lecturer • Scott Cameron – Quality Assurance • Halfbrick Studios

References • Secrets of the game business / edited by François Dominic Laramée, Hingham, MA: Charles River Media, c2005 – has a section on Quality Assurance in QUT Library • Dasso, Aristides Verification, Validation and Testing in Software Engineering, Hershey, PA: Idea Group, Inc., 2006 – online book via QUT library on software testing, as there are no actual books on game QA proper

ITB/ITN751 Games Production Lecture 6 Quality Assurance