The QA Focus Perspective

The QA Focus Perspective Marieke Guy UKOLN M.Guy@ukoln.ac.uk http://www.ukoln.ac.uk/qa-focus/ QA Focus – Supporting JISC's 5/99 Programme

The QA Perspective • If you want to sort out the problems on your Web site you can do so by developing a Quality Assurance Framework • This involves: • Testing • Problem tracking and corrective action • Policies • Procedures • Documentation QA Focus – Supporting JISC's 5/99 Programme

What Can You Test? • Functional testing • Compatibility testing • Load/performance testing • Stress testing • Usability testing • Security testing • Integration of unit testing • Link testing • HTML and CSS validation • Reliability testing • Regression testing • … QA Focus – Supporting JISC's 5/99 Programme

Automated Testing • Should be scalable • Only suitable approach for large sites • Can push (email) information about problems • Many free testing tools available • but: • Will not spot all usability/accessibility problems • Can provide too much information • Comprehensive and configurable testing tools can be expensive QA Focus – Supporting JISC's 5/99 Programme

Manual Testing • Can spot usability/accessibility problems • Can make use of one’s community/user base/peers • Can use of key pages which will identify problems which occur elsewhere • Useful for testing new Web sites • “5 users can spot most of your problems” • but: • Not suitable for testing large numbers of pages • Quality of feedback may be variable • Often not useful for testing established Web sites QA Focus – Supporting JISC's 5/99 Programme

Approaches to Automated Testing • Automated testing can make use of: • Web-based tools • Desktop tools • UKOLN has made use of Web-based testing tools: • Monitor accessibility, HTML, CSS, compliance, page size, links quality, nos. of links, etc. • Benchmarking approach provides comparison with one’s peers http://www.ukoln.ac.uk/qa-focus/surveys/ QA Focus – Supporting JISC's 5/99 Programme

Peer Review • “Give me comments on my Web site and I’ll also test yours” • Can help in community-building • Particularly useful for programmes in which project Web sites being developed at same time • But: • Can be difficult in processing unstructured responses • Need to manage process QA Focus – Supporting JISC's 5/99 Programme

Problem Tracking • There are three stages in a corrective action system: • The short term fix • The cure • Prevention of errors – identify potential causes of error by analysing data to setect trends which if allowed to continue would result in errors QA Focus – Supporting JISC's 5/99 Programme

Documentation • 3 Levels of documentation: • Policy e.g. Quality Manual • System e.g. Processes and procedures • Work practices e.g. instructions More detailed, more specific, applies to fewer people QA Focus – Supporting JISC's 5/99 Programme

Definitions • Policy A plan or course of action intended to influence and determine decisions • Procedure A way of performing or effecting something • Process A series of actions, changes, or functions bringing about a result QA Focus – Supporting JISC's 5/99 Programme

Documentation • Good documentation • Policies, procedures and audits • Define procedures for all processes • Requirements • Specifications • Mission statements or statement of direction • Define your audience • Document the testing suite and tools • Document authoring tools etc. • Document problems tracking • A Quality Management System (QMS)?? QA Focus – Supporting JISC's 5/99 Programme

Issues I • Goals for testing will be related to the kind of site you have • Testing time is limited – use automated tools • Documentation is critical – to make sure practices are repeatable • Implement your QA procedures slowly over time to avoid stifling productivity • QA planning at the start of development means less time fixing things at the end QA Focus – Supporting JISC's 5/99 Programme

Issues II • Motivating/persuading people to implement a QA Culture can be difficult – run workshops to show people what you are aiming for • Make sure you have a feedback mechanism for the procedures you are implementing • Having a great team/management is crucial QA Focus – Supporting JISC's 5/99 Programme

The QA Focus Perspective • QA Focus: • Funded by JISC • Currently provided by UKOLN and AHDS • Currently supports JISC's 5/99 programme • Staff: • Brian Kelly, Project manager • Marieke Guy, QA Focus officer, UKOLN • Hamish James, QA Focus officer, AHDS • Gareth Knight, QA Focus officer, AHDS QA Focus – Supporting JISC's 5/99 Programme

Approach Taken • Two possible approaches to ensuring compliance with standards and best practices: • Enforce • Inspect all project’s work • Strict auditing, with penalties for no-compliance • Encourage • Training for project staff • Developmental, explaining reasons for compliance, documenting examples of best practices and providing advice on implementation and monitoring The latter approach is preferable, especially in a HE context QA Focus – Supporting JISC's 5/99 Programme

QA Focus Work • Documentation and Advice • Briefing Papers • Case Studies • FAQs • Descriptions of tools and architectures for projects to implement best practices • In-house QA • Surveys • Providing Motivation • Workshops • Liasing • The QA Focus Toolkit QA Focus – Supporting JISC's 5/99 Programme

Documentation: Briefing Papers • Advisory briefing documents are being produced • These are: • Brief, focussed documents • Informed by findings of the surveys QA Focus – Supporting JISC's 5/99 Programme

Documentation: Case Studies • Case Studies are being commissioned • These are: • Written by projects themselves • Describe the solution adopted to a particular problem • Include details of lessons learnt – not just a press release! QA Focus – Supporting JISC's 5/99 Programme

Web Site Testing Tools • Full list of Web site testing tools available from database • Surveys run with a number of the tools • In the future there will be reviews of tools http://www.ukoln.ac.uk/qa-focus/qa-projects/ QA Focus – Supporting JISC's 5/99 Programme

Describing Tools and Approaches • As an example of approach which provides easier access to testing tools, see the ,tools approach deployed on UKOLN Web site. • This approach: • Applies to all resources on Web site • Covers HTML and CSS validation and various other tests • Some recursive apps (,rvalidate) • Easily implemented with single line redirect QA Focus – Supporting JISC's 5/99 Programme

In-house QA • Policies • Procedures • Audits • To date: • Web Standards - XHTML, CSS • Linking - links to and from • Accessibility • Usage Statistic • Metadata QA Focus – Supporting JISC's 5/99 Programme

Documentation and Advice I • We have: • Surveyed project Web sites and identified areas of lack of compliance with standards and best practices • Demonstrated examples of the potential importance of compliance for repurposing resources QA Focus – Supporting JISC's 5/99 Programme

Documentation and Advice II • In addition we need to provide: • Brief focussed advice on the standards • Information on how to monitor compliance • More Case studies on solutions deployed by projects themselves • Guidance on dealing with implementation difficulties and what to do when strict compliance is difficult to achieve • Reviews of testing tools QA Focus – Supporting JISC's 5/99 Programme

Surveying The Community • Surveys of project Web sites have been carried out in order to: • Obtain a profile for the community • Identify examples of best practices • Identify areas in which further advice is needed • Surveys included: • Accessibility • 404 error pages • HTML & CSS compliance QA Focus – Supporting JISC's 5/99 Programme

Survey Philosophy I • The surveys made use of freely-available Web-based tools: • Methodology is open • No software needs to be installed locally (apart from Web browser) • Findings can be reproduced • Latest results can be obtained by clicking on link to testing service QA Focus – Supporting JISC's 5/99 Programme

Survey Philosophy II • The surveys typically examined project entry points and not entire Web site as: • This page has the highest profile • The aim is to validate a methodology which can be deployed by projects themselves, not to test every page on behalf of the projects QA Focus – Supporting JISC's 5/99 Programme

Survey Findings • Findings available from http://www.ukoln.ac.uk/qa-focus/surveys/ QA Focus – Supporting JISC's 5/99 Programme

Providing Motivation • There is a need to explain why compliance with standards is important and to provide motivation for projects to update their tools, authoring procedures, etc.This may involve • More workshops • Evaluating project's QA plans • Liaising with projects over the requirements of the key QA procedures applicable to their work • Liaising with the Advisory Services over their provision of advice and address maintenance and feedback mechanisms • Overseeing QA work QA Focus – Supporting JISC's 5/99 Programme

Self Assessment Toolkit • We will be developing a self-assessment toolkit for projects to use, by individual projects or across project clusters • The toolkit will consist of: • Examples of QA procedures • Documented examples of use of testing tools • Self-assessment questionnaires • Advice on standards and best practices • Case studies, FAQs etc. http://www.ukoln.ac.uk/qa-focus/toolkit/ QA Focus – Supporting JISC's 5/99 Programme

QA for Digitisation • Do it once…..do it right: • Project is fundamentally dependent upon the quality of original product • Quality is the pre-requisite to preservation • Quality expectations will only grow • Delivery problems can be fixed, but capture problems normally can’t QA Focus – Supporting JISC's 5/99 Programme

QA For Web Sites • The Web is the main delivery mechanism for projects and services • An increasing awareness of the importance of accessibility, use of new devices (PDAs, WAP, e-books, …), repurposing of Web content (e.g. archiving) • But invalid HTML is still the norm • Look to standards – XHTML, XML, CSS, WAI • If proprietary formats need to be used, flag them and use in most open way • QA Focus is trying to explain why and how QA Focus – Supporting JISC's 5/99 Programme

Next Steps Next Steps • Extended Coverage We will be moving on from Web and digitisation to include other areas including: • Metadata • Multimedia • Software development • Deployment into service • … QA Focus – Supporting JISC's 5/99 Programme

Next Steps – Metadata • Thoughts on QA for metadata: • Syntax Check syntax on embedded DC • Cataloguing Rules Ensure projects have appropriate cataloguing rules • Interoperability Ensure metadata can interoperate with third parties • Fitness for purpose Ensure metadata is appropriate for its purpose • Change control Ensure architecture for managing metadata can cope with change control (cf Exploit Case Study) QA Focus – Supporting JISC's 5/99 Programme

Next Steps – Software • Thoughts on QA for software: • Specification Process for developing systems spec, agreeing spec with stakeholders, etc. • Methodology Documentation of systems development methodology (cf UML case study) • Testing Documentation of testing methodology • Deployment Awareness of deployment challenges QA Focus – Supporting JISC's 5/99 Programme

Limitations • There are a number of limitations to the work we have carried out so far: • Project Web sites have different purposes (information about the project; communications with project partners; project deliverables themselves; etc.) • Projects have different levels of funding, resources, expertise, etc. • Projects are at different stages of development (and some have finished) QA Focus – Supporting JISC's 5/99 Programme

QA Focus And You! • What relevance has QA Focus for you? • You can deploy QA Focus practices for your projects, etc • You can write case studies and advisory documents: • Seen to be working with JISC • We can promote your approaches and work • This will help instigate best practices within your group • You can develop QA procedures in innovative areas (e.g. OAI, thesaurii, ePrints, e-Books, etc.) which QA Focus can use QA Focus – Supporting JISC's 5/99 Programme

Questions Any questions? QA Focus – Supporting JISC's 5/99 Programme

The QA Focus Perspective