Welcome, Examiners!

1. Welcome, Examiners! Washington State Quality Award Return Examiner Training Webinar 2009

2. First things first! • THANK YOU! • WSQA’s mission: Improving the way we live, learn, and work in Washington by helping organizations improve through the use of the Baldrige Criteria for Performance Excellence

3. Objectives For This Class • Review Changes to the Baldrige Criteria • Strengthen Skills in Areas of Opportunity • Review Resources for Examiner Software

4. Agenda: • WSQA organization overview • Baldrige criteria changes • WSQA ethics review • WSQA process Full Vs Lite • Review process • Individual review worksheets • Consensus process • Improvements • Examiner Software • Summary

5. For a Successful Course… • Be an active participant • Give only constructive feedback • Be courteous to your colleagues • Honor diverse opinions • Seek to understand • Ask questions- Can use chat capability, also just ask • Have fun!!

6. Introductions • Name, organization, what you do • Questions and expectations? • WSQA or National examiner or applicant experience? • What do you like to do when you’re not working?

7. What is WSQA? • Created by State law to: • Promote excellence • Recognize achievement • Educate and train • Patterned after the Baldrige National Quality Award • Awards presented annually by the Governor • Awarded to private, public, and not-for-profit organizations in manufacturing, service, education, and healthcare

8. What is WSQA? WSQA is a 501(c)(3) nonprofit organization supported by • Corporate donations • Individual memberships • Workshop, collaborative and conference fees • Many (many, many!) volunteer hours

9. Who is WSQA? • Examiners • Lite Assessment • Full Assessment • Certification follows assessment • Senior Examiners • Level 3 Team Leads • Scorebook Editors • Panel of Examiner & Process Development (PEPD) Mentors • Panel of Judges • Board of Directors • Executive Director and support team

10. 2. Baldrige criteria changes

11. Baldrige criteria booklet Beginning in 2009 revised every 2 years

12. Baldrige 2007 Criteria Changes • Strategic advantages • Workforce engagement • Leadership- performance measures • Strategic Planning-innovation, strategic advantages

13. Baldrige 2008 Criteria Changes • Order of information in the booklet • Scoring system description • Diagrams added for core values and maturity levels • Added strategic advantages as a term to the Glossary

14. 2009 Criteria Changes • Customer Focus/ Engagement • Organizational Core Competencies • Sustainability and Social Responsibilities

15. Customer Focus/ Engagement • Customer commitment to your brand and offerings • Characterized by loyalty and willingness to make an effort to do business with you, willingness to do business with you

16. Core Competencies • Areas of greatest expertise • Strategically important • Provide market place advantage • May involve technologies, unique offerings, market place niche, business acumen

17. Sustainability • Organization’s ability to address business needs and agility and strategy to address the future • Considers external and internal factors • Considers impact on society and well being of environment, social and economic systems

18. Conflict of interest issues • WSQA seeks to avoid conflict of interest to protect process integrity • Conflicts are frequent! • First step in assignment process: determine any conflicts

19. Code of Ethical Conduct • Why are ethics of utmost importance to WSQA and this process?

20. 4. WSQA Full and Lite Process Comparison table

21. 5. WSQA Process Objective: Review key steps in the process. Review key process opportunities.

22. Stages of the Assessment Process

23. Preparing for independent review • Read the criteria • Read the application • Familiarize yourself with the online tool • All information on the WSQA website: webinars and manuals http://www.wsqa.net/extimetable.php • Check in with Team Lead on Key Factors and after entering category 1 and associated results

24. Independent review • Select the most relevant Key Factors (from list) • Read the criteria for the category • Read (and annotate) the relevant section of the application • Identify and record the applicant’s processes • Record observations. Note missing responses and significant observations • Recommend a scoring range for the category

25. Independent review steps - process • Review related criteria • Read the category under review, take notes • Select relevant key factors from list • Identify & record processes • Record observations on ADLI: • Approach • Deployment • Learning • Integration • Identify any significant or missing responses

26. Record observations on “ADLI” • “Approach” refers to • Methods used to accomplish the process • How appropriate the methods are to the item requirements • The effectiveness of their use • The degree to which the process is repeatable and is based on reliable data and information (i.e., systematic)

27. Record observations on “ADLI” “Deployment” refers to the extent to which • The approach is applied in addressing relevant and important item requirements • It is applied consistently • It is used by all appropriate work units

28. Record observations on “ADLI” “Learning” refers to • Refining the approach through cycles of evaluation and improvement • Encouraging breakthrough change to your approach (innovation) • Sharing refinements and innovations with other relevant work units and processes

29. Record observations on “ADLI” “Integration” refers to the harmonization of • Plans • Processes • Information • Resource decisions • Actions • Results • Analyses to support key, organization-wide goals

30. Record observations on “ADLI” Questions to ask in analyzing “Integration” • Do the individual components of a performance management system operate as a fully interconnected unit? • Is the approach aligned with the organizational needs identified in the Organizational Profile?

31. Process diagnosis: key concepts Integration examples: • Alignment of objectives and action plans with strategic challenges AND mission, vision, values • Alignment of product/service delivery methods with KEY customer requirements • Alignment of key process measures with KEY customer requirements

32. Process diagnosis: key concepts 1. Approach 4. Integration PROCESS 2. Deployment 3. Learning

33. Building the ADLI worksheet How would you score this single response?

34. Independent review steps - results • Review the Criteria to identify required results • Review the relevant process in the application to identify KEY results important to the applicant (use KF’s, application, your diagnosis) • Review the Results in the application • What IS there? • What ISN’T there that should be? • Evaluate & score on the basis of LeTCI

35. Results evaluation: key concepts Levels • Numbers that position results on a meaningful measurement scale • Permit evaluation relative to • Past performance • Projections • Goals • Appropriate comparisons

36. Results evaluation: key concepts • Trends • Numbers that indicate direction and rate of change • Provide a time sequence of performance • Require at least three data points • Criteria call for trend data on • Product, service performance • Customer, workforce satisfaction • Financial performance • Market performance • Operational performance (cycle time, productivity)

37. Results evaluation: key concepts Comparisons • External requirements • External benchmarks • Similar OR different organizations with similar characteristics

38. Results evaluation: key concepts • Integration • Results align with key factors, e.g., • Strategic challenges, • Workforce requirements • Vision, mission, values • Results presented for • Key processes • Key products, services • Strategic accomplishments • What examples can you think of? • Strong integration? • Not-so-strong integration?

39. Results evaluation: key concepts • Strong integration: results presented for • Key areas addressing strategic challenges, • Key competitive advantages • Key customer requirements • Not-so-strong integration: • Results missing for the above • Results presented that the Examiner can’t match to process items or Organizational Profile

40. “Key” results • Refers to elements or factors most critical to achieving the intended outcome • In terms of results, look for • Those responsive to the Criteria requirements • Those most important to the organization’s success • Those results that are essential elements for the organization to pursue or monitor in order to achieve its desired outcome

41. Break time!

42. Stages of the Assessment Process

43. WSQA Consensus: Step-by-step

44. WSQA Consensus: Step-by-step

45. WSQA Consensus: Step-by-step

46. Stage 2 Consensus • Webinar http://www.wsqa.net/extimetable.php • Comments from all Independent reviews • Ensure comments reflect scoring. - what is preventing applicant from moving to next level? Is this explained in the comments

47. Consensus preparation and call(s) • Each Examiner responds with agreement, suggestions for changes, or disagreement, stating rationales • Category Lead facilitates consensus • Team Lead and/or PEPD mentor leads final scoring discussion • Can take anywhere from 4-8 hours (depending on what?)

48. Consensus and conclusion • If using online tool during consensus, changes may be made in real time • If not, Category Leads update comments & scores after call according to consensus decisions • Category Lead notifies Team Lead and Scorebook Editor when done • Scorebook Editor compiles consensus scorebook with final edits, notifies WSQA • Consensus scorebook should be finalized ASAP

49. Comment, scoring considerations • “Benefit of the doubt” • Give credit for what’s in the application • Don’t penalize for incidental exceptions • Not every process must show complete “DLI” • Not every results example must include “TCI”

50. Comment, scoring considerations • For LITE, only one score for entire results Category 7 • Propose a score for results related to your categories • Be prepared to reach team consensus for all of Category 7 • Using scoring language can help the applicant understand comments and score