California Team Excellence Award CTEA 2011 Judges Training

1. California Team Excellence Award (CTEA) 2011 Judges Training 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 1

2. 2011 CTEA Judges Training Your participation is a critical part of encouraging and recognizing team excellence in California! Thank you for your service! Outcomes - What Will You Learn? How the CTEA Award Program works. How to evaluate and score team presentations. How to reach consensus among the members of your judging panel. How to write effective feedback comments to help teams and their organizations improve. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 2

3. Two-Phase Training This training is required of all CTEA judges for the 2011 cycle, even those who have had prior experience as CTEA judges. Phase 1 � Perform this self-administered training prior to Phase 2. Phase 2 � Join one of several scheduled judges conference calls. Dates are listed in Lesson 4 and on the website: http://www.calexcellence.org/cteajudges.html 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 3

4. Outline of Phase 1 Training Lesson 1: Overview of the CTEA Program and the Judging Activities. Lesson 2: How to Score a Team�s Project Presentation. Lesson 3: How to Write Effective Feedback Comments. Lesson 4: Next Steps � Phase 2. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 4

5. Assemble the Necessary Materials Before proceeding download all the documents listed below from the CTEA Judges Information webpage so you can refer to them during this training. http://www.calexcellence.org/cteajudges.html CTEA Program Brochure.pdf (on CTEA homepage) ASQ-ITEA Award Criteria.pdf (on Team Resources page) ASQ-ITEA Glossary of Terms.pdf (on Team Resources page) CTEA Judges Workbook.xls ASQ-ITEA Scoresheet Sample.pdf ASQ-ITEA Presentation Sample.pdf CTEA Judges Guidelines.pdf CTEA Feedback Report Template.doc ASQ-ITEA Feedback Report Sample.pdf 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 5

6. Lesson 1 Overview of the CTEA Program and the Judging Activities 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 6

7. State & International Recognition Programs (1) International Team Excellence Award (ITEA) This Is administered by the American Society for Quality (ASQ). http://wcqi.asq.org/team-competition/index.html There is an annual cycle with a live competition and award ceremony at ASQ�s International conference in May. California Team Excellence Award (CTEA) California Council for Excellence (CCE) administers the CTEA. www.calexcellence.org Several countries and states administer local award programs that generally mirror the international program. There is an annual cycle with a judging event in October and an awards ceremony the following spring. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 7

8. State & International Recognition Programs (2) The CTEA Program is officially affiliated with the ITEA Program. CTEA uses the ITEA criteria for judging. CTEA also functions as a feeder program in which one or more high-scoring CTEA teams are eligible to compete as finalists in the ITEA Program. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 8

9. CTEA Mission Statement We promote overall effectiveness through team-focused management. We provide the training, tools, knowledge, and assessment skills needed to excel through team participation, recognition, and celebration. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 9

10. CTEA Program Review the CTEA Program Brochure This brochure explains how the CTEA program operates. Key points are that you will be assessing a team presentation of an improvement project that: Covers one completed team project from the previous 24 months; Describes how the team addressed the ITEA criteria during the course of their project; Lasts no more than 30 minutes; Is usually a live presentation but can be in DVD form. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 10

11. Benefits of Being a CTEA Judge Observing use of tools and methods you can apply in your own organization. Honing your knowledge and skills about what constitutes best practice in team excellence as embodied in the ITEA Criteria. Networking with fellow judges from various sectors who are highly competent professionals in the application of team-based continuous improvement. Obtaining recognition from your own organization for achieving the status of CTEA judge. Being energized by seeing the outstanding results accomplished by these teams as well as the professional and personal impacts these achievements have on team members. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 11

12. Your Tasks as Judges Judges have two primary tasks: Score the presentation according to the criteria, striving for accuracy so that the scoring is done consistently across judges. Write actionable and non-prescriptive feedback comments that will be useful to the team and their sponsoring organization. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 12

13. Your Obligations as Judges Having no conflict of interest with the organization(s)/team(s) that you are judging. Applying only the ITEA criteria and not applying any other opinions or professional requirements. Holding the information confidential about which team/organization(s) you judged and the contents of their presentations. Be willing to reach consensus with your fellow judges by being open to other points of view. Giving teams the benefit of the doubt. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 13

14. Organization of Judges The judges are placed into panels of 3-5 people. The composition is balanced with people of varying degrees of judging experience and varying organizational backgrounds. Each panel has a lead judge assigned, an experienced judge who will lead the panel�s activities during the day of the judging event. Coordinates with any follow-up activities. Completion of final Feedback Report. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 14

15. Judging Event Timeline During the morning of the judging event, listen to and individually score a team presentation. At the end of the presentation you have the option of asking questions of the presenters (or review the DVD) for up to 5 minutes. The questions must only be for clarification or verification of what was said in the presentation relative to the criteria. They cannot pertain to issues outside the criteria. Then adjourn to a private location to finish individual scoring, reach consensus on the scores, and then write draft feedback comments. Note: Some judging panels may be asked to judge two team presentations rather than just one. This will be discussed during the Judges Conference call if need be. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 15

16. Presentation Time Limit - 30 Min. What happens if the presentation goes beyond 30 minutes? The lead judge tells the judging panel to stop scoring at the 30 minute mark but allows the team to finish its presentation. Anything presented after the 30 minute mark is not considered in judging and scoring. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 16

17. Your Customers The organizations sponsoring the participating teams. The participating teams. Other judges because achieving consistency across judges is important for accuracy and fairness of assessment. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 17

18. Lesson 2 How to Score a Team�s Project Presentation 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 18

19. ASQ-ITEA Criteria Summary Read the ASQ-ITEA Award Criteria The following 5 slides show the summary form of the ASQ-ITEA Award Criteria along with the point scores. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 19

20. CTEA Award Criteria Summary (1) 1. Project Selection and Purpose (9 items @ 3 points = 27 points possible) 1A. Explain the methods used to choose the project. (Provide specific examples of techniques and data used.) a. Describe the types of data and/or quality tools used to select the project, and why they were used. b. Explain the reasons why the project was selected. c. Describe the involvement of potential stakeholders in project selection. 1B. Explain how the project supports/aligns with the organization's goals, performance measures, and/or strategies. a. Identify the affected organizational goals, performance measures, and/or strategies. b. Identify the types of impact on each goal, performance measure, and/or strategy. c. Identify the degree of impact on each goal, performance measure, and/or strategy, and how this was determined. 1C. Identify the potential stakeholders (who may be impacted by the project) and explain how they may be impacted by the project. a. Identify potential internal and external stakeholders and explain how they were identified. b. Identify the types of potential impact on stakeholders and explain how these were determined. c. Identify the degree of potential impact on stakeholders and explain how this was determined. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 20

21. CTEA Award Criteria Summary (2) 2. Current Situation Analysis (6 items @ 4.5 points = 27 points possible) 2A. Explain the approach/process the team used to identify the potential root cause(s)/improvement opportunity(ies). a. Describe the methods and tools used to identify possible root causes/improvement opportunities. b. Describe the team�s analysis of data to identify possible root causes/improvement opportunities. c. Describe how or if any of the stakeholders were involved in identifying the possible root causes/improvement opportunities. 2B. Describe how the team analyzed information to identify the final root cause(s)/improvement opportunity(ies). (Include any appropriate validation.) a. Describe the methods and tools used to identify the final root cause(s)/improvement opportunity(ies). b. Describe the team�s analysis of data to select the final root cause(s)/improvement opportunity(ies). c. Identify the root cause(s)/improvement opportunity(ies) and explain how the team validated the final root cause(s)/improvement opportunity(ies). 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 21

22. CTEA Award Criteria Summary (3) 3. Solution Development (9 items @ 3 points = 27 points possible) 3A. Explain the methods used to identify the possible solutions/improvement actions. a. Describe the methods and tools used to develop possible solutions/improvement actions. b. Describe the team�s analysis of data to develop possible solutions/improvement actions. c. Indicate the criteria the team decided to use in selecting the final solution(s)/improvement action(s). 3B. Explain how the final solution(s)/improvement action(s) was/were determined. a. Describe the methods and tools used by the team to select the final solution(s)/improvement action(s). b. Describe the team�s analysis of data to select the final solution(s)/improvement action(s). c. Describe the involvement of stakeholders in the selection of the final solution(s)/improvement action(s). 3C. Explain the final solution(s)/improvement action(s), validation, and the benefits expected to be realized by implementing the team�s solution(s)/improvement action(s). a. Describe the final solution(s)/improvement action(s) and explain how the team validated the final solution(s)/improvement action(s). b. List the types of tangible and intangible benefits that are expected to be realized by implementing the team�s solution(s)/improvement action(s). c. Explain how the team used data to justify the implementation of the team�s solution(s)/improvement action(s). 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 22

23. CTEA Award Criteria Summary (4) 4. Project Implementation and Results (9 items @ 3 points = 27 points possible) 4A. Explain how buy-in/agreement was achieved for implementation. a. Indicate the types of internal and external (if applicable) stakeholder involvement in implementation. b. Describe how various types of resistance were identified and addressed. c. Explain how stakeholder buy-in was ensured. 4B. Explain the approach used by the team to implement its solution(s)/improvement action(s) and to ensure the results. a. Describe the plan developed by the team to implement its solution(s)/improvement action(s). b. Describe the procedure, system, or other changes that were made to implement the solution(s)/improvement action(s) and to sustain the results. c. Describe the creation and installation of a system for measuring and sustaining results. 4C. Describe the results achieved. a. Indicate the types of tangible and intangible results that were realized. b. Explain how the project�s results link with the organization�s goals, performance measures, and/or strategies. c. Describe how results were shared with stakeholders. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 23

24. CTEA Award Criteria Summary (5) 5. Team/Project Management and Project Presentation (4 items @ 4.5 points = 18 points possible) 5A. Explain how the team members were selected and how they were involved throughout the project. 5B. Explain how the team was prepared to work together in addressing the project. 5C. Explain how the team managed its performance to ensure it was effective as a team. 5D. The team will also be judged on the clarity and organization of its presentation. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 24

25. ASQ-ITEA Criteria in Detail Read the ASQ-ITEA Criteria in Detail (pay attention to the details provided for each item and the Notes) Criteria Summary covered in previous slides. Criteria in Detail. The detailed version includes more information about what is expected for each criterion item. The detailed version also includes Notes for you to use to enable you to do more accurate scoring. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 25

26. Scoring the Team Presentation Judges evaluate the extent to which each item in the Criteria is addressed in the presentation and assign one of the four possible scores: Not Covered (0 Points) Totally missing (No approach evident) Unclear (1-1.5 Points) Touched upon, but not clear. Not enough information is provided to determine if the team�s approach met, or could meet, the criterion requirements. (Approach evident) Meets Criteria (2 or 3 Points) Sufficient information is provided to determine that the team�s approach met the criterion requirements. (Approach applied with results evident) Exceeds (3 or 4.5) Points The team�s approach goes beyond meeting the criterion and provides additional clarity indicating increased accuracy in the team�s analysis, actions, and/or conclusions. Integration with other criterion items is apparent and enhances the team�s overall results. A �Best Practice� or �Role Model� approach. (Approach broadly integrated and evident) 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 26

27. Scoring Tips (1) Note that �Exceeds� scoring level descriptions are cumulative. What is looked for in �Exceeds� must include what is in �Meets�. The scoring level of �Unclear� is when not enough information is provided to determine if the team�s approach met or could meet the criteria requirements. The scoring level of �Not Covered� is not noted in the guidelines because it receives �0� points. These are meant to be general guidelines for scoring to serve as a standard for evaluating team presentations. All teams are still encouraged to be as creative and as innovative as possible in sharing their stories. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 27

28. Scoring Tips (2) Criteria 2Aa, 2Ba, 3Aa, and 3Ba ask the team to describe their tools/methods. Criteria 2Ab, 2Bb, 3Ab, and 3Bb ask the team to describe their analysis of data. There are 13 criteria with two parts. If you rate the two parts at different levels, you must score it at the lower level. For example, if one part is �Meets� and the second part is �Exceeds�, the item is scored at �Meets�. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 28

29. Scoring Tips (3) Read the ASQ-ITEA Glossary of Terms ASQ-ITEA has constructed a Glossary of Terms that is very important for judges. In particular be sure to study the information given for �Describe� because each of the four words listed there requires a different level of information. Also be sure to study the term �Integration� because teams that demonstrate this characteristic should score better and are more likely to attain the �Exceeds� level than those that do not. Integration explained on the next two slides. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 29

30. Integration Explained (1) The Criteria in Detail draw attention to where an item detail has a connection (cross-linkage) to other criterion item(s). The Criteria show that addressing a criterion item detail in one area has an impact on one or more other criterion areas. How the team addresses these cross-linkages indicates the degree of integration within the team project, stakeholder, and organization direction. This is a significant factor for a team to score at the �Exceeds� level. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 30

31. Integration Explained (2) �1Cb. Identify the types of potential impact on stakeholders and explain how these were determined. Stakeholders may be impacted in many different ways by the project, from quality to timeliness. Demonstrating that the team understands how the project may have different impacts on the different stakeholder groups is important. As with 1Bb (type of goal impact), the impact is implied to be directional. NOTE: This section could later tie to 4Ab, as correctly predicting potential negative impacts early in the project can help the team prepare to overcome resistance later. The Cross-Linkage in 1Cb is with 1Bb and 4Ab, if addressed per the criteria, would indicate there is integration in the team�s approach across the project and/or organization. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 31

32. Scoring Guidelines Review the CTEA Judges Workbook (all 3 tabs) During the team presentation, each judge uses their Judges Worksheet to take notes and to evaluate the presentation. You will have the forms in all 3 tabs given to you at the judging event. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 32

33. Judges Worksheet (Tab 1 in the Judges Workbook) This is the Judges Worksheet to be used during the teams presentation. Refer to the worksheet during the independent review and during consensus with your team. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 33

34. Reaching Consensus (1) After doing your individual scoring, the judges must reach consensus on their scores across the panel. A panel member collects and posts all scores to the Consensus Score Sheet. No consensus discussion is necessary on an item if there is only one scoring level deviation among the judges� scores. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 34

35. Reaching Consensus (2) Review the Consensus Scoresheet (Tab 2 of the Judges Workbook) If there is more than one scoring level deviation, the panel members must adjust their individual scores to be within one deviation of each other (example on next slide). No judge can have a score that deviates two levels from another judge. This is done through dialogue about reasons for the respective scoring. Each judge supports their rational for the scoring and discuss as a group. Then the judges� scores are averaged on the Consensus Scoresheet. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 35

36. Consensus Scoresheet Sample This is sample of an initial Consensus Scoresheet. Notice 2 scoring level deviation on items 1B.b and 1C.b. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 36

37. Final Scoresheet SampleTab 3 in Judges Workbook Review the ASQ-ITEA Scoresheet Sample The results of the Consensus Scoresheet are automatically populated in the Final Scoresheet. Review the CTEA Final Scoresheet (Tab 3). 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 37

38. Familiarizing Yourself with a Presentation Review the ASQ-ITEA Presentation Sample The final step in understanding scoring is to familiarize yourself with what a presentation looks like. While each team presentation is different, one is provided as part of this training package and available on the CTEA website. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 38

39. Lesson 3 How to Write Effective Feedback Comments 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 39

40. Purpose of Feedback Reports A valuable tool teams and their organizations can use to take action. Celebrate Strengths Consider Suggestions for Improvement (SFIs) The feedback report has two components. Total Score (from judges consensus) Feedback Comments Strengths Suggestions for Improvement 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 40

41. Qualities of �Well Written� Feedback Comments Read the CTEA Judges Guidelines Succinct, specific narrative. Constructive and actionable. Clear to the team why included. Complete sentences w/o grammar errors. Non-prescriptive. Address important elements. Supports score and evaluation. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 41

42. Do�s & Don�ts of Comment Writing Do� Use Facts w/o judgment. Be specific about what is missing (�it is not clear� comments). Use Criteria language and link to examples in the team�s presentation. Be impactful and answer the �so what�. Don�t� Use Opinions. Be prescriptive (Avoid �could, should or would�). Be overly technical in your writing so that the team is unable to understand feedback. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 42

43. Feedback Comments Should Have the Following Items Number of the Criteria item. Number on the presentation slide(s) that the comment addresses. Language of the criteria item in the form of a brief quote or paraphrase from the item. Language of the team in the form of a brief quote or paraphrase from the relevant slide(s). 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 43

44. Adhere to Style Guidelines Use a polite, professional, positive tone. Use active voice, present tense. Use the team's terminology. Avoid jargon or acronyms except when used by the team. Refer to �the team�, not name. Reference figure numbers when appropriate. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 44

45. Determining What to Emphasizein Feedback Comments Strength Comments should be the highest scoring items. SFI Comments should be the lowest scoring. Comments on integration should only be written if there are noticeable issues with respect of cross-linkages (either as Strengths or SFIs). 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 45

46. Examples: Well Written Strength Comments �The presentation effectively identified the internal and external stakeholders and clearly assigned rated measures to show how much they will be impacted by the project (Slide 17)(1Cb).� �Excellent information was provided explaining why this project was selected (Slide 3). Six sigma tools were used to justify the project selection and were clearly explained and illustrated in slides 6 � 9. Additionally the use of data to substantiate this project�s selection was highly informative.� 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 46

47. Example: Poorly Written Comment �The team did a good job when they presented their project.� Issues with Strength Comment: No support with examples to let the team know what was good. Does not reinforce practices. Does not describe the impact. Does not relate to the criteria. Not in present tense, active voice. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 47

48. �Tips� for SFI Opening Statements Although the team addresses/states/indicates� While the team demonstrated � it is not clear/apparent/evident how/what the team... The team does not address/report how/what� 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 48

49. Examples: Well Written SFI Comments �The team clearly explained the types of impact and the degree of impact of the project on both internal and external stakeholders; however in order to strengthen this item the judges were searching for information on �how the scores and measurable data were obtained.� (1Cb & 1Cc).� �While the team showed how they used six sigma tools DMADV and SIPOC to identify potential stakeholders; there was no evidence provided to describe how these potential stakeholders were involved in the selection of the project (Slide10).� 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 49

50. Example: Poorly Written SFI Comment �It was impossible for us to tell what the team achieved. Goals and measures were glaringly omitted. Without an action plan that includes goals it will be impossible for the team to achieve meaningful results. The organization should be using the balanced scorecard to help them.� Issues with Poorly Written SFI Comment: Critical tone �glaringly omitted�. So what is opinion, not fact �Without an action plan that includes goals it will be impossible for the team��. Prescriptive. No linkage to criteria. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 50

51. Refining Your Comments What is the real strength or SFI? Is it clear? What team practices in the comment support the Strength/SFI? Is the �so what� linked to the main point of the comment? Is the �so what� factual? 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 51

52. Feedback on Integration Earlier the need to include integration (or lack thereof) in the scoring was mentioned If the team showed a high score due to integration where the Criteria call for a cross-linkage, then you should write a Strength feedback comment to that effect On the other hand, if the team showed a direct lack of integration where the Criteria call for a cross-linkage, then you should write an SFI comment to that effect 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 52

53. Reaching Consensuson Feedback Comments How the judging panel does the work of writing draft feedback comments is up to the panel, and the lead judge will provide suggestions and guidance. There is a requirement for the judges to reach consensus on their feedback comments. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 53

54. Supporting Materials for Writing Feedback Comments Review the CTEA Feedback Report Template Read the ASQ-ITEA Feedback Sample The two supporting documents named above are provided to you for writing feedback. The first is an actual feedback report for the ASQ-ITEA presentation provided earlier (HD Supply). The second is a template you will be provided at the judging event. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 54

55. Lesson 4 Next Steps � Phase 2 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 55

56. Schedule (1) Attend one of the following judge conference calls using the call-in information: 866-906-9888 x6160429 Tuesday, September 20, 8:30 - 10:30 am Wednesday, September 21, 12:30 - 2:30 pm Thursday, September 22, 8:30 - 10:30 am (Back-up Call) Tuesday, September 27, 8:30 - 10:30 am We will answer questions regarding the process that you may have at that time. We will discuss logistical issues, locations, schedules, etc. We will verify that you have all documentation. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 56

57. Schedule (2) Attend your assigned judging event(s): October 3, 2011: Los Angeles - Good Samaritan Hospital (Site Coordinator: John Corbett) October 4, 2011: San Diego - Sharp HealthCare (Site Coordinator: Rusty Robinson) October 6, 2011: Sacramento - National University (Site Coordinator: Ron Bane) Respond to your lead judge�s request for comments on the final draft of the feedback report within 1 week of receipt. The Lead Judge will submit the final feedback report to the CTEA Council Chair by October 21, 2011. 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 57

58. Schedule (3) The CTEA Council Chair provides the feedback reports to the teams and their sponsoring organizations: November 14, 2011 CTEA Recognition and Awards Ceremony: April 19-20, 2012 ASQ ITEA Finalist Presentations and Awards Ceremony, ASQ World Conference, Anaheim CA: May 21-23, 2012 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 58

59. Conclusion Thank you for taking Phase 1 of the CTEA Judges Training See you in October! 2011_CTEA_Judges_Training http://www.calexcellence.org/ctea.html 59