BIR Session Team Lead Update

1. BIR SessionTeam Lead Update December 2, 2011

2. Welcome • Introductions—Please share • Your name • Your affiliation/employer and the • Type of institution for which you will be the team lead • IHE-CSU, UC, Private • LEA-School District, County Office • Other

3. Topics to be Addressed Two-Month Out Pre-Visit Site Visit Team Role of CTC Consultant Work Prior to Site visit Days 1-4 Mid-Visit Report

4. Topics to be Addressed (cont’d.) Program Sampling Decisions on Standards Accreditation Recommendation Stipulations Team Report Exit Report COA Presentation

5. Two-Month Pre-Visit • Goals • Use 2-month out checklist • Meet the Institution/Program leadership • View the institution and the hotel • Verify adequacy of: meeting rooms (institution and hotel), internet service, visit meal and transportation plans • Review the Interview Schedule and provide feedback to the institution

6. Two-Month Pre-Visit (cont’d.) • Goals • Confirm status of all programs, including Intern option • Operating • Transitioning • Inactive • Confirm locations of programs • Finalize clustering of programs for program sampling

7. Schedule for Two-Month PV • Team lead and consultant meet night before, discuss working styles, consultant’s supporting role • Morning—Meet with institution and see the meeting room at the hotel • Institution—Meet with leadership to review the topics on the “Checklist” • Usually completed by early afternoon

8. Two-Month Out ChecklistTeam Lead’s Focus • Confirm with state consultant • Arrival time • Lodging at hotel for the visit • State Consultant will focus the discussion • Team Lead—interview schedule—including program sampling grouping!

9. Site Visit Team • Functions as a team—no one team member’s views may be allowed to be worth more than others • Consensus is that while not all members of the team may personally agree with a proposed decision, all members understand the bases for the decision and all members can support it • In the process of achieving consensus, team members must seek to understand one another’s views and be willing to set aside personal views in reaching a collective decision • Use your State Consultant

10. Composition of the Team Team Lead (1) Common Standards {NCATE} Cluster (1-4) Program Sampling Cluster (1-4) State Consultant (1-2) State Consultant will share team member names/contact info and cluster assign-ments but are not truly a part of the site visit team

11. Role of the CTC Consultant • Works with Institution for one year before SV on meaning and intent of standards, documents, schedule, logistics, etc. • Sends team lead or team members: • institution documents for review • Guidance for Program Cluster Team Members • Handles logistical issues—food, internet, etc. • Keeps line of communication open between team and Institution • Deals with integrity of “process” • Helps team in deliberations as well as editing and reviewing report

12. Work Prior to Site Visit (and before conference call) Team Lead asks team members to: Watch Program or Common Standards Cluster archived video (leads may want to as well) Common Standards Cluster: Read Common Standards Narrative (all) Read Biennial Report Response Prepare draft report for “your” standards Program Cluster—for your programs: Read Program Summary(ies) Read Program Assessment Preliminary Report feedback Read Biennial Report and CTC Response Prepare draft of program report(s)

13. Work Prior to Site Visit (cont’d.) • One week out (or more): conference phone call(s) to discuss findings • Team lead tells team to: • Be prepared to look for additional information that verifies what was in documents read • Review previous COA reports to be familiar with technical writing required • Prepare questions/areas of concern to send to Team Lead before 1st day of SV • Develop first draft of team report—see next slide

14. 1st Draft of Report Completed prior to site visit Will assist in guiding interviews—can include questions that need to be asked during interviews Use program summaries or Common Standards narrative Fill in what is stated in the narrative as a frame—can be removed, altered, augmented, etc., as needed at the visit.  

15. At the SV—Pacing of the Work • Day 1: Get to know each other and the institution, raise questions, gather information (possibly have interviews during reception and/or learn about institution and programs) • Day 2: Gather and sort a lot of information, identify what is known and what needs to be found, draft of report on what is known • Day 3: Hone in on the unknown, team decisions, write report, recommendations, edit report • Day 4: Review and finalize report, meet with institutional leadership and give Exit Report

16. First Team Meeting Team Lead and the State Consultant organize the meeting Introductions Set tone for the visit Each team member has a voice, but no individual dominates the discussions

17. Meetings will take place each evening and at lunch each day Discussions assist team members in incorporating evidence into report as they get it Team lead helps members look at language of standards and helps them understand how to incorporate the identity of sources as each report is written.

18. Bring Data—from Multiple Sources— into the Team’s Discussion Common Standards {NCATE} Cluster shares info Program Sampling Cluster shares info Work to confirm from multiple sources—documentary and interview Interviews—cite stakeholder group and program Biennial Reports and Program Assessment A single instance of evidence will be considered an outlier unless issues came from a “constituency” where all or most faculty, candidates, etc., agree—then it becomes a concern or possible issue

19. Progress Towards Findings on Standards By end of Day 2, the team should have enough evidence from multiple sources to: Make a preliminary determination on most of the Common Standards Confirm the Program Assessment findings Discussions on 2nd night focus on: Findings as they relate to Program and Common Standards What other information/evidence is needed by the team to complete findings on all standards 19

20. Mid-Visit Report • Purpose—Dean/Director, Team Lead and State Consultant(s) • Must be in writing (use form) - Identify what the team still needs to know in order to make a determination about standards - Ask for evidence team members have not been able to locate up to this point • Discuss areas of concern or “red flags,” if any, that have emerged so far • Each Morning and Evening—an informal reporting takes place

21. Group Decision-Making Site visit team makes a consensus decision Review all parts of the standard Review all evidence Use only evidence gathered by the team in interviews or provided by the institution at the visit, during program assessment or biennial report process. Is there a pathway of evidence from multiple sources? Do the multiple sources of evidence point to the same conclusion, or are there discrepancies? 21

22. Suggestions for Team Lead Use flip charts or an overhead projector to identify questions and concerns during the first night and then revising them every night Use LCD for whole group review of the draft report Require team members to complete some work prior to the start of the visit With CTC consultant, check in regularly with team members about their progress and read a draft of each member’s report early in the visit: Falling behind? Not a strong writer? Showing a bias? Unsure how to incorporate the source of information into the writing?

23. Sampling Across Programs Only ONE stakeholder group will be interviewed at a time—likely divided into teaching and services/advanced groups (i.e., teaching—MS/SS and Ed Spec, services/advanced—reading, admin, librarian, PPS) • Candidates • Completers • Program coordinators • Faculty and Instructional Personnel • Field supervisors—programmatic and district-based • Employers • Credential analysts • Advisory boards

24. Sampling Across Programs (cont’d.) Review at 10,000 foot level Standards preliminarily aligned during Program Assessment are contraindication to “dig deeper” Constituents bring up things that were not “on paper” during PA Team discussion and, possibly, phone call to CTC administrators to “re-open” preliminarily aligned standard (s)

25. Findings on Standards Met—All phrases of the standard are evident and effectively implemented. Met with Concern(s)—One or more phrases of the standard are not evident or are ineffectively implemented. Not Met — Significant phrases of the standard are not evident or are so ineffectively implemented that it is not possible to see the standard in the program. 25

26. Accreditation Recommendation • Site Visit Team makes a consensus Accreditation Recommendation to the COA • Accreditation • Accreditation with Stipulations • Denial of Accreditation • Team agrees on the recommendation and crafts a Rationale for the recommendation

27. Team Recommendations • Accreditation • Nothing required • 7th year Report • Report in next Biennial Report • Accreditation with Stipulations • Accreditation with Major Stipulations • Accreditation with Probationary Stipulations • Denial of Accreditation

28. Stipulations • State what the program sponsor must do to meet the standards and gain accreditation • Define only the what, and not the how • May include a re-visit • Are drafted by Team Lead and CTC Consultant and reviewed by team before inclusion in team report • Are recommendations to the COA, which can accept, reject, or revise them

29. How to Divide Writing Responsibilities • Depends on the size of the team • Team lead makes these decisions, but asks for members’ preferences • Common Standards—approximately 1 page • Programs—approximately 2 pages • Each team member is responsible for the full team report

30. Responsibility for Understanding the Common Standards Each team member is responsible for the content of the Common Standards Use the Glossary! Team Lead and CTC Consultant are resources—Team Leads need to be very comfortable with the Common Standards

31. Common Standards Assignments Prior to the visit, assign specific Common Standards to team members—primary and secondary Team Lead may have to write to 1 or more CS if it is a small team Begin draft of CS Report before SV, refine on 1st and 2nd evenings, finalize on last evening

32. CS Pre-Visit Worksheet Each Common Standards Cluster member will review the Common Standards Document and note evidence reviewed in the top box (see next slide) Team members should note evidence they want to look for in the lower box Arrive at Visit with the Pre-Visit Worksheet completed—Some team leads ask that Worksheet is emailed prior to arriving at visit

34. Report for Each Common Standard • Describe how the institution/program operates related to the language in the standard • Cite examples of evidence to: • Flesh out and “personalize” how the institution/ program meets key parts of a standard • Note innovative, unique, and/or noteworthy practices used by the institution/program to meet key parts of a standard • Indicate triangulation from multiple sources • Report must be congruent across Common Standards Reports, Program Reports, Standard Findings, Accreditation Recommendation and Rationale

35. Format for Common Standards Report • Writing Worksheet: use during the collection of evidence • Report Template: prompts to assist in the writing of the report —means to replace with your own writing Blue Text

36. Program Assessment and Program Sampling Program Assessment began 2 years before the site visit Many Program Standards will have been identified as “Preliminarily Aligned” Other Program Standards will be reviewed onsite—notes to SV team important Sample across 3 categories: Program Design, Course of Study (curriculum and field experience), and Candidate Competence

37. Program Summary • Developed by institution program when submitting the program assessment report. Provides contextual information on the program (2-4 pages total) • Program Design • Course of Study (curriculum and field experience) • Candidate Competence • Program Summaries must be read prior to arriving at the site visit and will serve as the foundation for the Program Report.

38. Using Biennial Reports • Biennial Reports are available for all visits. They identify key assessments used to assess candidate competency and unit/program modifications made based on data. (CS 2 and 9 member should review) • CTC Response points out the level of data analysis and the plans to use the analysis to drive program modifications. • Biennial Reports (for programs reviewed by program cluster member) and CTC Response must be reviewed prior to arriving at the site visit.

39. When would a team member go to the specific program standards? • If a concern seems to arise during the site visit—e.g., candidates talk about not being supervised in field experience—pin down which program(s) and try to triangulate with other stakeholder groups • If the concern is confirmed by multiple stakeholder groups, or multiple members of a constituency, go to the adopted program standards to detail the issue in the program report

40. MS, SS, and Ed Sp Intern Programs 120 hour Pre-Service • Required prior to the issuance of the Intern Credential. • Team members focusing on one of these programs should understand the Pre-service component for the program and the tracking process to ensure that all individuals recommended for the Intern Credential have completed the 120 pre-service (June 2008).

41. MS and SS Intern Programs • Early Completion Option • All MS and SS Intern programs must offer an ECO in the areas of MS, SS-English, SS-Math, and SS-Science IF the program offers the internship for the credential. • Check on the advising process and how many individuals have elected ECO. • May not have any candidates who have elected ECO…that is ok

42. TPA When visit has MS and/or SS Program(s): • State Consultant will provide questions for Program Sampling Team Member(s) to ask • TPA Coordinator • Faculty • Assessors • Supervisors • Candidates

43. Program Report Program Summary is the genesis of the Program Report As a team, make decisions for each standard If one or more standards are not fully met, identify the standard and detail the evidence that led to the standard being “Met with Concerns” or “Not Met” Provide examples and evidence how the institution meets the program standards

44. Program Report (cont’d.) • If program has multiple pathways, a decision needs to be made as to how to report on these • Multiple/Single combined or separate? • Information about each pathway? • Write on each of the 3 main areas • Program Design • Course of Study (curriculum and field experience) • Candidate Competence • Will make one program finding even if there are multiple pathways

45. Exit Report • Prior to the Exit Report, team lead and state consultant meet with leadership to review the report and answer any questions • Exit Report—public, factual, no questions • Institution introduces state consultant • State consultant provides brief explanation of process up to now • Team Lead presents findings and accreditation recommendation • State consultant reviews COA meeting and appeal options

46. Presentation to the COA Specific date and scheduled time—state consultant will arrange this (team may not need to attend if decision is Accreditation) COA will have read the Site Visit Report thoroughly so provide contextual information about the visit to indicate how the findings were determined and considerations in making the recommendation Order of the presentation: consultant provides brief overview, team lead provides brief summary of decision COA Members will ask questions of institution, team lead or state consultant Watch 2-3 presentations from the COA Archive

47. Support • Assigned State Consultant • BTSA Induction Visits • Karen Sacramento and Gay Roby • Cheryl Hickey or Teri Clark • BIR Web page: http://www.ctc.ca.gov/educator-prep/BIR.html • Accreditation Handbook, Chapter 12: Team Leadership: http://www.ctc.ca.gov/educator-prep/accred-handbook/AH-Chapter-12.doc