On-Site Evaluation Visit: Roles, Responsibilities, and Conduct

1. Module 6 The On-Site Evaluation Visit

2. Learning Objectives Following training, the participant will: • Know the role and responsibilities of the evaluation team. • Know the role and responsibilities of the program director. • Have an example of a typical schedule for an on-site visit.

3. Evaluation Team Philosophy • The Evaluation Team has an obligation to both FEPAC and the Institution being evaluated to be fair and objective. • On-site evaluators represent FEPAC to the Institution. • Compliance/non-compliance must be to FEPAC accreditation standards. • Findings should NOT be • Based on personal opinion or • Compared to the way the program is conducted at the evaluators’ institutions.

4. Evaluator Roles and Conduct Evaluators should • Remember that they are guests of the Institution, • Demonstrate a professional attitude, behavior, speech, and dress, • Be aware of the possibility that the institution faculty and staff may be stressed and/or sensitive to perceived criticism, • Remain objective, • Only provide FEPAC materials and guidance, • Remember that the primary purpose of the on-site evaluation is fact finding, not consultation.

5. Evaluator Roles and Conduct Evaluators should NOT • Interrogate or lecture to institution personnel, • Disagree, argue, or negotiate - just collect facts, • Opine as to what the Commission will do, • Advise: You are not a consultant, • Show Applicant the report and/or comments.

6. Evaluator Responsibilities 1. Understand the accreditation standards and the requirements for writing the self-study report. 2. Thoroughly read the self-study before the visit. 3. Compare the reported activities with those required by standards. 4. Plan to arrive so that a team meeting can be held before the initial visit to the Institution. 5. Conduct interviews. • Purpose is fact-finding and clarification of submitted documents. 6. Listen/assess situation.

7. Evaluator Responsibilities 7. Provide the Team leader with report/comments/ deficiencies in assigned areas. 8. Adhere to the schedule. 9. Have a draft of the evaluation report BEFORE leaving the institution. 10. Send a final report to the Accreditation Director within 10 days of the site visit.

8. Additional Responsibilities of the Lead Evaluator (Academic Member) • Communicate with the Institution contact. • Eliminate any final summation surprises. • Act as facilitator for all Evaluation Team members. • Confidentially mediate any issues between Team members and Institution staff. • Coordinate all Evaluation activities to avoid time-wasting activities and duplicated efforts. • Assimilate and finalize all reports for the FEPAC Review Committee and Commission review. • Do not leave a copy of the draft Evaluation Report with the Institution. This report is pre-decisional until all reviews and Commission actions are complete.

9. Evaluator Checklist • Checklists are provided as a resource for Evaluation Team use during evaluations. • Can be copied for each member of the Evaluation Team to facilitate the collection of all relevant information. • Address each point of a specific standard. • Helps evaluators remember what to look for.

10. Responsibilities of the Program Director • Submit the application for accreditation. • Prepare the self-study report, with the assistance and input of other interest parties. • Work with the accreditation director and team leader to schedule the on-site visit. • Provide the FEPAC office with suggestions for lodging and travel. • All is coordinated through the FEPAC office.

11. Before the On-Site Visit Pre-Visit Checklist: • A checklist of activities that should be conducted by the Lead Evaluator before the On-site visit takes place. • Each evaluator should read the institution’s self-study report, prepare questions, and know which documents will provide evidence of compliance or non-compliance with each standard.

12. MEETINGS:Pre-Visit Team Meeting • This meeting must take place before the evaluation. • Typically occurs the evening before the evaluation, either at the hotel or a site selected near the institution being evaluated. • Arranged by team leader. • Purpose is to plan the visit.

13. Meeting Room at the Institution Items that should be in the team meeting room: 1. A lockable meeting room for the on-site team to meet privately. 2. Telephone (speaker phone preferable) for both inside and outside calls. 3. A PC with internet access and a printer. 4. A copy of the application and all supporting documentation.

14. MEETINGS:Program Director • The first meeting with the applicant Institution generally involves the Program Director. • Usually longer than most others. • Serves as an introductory meeting. • Serves as the first step in the evaluation. • Team should get information on: • program structure • mission • workload • budget • institution management.

15. MEETINGS:Faculty & Staff • The Team should offer to meet the Faculty and Staff to explain the evaluation process. • The Team will defer to the Program Director regarding staff participation in this type of a meeting. • There may be some operational, logistic, or space issues that may dictate the format and staff participation. • Suggested topics to cover during this meeting are included in the Evaluator’s Manual.

16. MEETINGS:Meeting with Administrators/CEO • The Team must meet with at least one representative of the Institution’s upper management (Dean, Provost, President). • This meeting should: • introduce all those involved, • briefly outline what the institution can expect during the process, • determine the degree of the administration’s support for the program, • should be fairly brief, • should cover “top level” topics and issues, • concerns raised by administrative assessors in self-study.

17. MEETINGS:Students • Meet with students separate from faculty. • Use to assess student perceptions and thoughts about the program. • Meet with each faculty member individually if possible.

18. MeetingsProgram Graduates • Likely to be done by phone. • Can be used to ask about post-graduate assessment efforts, whether graduate feels adequately prepared for workforce, quality of laboratory training.

19. MEETINGS:Conduct of Institutional Meetings • Meet with each group separately, outside the presence of the Program Director or others. • Ask each their views of the program. • Compare each group’s answers with self-study documentation and provided materials for Evaluators’ review, to determine the reliability of this information. • It is also recommended that the Evaluation Team hold a pre-summation meeting with the Program Director alone. This affords the opportunity to correct misperceptions, if any, before the Summation Conference is held.

20. MEETINGS:Summation Conference • Held at the end of the evaluation. • Includes the Evaluation Team, program director and any others the director chooses. • Defer to the Institution’s administration and/or program director’s choice whether this is open to the staff or to a smaller group consisting of department chairs, etc. • Suggested topics and format provided in the Evaluators Manual.

21. Sample Agenda • Visits typically last 2-3 days. • This sample agenda is for a two day on-site visit, but could be expanded to three days. • Allow at least 2 hours each in the first two days at the end of each day for “team private time” to review and assess where you are in the visit and for 3 hrs on the third day prior to the exit briefing to prepare the report.

22. Sample AgendaDay 1 8:30 Opening meeting of Team and Program Director 9:00 Tour of Forensic Science facilities 10:00 Meeting with program faculty (a separate meeting with a Department Chair may be warranted) 10:45 Meeting with College Dean 11:30 Lunch 1:00 Meeting with Provost/President 1:45 Tour of additional on-campus facilities available to forensic science students (library) 3:30 Review of documentation – team only 5:00 Team returns to hotel; dinner and evening meetings for the team; begin preparation of Draft Report

23. Sample AgendaDay 2 8:30 Observe class 9:30 Meetings with students 11:00 Phone call to graduates 12:00 Lunch 1:00 Finish draft report 4:00 Exit briefing – team and program representatives

24. The On-Site Evaluation Report

25. On-Site Evaluation Report: Content • Each standard is described in the online system. • Record specific observations and evidence. • Address all requirements of the standard and initial commission review. • Do not indicate compliance or non-compliance, rather tell why or why not a standard is met. Evidence and results, not conclusions. “Minutes from the curriculum review committee meeting were provided”. • Be specific in your observations: lack of detail may cause administrative delays.

26. On-Site Evaluation Report: Review by Commissioners • Team leader sends report to Accreditation Director within 10 days of site visit. • Accreditation Director assigns 2-person committee to review report. This may be done in advance of the On-site visit. • Committee made up of FEPAC Commissioners.

27. On-Site Evaluation Report: Review by Commissioners The Review Committee reviews the report Committee looks for: • Evidence of compliance, • inconsistent application of standards, • ambiguities in discussion of supporting criteria, • non-compliance issues, • correct assignment of identified problems to the appropriate standard. Lead Evaluator and Review Committee have the authority to change preliminary findings and/or commentary.

28. On-Site Evaluation Report:Institution Response • The Director of Accreditation sends reviewed report to Applicant. • Applicant may dispute findings. Evaluation Reports are “Pre-decisional.”

29. Final Commission Review & Decision Making Process • Commission makes accreditation decision (January meeting or February AAFS meeting) • Each team member receives a copy of the determination letter sent by FEPAC to the Institution. • Letter provides: • a complete summary of FEPAC deliberations, • final decision on the accreditation application for each forensic program reviewed.

30. CHEA Disclosure • According to FEPAC Policy 3.13 and CHEA requirements, all accreditation decisions and the basis for those decisions must be publicly disclosed. • These decisions are posted to the FEPAC website. • Programs are given the opportunity to provide a public response related to the decision, which is also posted to the FEPAC website.

31. Decision Appeal • When a program is denied accreditation, the program may appeal this decision. The program may not appeal conditional or probationary status. • During an appeal the Self-Study and team report are provided to the Appeal Board. Therefore, it is imperative that the report and self-study represent a clear and detailed view of the on-site evaluation.