1. Formative Evaluation UsingStudent Focus Groups Heidi M. Anderson, Ph.D. University of Kentucky College of Pharmacy (UKCOP)

2. Student Focus Groups • UKCOP uses student focus groups as part of the formative curricular evaluation process • Goals: • Systematic method to collect information about the curriculum • Assure confidential timely exchange between professors and students on their perceptions of course in progress

3. How it Developed • D. Joseph Clark, Washington University • Small Group Instructional Diagnosis (SGID) • 5-step process • Conducted once mid-semester • Adapted a hybrid version of SGID • Student Liaison Committee (SLC)

4. Who Participates in SLC? • 10 Students from each class for Fall/Spring • 9 members randomly selected from the 10 small groups • 1 member is the class Vice President • Facilitators from the Office of Education Innovation (OEI) • Assistant Dean • Assessment Coordinator • Recorder • Faculty observers periodically invited

5. Obtaining the Participants • Beginning of Fall semester students receive 1st electronic correspondence, that: • Explains purpose of Student Liaison Committee (SLC) • Describes the random selection process • 2nd correspondence, is sent: • Those 10 selected members, informed of dates, training, etc. • Entire class announcing the 10 individuals

6. Initial Training Meeting • Training on Providing Feedback • Purpose UKCOP process and assessment • Focus group methodology • Guidelines on offering constructive feedback • Relevant and Appropriate • Specific descriptions • Offer both positive and negative • Explain how to improve

7. Meeting Process • After each set of block exams the Student Liaison Committee meets to review the learning in each course • Block Exams approximately every 3 wk period • Each course is given 10-12 minutes for discussion • PY1 (#7) PY2 (#5) PY3 (#3) • SLC members represent their entire class • Solicit feedback prior to meetings

8. During the Meeting • Use a systematic 3-question format • Responses are discussed, consensus is reached and recorded • Data is organized into a report for the instructor • Report is sent electronically to the instructor and course coordinator 48 hours post meeting

9. Feedback Questions • What elements in the course helped students learn the material in this course? • What elements in this course hindered the learning? • What specific suggestions do you have to improve the course?

10. Post Meeting • The instructor takes a few minutes of subsequent class to respond to student feedback in the SLC report and to describe any changes that will occur to improve learning • Facilitator will arrange a follow-up session with the instructor ‘prn’

11. Evaluation of SLC Process

12. Evaluation of SLC • Spring 2005, conducted a survey to determine student perceptions about SLC • Purpose • Comfort providing input • Effectiveness of process • Accuracy of process • Benefits • Areas for improvement • Specific questions for the SLC members

13. Survey Response Rate • All 3 classes combined = 85% • PY1 = 90 • PY2 = 80 • PY3 = 85

14. I understand the purpose of the SLC Mean SD=1 D=2 A=3 SA=4

15. I feel comfortable providing information to the SLC member(s) for discussion at the next SLC meeting Mean SD=1 D=2 A=3 SA=4

16. The SLC is an effective way to communicate class perceptions to faculty and administration Mean SD=1 D=2 A=3 SA=4

17. The SLC accurately reflects the perceptions of the entire class Mean SD=1 D=2 A=3 SA=4

18. As a member of the SLC, the time required to participate was worthwhile Mean SD=1 D=2 A=3 SA=4

19. As a member of the SLC, I understood what was expected of me. Mean SD=1 D=2 A=3 SA=4

20. Benefits • Opportunity to voice class opinions to communicate concerns to faculty • The faculty response to the SLC feedback

21. Areas for Improvement • Feedback gathered from students by their SLC members before the meetings • Concern about selecting their own members to the SLC • Feedback from SLC members about the meeting

22. Suggestions • More effective methods to obtain student input before these meetings: • Email • Class announcement • Form • Summary of the meeting results to the class

23. References • Clark, J and Redmond, M. Small Group Instructional Diagnosis Final Report. Settle: Department of Biology Education, University of Washington, 1982. (ED 217-954). • Coffman, SJ Improving Your teaching Through Small-Group Diagnosis. College Teaching, 1991, 39(2), 80-82. • Angelo, TA., and Cross, KP. Classroom Assessment Techniques: A Handbook for College Teachers. San Francisco: CA. Jossey-Bass, 1993. • Wimer, M. Improving College Teaching: Strategies for Developing Instructional Effectiveness. San Francisco: CA. Jossey-Bass, 1990.