WCC Increased E valuations from 16% to 85%

1. EdTech Survey Summary WCC Increased Evaluations from 16% to 85%

2. Your Presentors SAMER SAAB CEO, explorance EMAIL: ssaab@explorance.com TEL.: +1.514.836.7264 FRANçOISBénéteau Solutions Engineering Manager EMAIL: fbeneteau@explorance.com TEL.: +1.514.938.2111 ext. 228 Gloria Eccleston Director, Online Learning and ServicesWashtenaw Community College EMAIL: geccleston@wccnet.edu TEL: +1.734.477.8903

3. About Washtenaw Community College • Location: Ann Arbor, Michigan United States • Enrollment: ~12,800 / semester • Number of programs:>125 • Number of online programs: 23* • Students taking at least one online course: ~6,800** • Full Time online students: ~2,400** • Learning Management System: Blackboard Learn • Student Information System: Banner * Adding 2-4 programs annually ** >18% growth each semester

4. Partner College beginning 2016 • Bluepulse– The Genesis of Social Feedback (Jan/16) • Blue – The All-In-One Evaluation Software (Jan/17) • Blue Analytics – Making sure there are no blind spots (Jan/17) • DIG – Data Integrity Gateway (Jan/18)

5. WCC’s Summative Evaluations • Evaluations are called Student Opinion Questionnaires (SOQs) • Same questions are used for all modalities (Online, Blended and Face-to-Face) • We have multiple evaluation periods beginning 3-weeks into the semester and concluding 2-weeks before the semester ends. • Some evaluation periods open for 1 day, others 15 days • Evaluation periods overlap

6. Student Opinion Questionnaires • Historically, data has been used to measure faculty performance • Contract Agreement between the Faculty and the College with extremely specific language about: • The questions • Specifically how the SOQ is administered • Validity and reliability of the data • When and how the data can be used

7. The Problem • Online course response rates averaged 16% • All data had to be thrown out • No feedback for faculty or accountability measures for the College • Our office was given 1-year to turn this around

8. The Problem (Continued) • WCC had been using a home-built system for managing online SOQs • Designed by programmer no longer employed at WCC • Five departments in two divisions could not determine ownership • No champion to support the administration of the evaluations or provide faculty support • Therefore, no confidence from any stakeholder group • Chaos or Opportunity?

9. WCC Contracts with explorance • Needed a solution that would satisfy the needs of: • Institutional Research • Vice President of Instruction • Faculty Union • Online Learning Division • Blackboard Learn Administration • Systems Administration • Development Operations

10. Five Semester Ramp up to Fully Online • Winter 2017 Student Opinion Questionnaires (SOQs) - 84.6% • Online SOQs for DL Online sections, face-to-face F2F and Blended on traditional paper • Spring/Summer 2017 - 47% • Online SOQs for DL and MM sections, Blue/ABBY paper for select F2F • Fall 2017 -51% • Online SOQs for DL, MM and select F2F, Blue/ABBY paper for all F2F • Winter 2018 • Online SOQs for DL, MM and Business/Computer division, balance paper with Blue & ABBY • Spring/Summer 2018 - 65.5% • Online for all SOQs

11. Timeline to Success April January

12. WCC’s Phase One – Results? • Winter 2017 – a snapshot • 242 online sections • 5217 surveys • 4319 completed, 819 not completed • 84.6% Response Rate

13. WCC’s Phase One – Head First • Winter 2017 – what did we do? • Communication Plan (Faculty approved) • Blackboard Access • Blackboard Course Site Blocking

14. WCC’s Phase One – Head First Winter 2017 – what did we do? • Communication Plan (Faculty approved) • Intentional actions with ‘strategic’ timing to students and faculty • Schedule published everywhere and anywhere • Guides for Faculty and our 24/7 Help Desk • Email updates – open communication channels • Simple Blue emails and reminders with links • Help Desk Support • Specific SOQ email with ticketing system

15. WCC’s Phase One – Head First

16. WCC’s Phase One – Head First Winter 2017 – what did we do? • Blackboard Access • NOTE: using two LMS instances W’17 • Tabs - Student SOQs Online and Faculty SOQs Online with link to SOQs • Use BPI Building Blocks • Links within each Bb course site • Tool link to LTI as well as a Course link • Institutional Level Pop-ups at log in • Use Blue Deep Integration

17. Specific roles in Blackboard required to access Bb tabs • Schedule of SOQs • Response module • Faculty reports Maximizing Blackboard IntegrationInstitutional Tabfor Faculty

18. Maximizing Blackboard IntegrationCommunity Tabs (Students)

19. Maximizing Blackboard IntegrationPop-Up at Log in • Student logs into Blackboard and immediately receives a pop-up • Shows all availableevaluations • Options: • Select an SOQ to enter Blue • Select “Remind me Later” to enter Blackboard • *Note: 70% through evaluation period the pop-up changes and denies access to course

20. Maximizing Blackboard IntegrationSOQs Online link in each Bb course site A tool link with a course link – the only way the blocking process will work!

21. WCC’s Phase One – Head First Winter 2017 – what did we do? • Blackboard Course Site Blocking • Restrict access to Blackboard course site • Strategic timing • Validation of data by WCC’s Institutional Research team • Feedback

22. Maximizing Blackboard IntegrationPop-up Blocks (Student) • Students are informed multiple times: • By the instructor’s communications • Blackboard Announcements • Automated and custom Blue emails • Standard Operating Procedure • Block initiated at approx. 70% through evaluation period • Student can only complete the evaluation, or go back to the Blackboard homepage Button changes to “Back to Institution Page”

23. Maximizing Blackboard IntegrationPop-up Blocks (Student) • Blue is intuitive enough to know which courses to block the student from • Blue blocks only ACTIVE tasks, not expired, deleted, or not ready surveys • Use DIG to fine tune this • Students can access a course if the evaluation has been completed • To our knowledge students cannot circumvent the block Button changes to “Back to Institution Page”

24. Maximizing Blackboard IntegrationPop-up Blocks (Student) • Required faculty support • Faculty wanted increased response rates and student feedback • Most, not all, were in favor of this initiative • Concerns will be addressed later • Faculty were requested to place high-stakes assignments in Blackboard during the evaluation period • Higher sense of urgency for students to gain access to Blackboard

25. Course Site Blocking - InstitutionalConcerns • Creating a negative (punitive) culture around evaluations • Student complaints • Face-to-face does not restrict course access • Institutional Research – impacting the validity of the data • Will the block variable impact the ratings of the professor? • Will students who are forced to complete evaluation rate lower due to frustration or anger? • Are there legal issues?

26. Concerns Addressed - Creating a Negative Culture • For our institution, the need for increasing response rates was primary concern • Without the data, another semester would be lost • We did a media campaign to explain the block prior to evaluations • Positive tone • Interviews with faculty expressing what they do with feedback • Interviews with students expressing the importance of giving feedback • There were a few students (about 6) that were very vocal about the block • Simply removed the Blue task and allowed the student to enter course • Most students that were unhappy, still completed the evaluation

27. Concerns Addressed - Institutional Research • IR was skeptical about the impact of the block • IR analyzed the pre-course block data and post-course block data … • No measurable change, thus the impact was insignificant! • Note: Explanation of the data IR obtained to measure pre and post block data.

28. Concerns Addressed - Legal Implications • The institution has a policy about denying students access to their grades. Students can still access grades through Banner. • If a student requested the course be unblocked, we honored their request.

29. Key Actions to Increase Participation • Communication • Strategic timing of automated emails to students with evaluations embedded • Strategic reminders to faculty • Response Rate Monitor

30. Key Actions to Increase Participation • Emailed divisional and departmental participation updates • Created the buzz and enthusiasm and kept it alive • Internal Help Desk included in communications • Keep them “in the know” and part of active process • Developed a separate ticketing system • Responsiveness • Demonstrate institution priority

31. The Outcome – Spring/Summer 2018 • 100% online SOQs – NO PAPER!! • Savings of 8 to 10 trees EACH semester • 695 sections • With 10 days to go, response rate at 35%(12,191 surveys, 4336 completed and 1654 expired) • Student and faculty requests to reopen evaluations – they wanted to provide feedback and share opinion!

32. The Outcome – Spring/Summer 2018 • 100% online SOQs – NO PAPER!! • With 10 days to go, response rate at 35%(12,191 surveys, 4336 completed and 1654 expired) • With 4 days to go, response rate at 41.41% • Course Block implemented • End of feedback gathering, response rate 65.55%

33. Challenges and Looking Forward • Overlapping evaluation periods • Course-block is manual • Numerous Blackboard Building Blocks • Configuration • Manual addition of the course-level links • Maintaining after LMS upgrades • Students annoyed, and don’t follow the guidelines provided • Demonstrate to students that their voice does matter with concrete examples of “you told us xxx, and we did xxx!

34. Blue – the evaluation software SAMER SAAB CEO, explorance EMAIL: ssaab@explorance.com TEL.: +1.514.836.7264 FRANçOISBénéteau Solutions Engineering Manager EMAIL: fbeneteau@explorance.com TEL.: +1.514.938.2111 ext. 228