2009 Professional Development Day

1. Student Survey of Instruction (SSI): Increasing Response Rates and Accessing Results 2009 Professional Development Day October 2009

2. Student Survey of Instruction Presenters • Jesse Coraggio, Director, Academic Effectiveness • Deb Montalvo,SSI Coordinator and Coordinator, Placement & Career Services Academic Effectiveness

3. Purpose • The SSI has undergone a number of changes during the last year, including a move to an online environment and improved access to results. Learn about these recent changes and best practices for increasing your student participation, as well as how to access your own SSI results online. Academic Effectiveness

4. Original SSI Items • Original SSI forms developed by committee • Items reviewed for content and lecture form aggregated into four categories (e.g., faculty student interaction, organization, presentation, and evaluation) • There are four forms (Scantron): Lecture, Non-lecture, Clinical A*, and Clinical B* • Results scanned by PCSB and entered into PeopleSoft Academic Effectiveness

5. eCampus SSI Items • Online SSI form was developed by eCampus to assess online courses • Form contained exact or slightly modified items as in Lecture SSI form with same four categories • Original online survey tool created to capture summative information in database Academic Effectiveness

6. Issues with prior process • Expenses related to Scantron forms, and numerous hours of sorting, organizing, and transporting forms. • Time consuming process of transcribing open-ended items. • Not all Scantrons were scanned and processed (scanning issues, data integrity issues, etc.) • Scantron results and online results captured separately and reported separately • Process conducted by multiple departments with no central ownership Academic Effectiveness

7. Performance Improvement • From Compliance to Performance Improvement Academic Effectiveness

8. New Online Survey Process • New SSI process designed to: • Provide stakeholders (Faculty, Program Administrators, etc.) information to assist in making decisions that will result in performance improvement • Provide more reliable and valid course evaluation information • Create centralized coordination of the process • Save the College time and money (est. $45,000 per year) • Provide more course time for instruction Academic Effectiveness

9. SSI Online Process Timeline • Fall 2007 • Established SSI Team • Factor analysis determined that lecture and online SSI forms have three factors (e.g., faculty engagement, preparation and organization, and course instruction) • Spring 2008 • 100 section pilot of FTF courses conducted • Fall 2008 • First collegewide administration (All ANGEL shells enabled) • Created an online reporting tool (PAs only) • Revised older survey tool Academic Effectiveness

10. SSI Online Process Timeline • Spring 2009 • Created customized usage reporting tools • Created new survey interface with omit capability • Added Faculty Representative to SSI Team • Created online reporting tool access for Faculty (No more paper) Academic Effectiveness

11. SSI Online Process Timeline • Summer 2009 • First collegewide summer administration • Modified faculty email notices to include link to real-time response data. • Began working with HEC on Clinical Solutions • Evaluated current SSI items and forms (SSI Revision Committee) • Fall 2009 • Implement self-service module so faculty can revise administration dates Academic Effectiveness

12. http://www.spcollege.edu/central/ssi/ Information about the Process Academic Effectiveness

13. http://www.spcollege.edu/central/ssi/ Information about the Process Academic Effectiveness

14. Accessing Results • All SSI survey reports are now available on the online SSI Administration site: http://it.spcollege.edu/ssi/ • Login by entering your SPC network username and password and selecting the Login button Academic Effectiveness

15. Accessing Results Academic Effectiveness

16. Accessing Results • Select Term from the drop-down menu (i.e. 0405) • Select how you would like to view the results • Each individual section with its own results OR Combined sections by instructor • Select the Search button Academic Effectiveness

17. Accessing Results • After selecting a term and indicating ‘Each Individual Section with its own results’… • View each individual section with its own results, as selected on the previous screen for Term: 0405. • Select any of the courses to view the SSI report for that course. Academic Effectiveness

18. Accessing Results • View the SSI results for the course selected • Select the “+” next to your Acad. Org to view averages for similar SSI survey forms (2-Year Online, Lecture, Non-lecture, etc.) within that academic org (listed as SOCSCI-LD in this example). Academic Effectiveness

19. Accessing Results • After selecting a term and ‘Combined sections by Instructor’… • View combined sections with results, as selected on the term selection screen (Term: 0405 in this example). • Select any of the instructor names to view their combined SSI report for that course. Note: multiple reports for a single instructor may be listed if the instructor taught courses with either different SSI survey forms, different academic orgs, or on different campuses. Academic Effectiveness

20. Accessing Results Academic Effectiveness

21. Accessing Results • Select ‘View in Adobe Reader’ for an electronic version of the report (pdf) • ‘Each Individual Section with its own results’ report produces individual SSI results, three comparative measures, graph, and student comments. • ‘Combined sections by Instructor’ report produces an instructor’s combined sections SSI results with three comparative measures, graph, and student comments by course. This report is somewhat similar to the SSI reports generated by AIS in the past. Academic Effectiveness

22. Changing Administration Dates • SSI is intended to be an evaluation of the instruction provided during the course. • It is administered after the students have participated in the majority of the course (no sooner than the 60% mark of the course) in order to provide the student an opportunity to experience the course instruction. • Length of the administration period is approximately the last 20%-25% of the course. • Faculty are provided default SSI administration start and end dates, and are permitted to make minor adjustments to these dates within a predetermined timeframe. This ensures a more standardized and consistent data collection process. Academic Effectiveness

23. Changing Administration Dates Locate the course you would like to change…  Select the Calendar icon Academic Effectiveness

24. Changing Administration Dates • Click on “Next Month” on the upper right hand side of the calendar to bring you to December. The sample calendar above shows the course ends on 12/18 and the SSI closes on 12/11. Academic Effectiveness

25. Changing Administration Dates • To extend the SSI to remain open until 12/18… • Select 12/18. It will turn blue through 12/18. This changes in real-time. • Should you make a mistake, simply click back on the original survey end date of 12/11 or select another. • The latest you can extend the survey to is 12/24. Academic Effectiveness

26. Response Rates Academic Effectiveness

27. Response Rate Comparison • Palm Beach Community College Online Course Evaluation • Palm Beach Community College debuted its online course evaluation system in Fall 2007. Until the Spring 2007 term, PBCC relied on a paper-based course evaluation system for its traditional classroom courses. • Response rates:Pilot testSpring 2007: 30% classroom, 26% distance learningCollegewideFall 2007: 20% classroom, 26% distance learningSpring 2008: 24% classroom, 29% distance learning Academic Effectiveness

28. Increasing Response Rates • While not the primary focus of the SSI Revision Committee (Summer 2009), there was considerable interest in identifying and implementing ideas and processes to increase the response rate on the SSI. • Through creative brainstorming at the meetings and online, a list of “best practices” was crafted. Academic Effectiveness

29. Increasing Response Rates SSI Revision Committee Recommendations • Send Constant Reminders • Stress Confidentiality • Imbed link in announcement page -- “inescapable messaging” • Move every week to top of unit • Link to “content” – graded or non-graded • Bring students to lab /use of lab • Give extra credit • Genuinely convey the message that the information will be used to make this class better • Hold activities on campus, e.g., a table highlighting importance for accreditation  Academic Effectiveness

30. Increasing Response Rates • There was some concern that item #5 (linking the SSI to accessing exams or other course materials) was not acceptable from a legal standpoint. • The question was posed to the Office of General Counsel, and, in brief, the answer was ‘yes’, the faculty could block access to course materials assuming that the content “blocked” is just the next module or assignment the student would see, not specifically an exam. • The Office of General Counsel also advised against giving extra credit for taking the SSI. Academic Effectiveness

31. SSI Resources • ‘Help with the SSI’ Section of website • SSI Frequently Asked Questions (FAQs) document • For questions or comments about the SSI process - contact your dean, program director, or academic chair • For technical assistance - contact your campus instructional technologist • SSI Administrator - mailto:SSI_Admin@spcollege.edu • Troubleshooting SSI links COMING SOON! Academic Effectiveness

32. Questions Academic Effectiveness

33. Student Survey of Instruction (SSI): Increasing Response Rates and Accessing Results 2009 Professional Development Day October 2009