Kenneth Janz , Ph.D. AVP and CIO Winona State University

1. Improving Assessment Efforts Integrating Educational Technology Wednesday, March 16, 2011, 8:30 – 9:15, Seville West EDUCAUSE Midwest 2011, Chicago, Illinois Ken Graetz, Ph.D. Director, TLT Winona State University Kenneth Janz, Ph.D. AVP and CIO Winona State University

2. Overview of Presentation • Overview of Winona State University (WSU) • e-Warrior: Digital Life and Learning Program • Assessment of Educational Technology • e-Warrior Assessment Plan • Key Findings • Ongoing Assessment Challenges • Questions

3. Founded in 1858 (Normal School) Public State Assisted Winona, Minnesota 9,000 students

4. Required Participation Dual-platform lease program since 1997 $480 semester

5. e-Warrior Video Goes Here

6. Winona is Unique…….. • Space Utilization (Currently at 93%) • Anytime, Anywhere Learning • Computer Labs • Computer Self-Efficacy • Access to Software • Innovative Pedagogy • e-books • enhanced communications • Faculty Can Count On What Students Can Bring To Class

7. Interesting Data…… Faculty Recruitment Awards Attracting Students Rankings The federal IPED’s report indicates Winona State’s five-year undergraduate graduation rate leads all other MnSCU Universities by at least 10 percent (IPED, 2009). 1997 WSU graduation rate was just over 40%.

8. *Core based on Fink (2003, p. 60 – 101)

9. What Makes the Target Hard to Hit? Greater academic integration New devices Better wireless access More applications Aging laptop programs Shifting pedagogies Budget cuts Device backlash

10. Where Are We Going • Winona State University Technology Master Plan • Runs July 2010 – June 2013 • Framework for integrating and prioritizing technology related needs

11. Research Questions Based on Conceptual Model *Core based on Fink (2003, p. 60 – 101)

12. Measuring How We Are Doing • Building Longitudinal Metrics • Student Surveys • Faculty Surveys • Alumni Surveys • Faculty / Student Focus Groups • Collect Course Information • Presentation / Publications • Existing Student Success Data • Facilitating Activities • Unit HLC Assessment

14. Multiple Measures (1 of 2) • Annual Faculty and Student Technology Survey • Faculty and Student Training Counts • Faculty and Student Training Evaluation Survey • Alumni Survey • Faculty Portfolio • Archival Log Data • Help Desk Satisfaction Survey • Faculty and Student Focus Groups

15. Multiple Measures (2 of 2) • Review Selected Course Information • Review Selected Course Assignments • e-Warrior Digital Life and Learning Program meeting attendance and minutes • Showcase Survey • Presentation and Publication Counts • Number of inquiries and visits to learn about the program. • Existing Student Success Data • Educause Core Data • Application and Course Specific Measures

16. Key Findings and Measures

17. Student-Related Questions • Are students satisfied with the tools? • Do first year students find the program “lame” • Do Seniors become disillusioned with the program? • Are students adding tools not provided through the program? • Do students use the tools for academic purposes extensively? • Do they use the tools more for academic vs entertainment? • Do they use the tools inside the classroom? • Does academic tool use vary by year in school? • How do students evaluate their instructors’ tool use? • Do perceptions vary by year in school?

18. Student satisfaction (5-point scale)

19. What grade would you give the e-Warrior Digital Life and Learning Program overall? • “A” = 38% overall • “B” = 53% overall • Both are satisfied, but Seniors gradethe program lower

20. On average, approximately how many hours per day do you use your laptop? • Normally distributed, averaging 5-6 hrs/day • No differences between first years and seniors

21. Of the total average time that you use your laptop during a typical day, how much of the time do you use it for class-related purposes? • Positively skewed, averaging 3-4 hrs/day • No differences between first years and seniors • Use their laptops fornonacademic purposesabout 2 hrs/day

22. On average, in the courses you use your laptop while in class, how frequently do you use it in class? • Students use laptopsin class 2-3 coursesper term • Students do notuse laptops inclass everyday • Seniors use laptops in classmore freq. thanfirst year

23. Application frequency (5-point scale)

24. Added Software • Communication Tools • Skype • Yahoo Messenger • Creativity/Productivity Tools • FinalCut Express • Audacity • Google Chrome • Picasa • Evernote • Soundflower • Entertainment Tools • VLC Player • Academic Tools • LaTeX • Dim Sum • Google Earth • Nook Study, other ebookreaders

25. Use D2L • I feel that all instructors should use the D2L system as it is a great way for students to know where they stand in the course, because they are able to view their grades regularly. Therefore, it would be nice if instructors would update grades for students as well on the site. Another reason D2L is a great system is that it helps with contacting classmates, and is great for instructors to update with all of the information from each class. • I like D2L but a lot of the Minne professors don't seem to use it much. Too much paper still handed out!! not green

26. More Engagement • Using photos or YouTube clips to supplement their lesson is one thing that a couple of teachers have done, that I found very beneficial. • Instructors need to understand that just reading off slides does not teach students effectively. Instead, incorporate examples or class activities or even more hands on learning. If it is a proven fact students learn more from hand on activities, you think the college would require material to be taught through hands on techniques. • Don't just read right off the slide you made, it's boring and no one learns anything.

27. Less Technology in the Classroom? • Thank you finally a question I have a strong opinion on. If your going to require laptops and charge as much as you do for them, no professor should be allowed to say you can't use them to take notes in class. I am well aware of the distraction they present. Figure something out, you cant be a laptop school if you have to take notes in one or two classes each semester by notebook. • I, personally, would prefer the instructors to use less technology because it is easier for me to learn by doing something myself, rather than a computer/calculator to do it for me.

28. Relationship With GPA • Positive correlation between cumulative GPA and frequency of laptop use in class; moderated by year in school • First year: r = +.19 • Senior: r = +.23

29. Student-Related Questions • Are students satisfied with the tools? • Do first year students find the program “lame” • Do Seniors become disillusioned with the program? • Are students adding tools not provided through the program? • Do students use the tools for academic purposes extensively? • Do they use the tools more for academic vs entertainment? • Do they use the tools inside the classroom? • Does academic tool use vary by year in school? • How do students evaluate their instructors’ tool use? • Do perceptions vary by year in school?

30. Faculty-Related Questions • Are faculty satisfied with tools and services? • Are faculty using tools in their courses effectively?

33. Faculty Exchange Faculty Exchange Video Goes Here

34. Ongoing Assessment Challenges • IMS Analytics • Assessment Related to Faculty Development

35. Questions