Evaluation & MathDL

1. Evaluation & MathDL Flora McMartin Broad-based Knowledge flora.mcmartin@gmail.com MathDL Workshop, October 2008

2. What We Know about Math Users Institution Type MathDL Workshop, October 2008 National survey of faculty (4,678 respondents, 242 Math)

3. Types of Digital Resources % of respondents reporting Very Frequent (VF) use (top box score) MathDL Workshop, October 2008

4. Most Popular uses of Digital Resources MathDL Workshop, October 2008

5. Importance of DL Features Percentage of respondents who ranked feature as most important MathDL Workshop, October 2008

6. Motivations for Using Digital Resources Scale: 1 = strongly disagree, 5 = strongly agree MathDL Workshop, October 2008

7. In Sum MathDL Workshop, October 2008 • No direct relationship between highly valuing educational digital resources and level of use • Faculty prefer search for finding materials • Barriers to use cannot be simply described

8. Overcoming Barriers to Use of DLs • Activation energy – how much? What is it? • Intrinsic motivation is a powerful force – how big a reservoir? • Students as extrinsic motivators • Tradition is another powerful driver (change is hard) • All work practices are different • Never underestimate the willingness of a motivated individual to do their own thing (or ignore you) MathDL Workshop, October 2008

9. Application to the MathDL MathDL Workshop, October 2008 Do these findings apply to MAA members? Study done in 2006, how have changes on Web since then (e.g., growth in popularity of Youtube, iTunes U, blogs, wikis, etc.) affected users? MathDL and MAA sites have changed what does that mean to users? And?

10. Evaluation, next steps MathDL Workshop, October 2008 • Need to understand user needs, preferences and behaviors • Are they aware of it? • Can they find it? • If they find it, will they use it? • What does it allow users to do that they could not do before? • Need to understand the environment and competition to • Act strategically • Develop marketing plans • Make strategic partners • Measure success and affordability (know when experiments work or fail)

11. User Panels • Panels repre WHY? • Users becoming more sophisticated and fickle –online survey response rates trending down • Ability to track users’ attitudes, behaviors over time • Panels carefully selected to match demographics of audience • 20 + panelists • Respond to 3 – 5 surveys over course of year • Potential Questions • Usability – e.g., is the site intuitive? • How do users move between MAA and MathDL? • How does use of Google work for/against use of MathDL? MathDL Workshop, October 2008

12. The Plan Selection Criteria • Teaching Experience • Types of School/institution • Type of course (online, traditional) • Use of Web/DL resources (professional development, integrated into course) • Math Course Topics (Calc, remedial math, etc.) 2 panels • Novice • Previous users of MathDL Implemented in 2009 MathDL Workshop, October 2008

13. More about Research on Faculty Use of the NSDL MathDL Workshop, October 2008 • For more details visit • http://serc.carleton.edu/facultypart • We wish to acknowledge the National Science Foundation for their support (DUE-0435398). The findings presented here do not necessarily represent the views of the NSF. Researchers: • Alan Wolf, University of Wisconsin, Madison • Josh Morrill, Morrill Solutions Research • Glenda Morgan, George Mason University • Ellen Iverson, Carleton College • Cathy Manduca, Carleton College • Flora McMartin, Broad-based Knowledge