EPIC Online Publishing Use and Costs Evaluation Program

1. EPIC Online Publishing Use and Costs Evaluation Program

2. Overview • Overview of Evaluation Program • Research update

3. Purpose of EPIC Evaluation Program • To investigate how online electronic resources affect different aspects of the scholarly communication process. • a) general perspective • b) through lens of EPIC projects

4. How Does the Shift to Electronic Resources Affect the Following Groups? • Publishers • Administrators/Information Technology • Librarians • Faculty • Students

5. Research Approach • Interviews • Focus Groups • Surveys • Weblog analysis Info gathering Survey development

6. Research Projects to Date • Librarian Focus Group • Librarian Online Survey • Faculty Interviews • Faculty Online Survey • Student Interviews • Student Online Survey • http://epic.columbia.edu/eval/

7. Librarian Online Survey

8. Methodology • a 15 minute online survey of librarians was conducted in Winter, 2002 • e-mail invitations were sent to librarians at higher education institutions across the country • All were librarians at institutions that are either CIAO or Earthscape subscribers, or have been identified as being potential subscribers • 1007 respondents (response rate: 32%)

9. Objectives • Learn what librarians perceive as the benefits and disadvantages of electronic resources. • Gain insights into how electronic resources affect the role of the librarian. • Gain insights into how electronic resources affect the role of print resources.

10. Conclusions from Librarian Survey • Electronic Resources: • Allow libraries to serve patrons more effectively • Put a strain on some libraries • Do not replace print resources

11. Faculty Survey

12. Methods • a 25 minute online survey of faculty and graduate students was conducted in Winter 2003 • e-mail invitations were sent to scholars at higher education institutions across the country • scholars were involved in the disciplines of International Affairs or Earth Sciences • 845 respondents (response rate 14%)

13. Objectives • Gain insights into how academics are using electronic resources in their academic work • Gain insights into the perceived benefits and disadvantages of electronic resources • Learn how academics perceive electronic resources as affecting undergraduate students’ scholarship • Examine differences based on stage of career, type of discipline, and type of institution

14. Respondent Profile

15. Gender & AgeN=845

16. Position & Type of InstitutionN=845

17. Academic Responsibilities and Field of StudyN=845

18. Electronic Resources and Academics’ Work Habits

19. Use of and Proficiency with Electronic ResourcesN=845

20. Electronic Resources’ Affect on Work HabitsN=845

21. Work Habits and The LibraryN=845

22. Perceived Benefits and Disadvantagesof UsingElectronic Resources

23. Benefits of Electronic Resources--ConvenienceN=845 Slide 1 of 2

24. Benefits of Electronic Resources—Improved Availability N=845 Slide 2 of 2

25. Disadvantages of Electronic ResourcesN=845

26. Benefits and Disadvantages by DisciplineN=845 All of the differences in means by discipline on this chart are statistically significant.

27. Benefits and Disadvantages by SeniorityN=845 All of the differences in means by seniority on this chart are statistically significant.

28. Teaching with Electronic Resources

29. Use of Electronic Resources for TeachingN=419

30. Use of Electronic Resources for Teaching by DisciplineN=419 All of the differences in means by discipline on this chart are statistically significant.

31. Electronic Resources & Students—BenefitsN=419 Slide 1 of 2

32. Electronic Resources & Students—ProblemsN=419 Slide 2 of 2

33. Class Time Spent Instructing Students How to Evaluate Online InformationN=419

34. Availability of Courses Instructing Students How to Evaluate Electronic ResourcesN=419

35. The Effect of Electronic Resources on Overall Quality of Students’ Learning ExperienceN=419 MEAN4.00

36. Satisfaction with Electronic Resources

37. Importance of and Satisfaction with ER

38. Importance of and Satisfaction with ER

39. Ability to download Ability to print Currency of info Speed of sites Reputation of providers Text in PDF or other standard format Full text available Search capabilities Ease of Navigation Availability of older/archived information Ability to tell if peer reviewed Important and Satisfied Important and Not Satisfied

40. Other suggestions • Reduce price • Standardize navigation and search procedures across databases • Synthesize databases so that they are accessible through a single portal • Faster turn-around time/More frequent updates • Clearly identify or label sources

41. Conclusions • Electronic resources have become the main tool for scholars’ information gathering. • Electronic resources are seen as providing convenience more so than increased availability of information. • The main problem with electronic resources is having to separate out reliable from unreliable information. • Most faculty believe students have an enhanced learning experience because of the information available through electronic resources. • Future efforts could focus on providing increased access to older and full text material, and improving search and navigation capabilities.

42. Christina Norman, Ph.D. Research Director, EPIC Columbia University 211 International Affairs Building 420 West 118th Street, Mail Code 3301 New York, NY 10027 cn2005@columbia.edu EPIC evaluation web site: http://epic.columbia.edu/eval/