Assessing Undergraduate Information and Technology Skills: Opportunities, Outcomes and Challenges Arizona State Univers

1. Assessing Undergraduate Information and Technology Skills:Opportunities, Outcomes and ChallengesArizona State University West Bee Gallegos Dennis Isbell Lisa Kammerlocher

2. Assessment Opportunities • North Central Accreditation imperative to demonstrate learning outcomes • AZ Board of Regents approves lower-division offerings • Previous self-assessment pilot for information and technology skills entrée to developing performance-based assessment

3. Project Goals • create a reliable web-based Information and Technology Skills Assessment (ITSA) • pilot ITSA and it’s administration to determine if it would be manageable and sustainable as an ongoing program • conduct pre and post-assessments of the first-year cohort, with the pre-assessment given early in the Fall 2001, and the post-assessment at the end of their academic year in the Spring 2002 • establish a baseline for incoming first-year student skills to use in future comparisons • determine if a commonly delivered library curriculum in UNI 101 and English (ENG) 102 results in an increased level of information competency for first-year students • determine if gender or race/ethnicity is a factor in the level of information competency gained by first-year students

4. Library Curriculum • Virtual and Experiential Tour Assignment • using services – reference and circulation • using collections – reference, media, journals, and books • using the catalog, a database • Plagiarism Unit

5. Development of the ITSA • Based on ASU West Information Competencies • Used models from James Madison University(4), University of California Los Angeles(5), and California State Universities(6) • Tested with library staff and students • Engaged assistance from database specialist to design web-based interface • Sought research expertise from colleagues across campus

6. Attributes Measured • Recognizing catalog and database structures and records • Identifying citations and attributing authorship • Evaluating information sources • Recognizing and applying search techniques • Recognizing effective research plans and search strategies

7. Administration of the ITSA • Negotiated access to UNI 101 courses for pre- assessment in early fall semester • Attempted to assess whole population (with no control group) as all students were enrolled in UNI 101 • Negotiated access to ENG 102 courses for post-assessment in late spring semester

8. Demographics (Arizona State University West, Institutional Planning and Research, 2002)

9. a p < .001, at 95% confidence level d p < .001, at 95% confidence level b p < .019, at 95% confidence level e p < .007, at 95% confidence level c p < .069, at 95% confidence level f p < .006, at 95% confidence level

10. Results – Figure 1

11. Results – Figure 2

12. Discussion • Information competency improved overall and in 4 of the 5 attributes • Suggests emphasis on information competency in curriculum influenced improved student performance (2) • Females increase in level of information competency was significant • Gender gap in education is shrinking (3) • Males increase in level of information competency not significant • Sample was too small to be conclusive • Ethnicity was not a factor in performance on the ITSA

13. Future Directions • Continue to improve reliability of the ITSA • Modify learning experiences in the attributes of citation and attribution, search techniques and research plans and search strategies • Seek improved opportunities for administration of the ITSA • Revisit how the ITSA will be used… pre-post vs. measure of competency at various academic levels • Establish score that defines information competency (4)

14. Bibliography • Arizona State University West, Institutional Planning and Research. (2002, Feb.) Student Characteristics Summary, Fall - 2001. • Arizona State University West. (2002). College Student Experiences Questionnaire, Cohort01-02, Posttest. [data file] • Campbell, J.R., Hombo, C.M. & Mazzeo, J. (2002). NAEP 1999 Trends in Academic Progress: Three Decades of Student Performance. Washington, D.C.: U.S. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics. (National Center for Education Statistics Report No. NCES 2000-469). • Cameron, L. & Feind, R. (2001, March 15-18). An Online competency test for information literacy: Development, implementation, and results. Paper presented at the Association of College & Research Libraries Tenth National Conference, Denver, CO. • Carvello, P.S., Herschman, J. & Mitchell, E. (2000, March 15-18). Assessing the Information Literacy of undergraduates: Reports from the UCLA Library’s information competencies survey project. Paper presented at the Association of College & Research Libraries Tenth National Conference, Denver, CO. • Dunn, K. (2002, Jan-March). Assessing information literacy skills in the California State University: a progress report. Journal of Academic Librarianship, 28(1), 26-35.

15. Presentation Web Page http://westcgi.west.asu.edu/libcontrib/northumbria03 • Literature Review and Bibliography • Access to Information and Technology Skills Assessment • First-Year Library Assignments • Information Competencies, Attributes and Fletcher Menu • PowerPoint Presentation • Contact Information