1 / 11

Measuring Students' Progress in CS1: Creating a Reliable and Valid Assessment Tool

This research paper discusses the creation of a reliable and validated assessment tool for CS1 courses, which is independent of specific approaches or programming languages. It addresses the lack of existing tools for this purpose and explores the potential of using alternative assessments. The paper also outlines the proposed work, including field testing and assessment of reliability and validity.

latrisha
Download Presentation

Measuring Students' Progress in CS1: Creating a Reliable and Valid Assessment Tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How Students “Measure Up”: Creation of an Assessment Tool for CS1 Adrienne Decker Department of Computer Science & Engineering University at Buffalo, SUNY Dr. William J. Rapaport, advisor adrienne@cse.buffalo.edu March 3, 2004

  2. Introduction • CC2001 and previous curricula do not provide implementation instructions. • Faculty innovate courses based on suggestions and their own interpretations. • Investigations into the usefulness of these innovations most often use lab grade, overall course grade, resignation rate, or exam grades • Cooper, Dann, & Pausch, 2003; Decker, 2003; Ventura, 2003 • Problem: not proven reliable or valid. SIGCSE DC 2004

  3. Reliability & Validity • Reliability • “Degree of consistency among test scores” • Validity • Reliability + Relevance (Marshall & Hales, 1972) SIGCSE DC 2004

  4. Goals of Research • Create a reliable and validated assessment tool for CS1 • Independent of objects-first, imperatives-first, or functional-first approaches. • Independent of any particular language. SIGCSE DC 2004

  5. Motivation • No such tool exists. • Other tools are not validated or created for this purpose. • AP Exam • Not CC2001, specific language, prediction not assessment • GRE Subject Test & ETS Major Field Tests • End of 4-year undergraduate curriculum SIGCSE DC 2004

  6. Previous Investigations • Predictors Research • Kurtz, 1980; Mazlack, 1980 • Leeper & Silver, 1982; Evans & Simkin, 1989 • Hagan & Markham, 2000; Wilson & Shrock, 2001 • Ventura, 2003 • ITiCSE 2001 Working Group • Tested programming skill • Too mathematical? SIGCSE DC 2004

  7. Researcher’s Previous Work • Performance of Non-Majors • Used exam grades to assess mastery of material • Analysis of AP Exam • Correlation for AP CS A Exam • No correlation for AP CS AB Exam SIGCSE DC 2004

  8. Proposed Work • Analysis of CC 2001 Programming-First Approaches • Creation of Test • Relevance Assessed by Community • Field Testing • Assessment of Reliability • Assessment of Validity SIGCSE DC 2004

  9. Open Issues • Is this even possible/feasible/reasonable? • Intersection of programming-first approaches too small • Should have a language component\ • Current status: Final (?) draft of proposal to committee for defense this semester. SIGCSE DC 2004

  10. Bibliographic References • Cooper, S., Dann, W., & Pausch, R. (2003). Teaching objects-first in introductory computer science. Paper presented at the 34th SIGCSE technical symposium on Computer Science Education, Reno, Nevada. • Decker, A. (2003). A tale of two paradigms. Journal of Computing Sciences in Colleges, 19(2), 238-246. • Evans, G. E., & Simkin, M. G. (1989). What best predicts computer proficiency? Communications of the ACM, 32(11), 1322 - 1327. • Hagan, D., & Markham, S. (2000). Does it help to have some programming experience before beginning a computing degree program? Paper presented at the 5th annual SIGCSE/SIGCUE conference on Innovation and technology in computer science education. • Joint Task Force on Computing Curricula. (2001). Computing curricula 2001 computer science. IEEE Computer Society & Association for Computing Machinery. Retrieved October 30, 2003, from the World Wide Web: http://www.computer.org/education/cc2001/final/index.htm • Kurtz, B. L. (1980). Investigating the relationship between the development of abstract reasoning and performance in an introductory programming class. Paper presented at the 11th SIGCSE technical symposium on Computer Science Education, Kansas City, Missouri. SIGCSE DC 2004

  11. References - Continued • Leeper, R. R., & Silver, J. L. (1982). Predicting success in a first programming course. Paper presented at the 13th SIGCSE technical symposium on computer science education, Indianapolis, Indiana. • Marshall, J. C., & Hales, L. W. (1972). Essentials of Testing. Reading, Massachusetts: Addison-Wesley Publishing Co. • Mazlack, L. J. (1980). Identifying potential to acquire programming skill. Communications of the ACM, 23(1), 14 - 17. • McCracken, M., Almstrum, V., Diaz, D., Guzdial, M., Hagan, D., Kolikant, Y. B.-D., Laxer, C., Thomas, L., Utting, I., & Wilusz, T. (2001). A multi-national, multi-institutional study of assessment of programming skills of the first-year CS students. SIGCSE Bulletin, 33(4), 1 - 16. • Ventura, P. R. (2003). On the origins of programmers: Identifying predictors of success for an objects-first CS1. Unpublished Doctoral, University at Buffalo, SUNY, Buffalo. • Wilson, B. C., & Shrock, S. (2001). Contributing to success in an introductory computer science course: A study of twelve factors. Paper presented at the 32nd SIGCSE technical symposium on Computer Science Education, Charlotte, North Carolina SIGCSE DC 2004

More Related