Value added some clarification
1 / 18

value-added: some clarification - PowerPoint PPT Presentation

  • Updated On :

Value-Added: Some Clarification. Presented by: Keston H. Fulcher, Ph.D. Christopher Newport University Virginia Assessment Group 3/2/2007 Note: Thanks to Dr. John T. Willse (UNC-G) for help with this presentation. Overview. What is Value-Added? Context: Historical and Current Affairs

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'value-added: some clarification' - Lucy

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Value added some clarification l.jpg

Value-Added: Some Clarification

Presented by: Keston H. Fulcher, Ph.D.

Christopher Newport University

Virginia Assessment Group


Note: Thanks to Dr. John T. Willse (UNC-G) for help with this presentation.

Overview l.jpg

  • What is Value-Added?

  • Context: Historical and Current Affairs

  • Approaches to Assessing Value-Added

  • Misconceptions of Value-Added

  • Reservations

Value added what is it l.jpg
Value-Added: What Is It?

  • Value-Added is an analytical strategy to determine the degree to which students change from the beginning to the end of a program. Astin (1985) referred to this type of change as talent development.

Historical context l.jpg
Historical Context

  • 80s – Greater call for accountability for higher education regarding student learning; value-added was considerable part of discussion.

  • A few states embraced value-added (e.g., Missouri and Tennessee)

  • 90s early 00s – De-emphasis on value-added, more emphasis on minimum competency

  • Mid 00s – Value-added once again at center of general education discussions

Current context national l.jpg
Current Context (National)

  • Catalyst?

  • Spellings Commission

  • Recommendations (U.S. Department of Education, 2006) :

    • “Higher education institutions should measure student learning using quality assessment data.”

    • “The results of student learning assessments, including value-added measurements that indicate how much students’ skills have improved over time, should be made available to students and reported in the aggregate publicly.”

Current context national6 l.jpg
Current Context (National)

  • Spellings Commission is considering logistical issues.

  • NASULGC and AACSU are working on developing recommendations for a “Voluntary Accountability System.”

  • The College Learning Assessment (CLA) is recommended by many groups to assess student gains in higher-order learning.

Current context virginia l.jpg
Current Context (Virginia)

  • Core Competency Assessment – State Council advocating value-added assessment for student learning.

  • Assessment of Student Learning Task Force charged by State Council to work out logistics. Initially, all members assessment experts.

  • This task force now is composed of presidents, provosts, and three assessment experts.

Slide8 l.jpg

  • Several issues here; value-added is just one of those issues.

  • Let’s examine just value-added.

Approaches to value added l.jpg
Approaches to Value-Added

  • Cross-Sectional

  • Pre-Post

  • Residual Analysis

Cross sectional l.jpg

  • Cross-Sectional: Compare scores from (say) sample of seniors against sample of freshmen on same test.

  • Note: Weakest of three designs; doesn’t control for differences between samples.

Pre post l.jpg

  • Pre-Post: Same set of students takes same test or equivalent tests at two points in time (i.e., repeated measures).

  • Change scores represent value-added.

  • Note: Conceptually most straight-forward approach to value-added. Takes considerable time to collect difference scores.

Residual analysis l.jpg
Residual Analysis

  • Residual Analysis: Determined by comparing the difference between actual scores and the scores predicted by some variable (or a set of variables), usually SATs or ACTs - Approach used by CLA

  • Note: Logistically easier to implement but conceptually a bit off-target. Value-added from this approach is normative.

Misconceptions of value added l.jpg
Misconceptions of Value-Added

  • Misconception: Measuring change is always unreliable

  • In the context of pre-post design, five conditions must hold for change scores to be unreliable (Zumbo, 1999):

    • the correlation between testing occasion one and testing occasion two is a large positive value

    • the observed variance of the two testing occasions is equal

    • the true score variance at both occasions is equal

    • the reliability at both occasions is equal, and

    • the correlation between true scores and true change scores is negative.

Other misconceptions l.jpg
Other Misconceptions

  • Value-Added means standardized assessment across institutions.

  • Value-Added is the CLA instrument.

    • Any instrument could theoretically be used for value-added assessment.

Reservations of value added l.jpg
Reservations of Value-Added

  • Affected by attrition

  • Expensive (especially for pre-post)

  • Item memorization

  • Doesn’t answer all analytical questions

Value added summary l.jpg
Value-Added, Summary

  • Some limitations

  • Answers a great question

  • In terms of the national debate about standardized testing across the nation…

  • We should be clear. Are we actually criticizing value-added, or an instrument, or standardization across schools….???

References l.jpg

  • Astin A. W. (1985). Achieving Educational Excellence: A Critical Assessment of Priorities and Practices in Higher Education. San Francisco: Jossey Bass.

  • U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, D.C. Also available at

  • Zumbo, B. D. (1999). The simple difference score as an inherently poor measure of change: Some reality, much mythology. In Bruce Thompson (Ed.). Advances in Social Science Methodology, Volume 5, (pp. 269-304). Greenwich, CT: JAI Press.