value added some clarification
Download
Skip this Video
Download Presentation
Value-Added: Some Clarification

Loading in 2 Seconds...

play fullscreen
1 / 18

Value-Added: Some Clarification - PowerPoint PPT Presentation


  • 171 Views
  • Uploaded on

Value-Added: Some Clarification. Presented by: Keston H. Fulcher, Ph.D. Christopher Newport University Virginia Assessment Group 3/2/2007 Note: Thanks to Dr. John T. Willse (UNC-G) for help with this presentation. Overview. What is Value-Added? Context: Historical and Current Affairs

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Value-Added: Some Clarification' - Lucy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
value added some clarification

Value-Added: Some Clarification

Presented by: Keston H. Fulcher, Ph.D.

Christopher Newport University

Virginia Assessment Group

3/2/2007

Note: Thanks to Dr. John T. Willse (UNC-G) for help with this presentation.

overview
Overview
  • What is Value-Added?
  • Context: Historical and Current Affairs
  • Approaches to Assessing Value-Added
  • Misconceptions of Value-Added
  • Reservations
value added what is it
Value-Added: What Is It?
  • Value-Added is an analytical strategy to determine the degree to which students change from the beginning to the end of a program. Astin (1985) referred to this type of change as talent development.
historical context
Historical Context
  • 80s – Greater call for accountability for higher education regarding student learning; value-added was considerable part of discussion.
  • A few states embraced value-added (e.g., Missouri and Tennessee)
  • 90s early 00s – De-emphasis on value-added, more emphasis on minimum competency
  • Mid 00s – Value-added once again at center of general education discussions
current context national
Current Context (National)
  • Catalyst?
  • Spellings Commission
  • Recommendations (U.S. Department of Education, 2006) :
    • “Higher education institutions should measure student learning using quality assessment data.”
    • “The results of student learning assessments, including value-added measurements that indicate how much students’ skills have improved over time, should be made available to students and reported in the aggregate publicly.”
current context national6
Current Context (National)
  • Spellings Commission is considering logistical issues.
  • NASULGC and AACSU are working on developing recommendations for a “Voluntary Accountability System.”
  • The College Learning Assessment (CLA) is recommended by many groups to assess student gains in higher-order learning.
current context virginia
Current Context (Virginia)
  • Core Competency Assessment – State Council advocating value-added assessment for student learning.
  • Assessment of Student Learning Task Force charged by State Council to work out logistics. Initially, all members assessment experts.
  • This task force now is composed of presidents, provosts, and three assessment experts.
slide8
Whoa!
  • Several issues here; value-added is just one of those issues.
  • Let’s examine just value-added.
approaches to value added
Approaches to Value-Added
  • Cross-Sectional
  • Pre-Post
  • Residual Analysis
cross sectional
Cross-Sectional
  • Cross-Sectional: Compare scores from (say) sample of seniors against sample of freshmen on same test.
  • Note: Weakest of three designs; doesn’t control for differences between samples.
pre post
Pre-Post
  • Pre-Post: Same set of students takes same test or equivalent tests at two points in time (i.e., repeated measures).
  • Change scores represent value-added.
  • Note: Conceptually most straight-forward approach to value-added. Takes considerable time to collect difference scores.
residual analysis
Residual Analysis
  • Residual Analysis: Determined by comparing the difference between actual scores and the scores predicted by some variable (or a set of variables), usually SATs or ACTs - Approach used by CLA
  • Note: Logistically easier to implement but conceptually a bit off-target. Value-added from this approach is normative.
misconceptions of value added
Misconceptions of Value-Added
  • Misconception: Measuring change is always unreliable
  • In the context of pre-post design, five conditions must hold for change scores to be unreliable (Zumbo, 1999):
    • the correlation between testing occasion one and testing occasion two is a large positive value
    • the observed variance of the two testing occasions is equal
    • the true score variance at both occasions is equal
    • the reliability at both occasions is equal, and
    • the correlation between true scores and true change scores is negative.
other misconceptions
Other Misconceptions
  • Value-Added means standardized assessment across institutions.
  • Value-Added is the CLA instrument.
    • Any instrument could theoretically be used for value-added assessment.
reservations of value added
Reservations of Value-Added
  • Affected by attrition
  • Expensive (especially for pre-post)
  • Item memorization
  • Doesn’t answer all analytical questions
value added summary
Value-Added, Summary
  • Some limitations
  • Answers a great question
  • In terms of the national debate about standardized testing across the nation…
  • We should be clear. Are we actually criticizing value-added, or an instrument, or standardization across schools….???
references
References
  • Astin A. W. (1985). Achieving Educational Excellence: A Critical Assessment of Priorities and Practices in Higher Education. San Francisco: Jossey Bass.
  • U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, D.C. Also available at http://www.ed.gov/about/bdscomm/list/hiedfuture/reports/pre-pub-report.pdf
  • Zumbo, B. D. (1999). The simple difference score as an inherently poor measure of change: Some reality, much mythology. In Bruce Thompson (Ed.). Advances in Social Science Methodology, Volume 5, (pp. 269-304). Greenwich, CT: JAI Press.
ad