1 / 23

Madhabi Chatterji, Ph.D.

An Assessment Model for Development of Student Talent in Higher and Vocational Education. Madhabi Chatterji, Ph.D. Associate Professor of Measurement and Evaluation, Director, Assessment and Evaluation Research Initiative (AERI) Teachers College, Columbia University

teo
Download Presentation

Madhabi Chatterji, Ph.D.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Assessment Model for Development of Student Talent in Higher and Vocational Education Madhabi Chatterji, Ph.D. Associate Professor of Measurement and Evaluation, Director, Assessment and Evaluation Research Initiative (AERI) Teachers College, Columbia University E-mail: mb1434@columbia.edu Forum at Ningbo Polytechnic Institute, China June 15th, 2011

  2. Purpose To Present: • A Diagnostic Classroom Assessment Model Designed to Nurture Student Talent and Expertise in Specific Curricular Domains Drawing on: • Research in the Cognitive Sciences with Implications for : -Designing Classroom Instruction -Designing and Using Student Assessment -Skill Acquisition and Talent Development in Learners [Based on a sampling of the literature from 1970s-present]

  3. Purpose (continued…) With a Brief Application Example: Results of empirical study showing how the assessment model worked with elementary school teachers and students in New York, using mathematics domains (i.e., Long Division in Arithmetic) To Stimulate Discussion on: Applications in higher and vocational education settings? Applications in international education settings, e.g., in China and other nations?

  4. Background terminology … What is the field of “Cognitive Science”? Researchers and theorists from different fields, such as psychology, educational psychology, computer science, linguistics, anthropology and neuroscience, who have used experimental and other research methods to understand the workings of the human mind and how “cognitions” ( what we know) develop. Implications for Educational Assessment? This emerging body of knowledge is changing how we think about designing and using educational assessments—both in the classroom and on a large scale

  5. Background terminology… What is “Educational Measurement”, “Assessment”, or ‘Testing”? These terms are used inter-changeably to denote formal or informal processes that educators use to score, rate, classify or numerically code student responses to structuredquestions,tasks, or items. The score or rating is then used as evidence to make inferences about the students’ knowledge, skills or abilities in a domain. “Test”: Traditional measuring instrument, made up of paper and pencil items, either multiple choice or structured response formats, that can be scored objectively. “Assessment”: A variety of measuring instruments, including constructed-response written tasks, behavior-based assessments, interview-based assessments, portfolio-based assessments, all requiring more subjective appraisals by observers or examiners.

  6. Background terminology … Why Do We Assess Students? I. For Making Summative Decisions, such as: • In the classroom, for assigning student Marks (A-F) or Pass/Fail “grades” at the end of a unit, semester or year • At the institutional level, for certifying competence in individual workers/professionals, admitting individuals to special programs, selecting or recognizing the best applicants for an award • At the classroom and institutional level, meeting external accountability and accreditation requirements for “quality”, and for evaluating whether institutions, classrooms, programs and services are performing as intended

  7. Background terminology … Why Do We Assess Students? II. For Making Formative Decisions • Continuously improving the quality of delivery of instruction, programs, services • Helping curriculum developers, instructors, and staff plan, set goals, and make better design/implementation decisions • Diagnosing learner needs and gaps in targeted domains • Fostering learner development in targeted domains, by changing how we teach • Mapping growth and development for groups of learners in targeted domains Assessment for Talent Development Focuses on the Last Three!

  8. Cognitive Sciences Research Which Principles are Useful in Devising a Model of Classroom Assessment for Talent Development in Students? Implications for Assessment, Teaching, and Learning

  9. Principles from Research-Implications for Assessment Cognitive Modifiability and Continuous Human Development • Ability or talent in a domain is NOT a fixed trait! Human capacities-- intellectual, social, and emotional behaviors --are developmental and can be changedcontinuously • Development in domains of knowledge, skills, competencies is progressive. • Proximal and dynamic assessment helps remove cognitive blocks IMPLICATIONS: Domains and tasks must be developmentally ordered when designing educational assessments. DYNAMIC MODEL: Test—Identify Errors---Re-teach-Mediate—Test Again (Repeat) (Feuerstein, Rand, & Hoffman, 1979; Goleman, 1994; 2007; Kuhn, Katz, & Dean, 2004; Pressley, 1995; Siegler, 2000; Vygotsky, 1978; Wellman & Gelman; 1992).

  10. Principles from Research-Implications for Assessment Continuous Expertise Development in a Domain involves Error Diagnosis, Clear Goals and Formative Instruction IMPLICATIONS • Assessment tasks or items must be designed reveal student errors, misunderstandings, and mistakesin the domain • Scoring rubrics must show strengths and weaknesses of learners in the domains of knowledge, skills, competencies • The results must be usable by teachers and learners to shape continuous student progress towards clear goals and culminating tasks ( via targeted feedback, coaching, practice) • The assessment culture must be learning centered and supportive –NOT punishing. (Anderson, 2005; Black & Wiliam, 1998; Dick, Carey, & Carey, 2005 ; Nichols, 1994; Tatsuoka, 1983; Webb, Franke, De, Chan, Freund, Shein, & Melkonian, 2009)

  11. Principles from Research-Implications for Teaching Expertise Develops when Past and New Learning are Effectively Connected and Meaningfully Situated in Real-life Tasks IMPLICATIONS • Tap into prior knowledge of learners that is relevant to learning something new on a developmental continuum • Scaffold instruction to build mental bridges • Facilitate transfer of learning and expertise development in a new skill, by showing similarities in old and new domains • Situate learning in real-life applications to facilitate connections (Anderson, 2005; Pellegrino, Chudowsky, & Glaser, 2001).

  12. Principles from Research-Implications for Learners Changes in Skill and Expertise Levels Occur with Self-Monitoring and Deliberate Practice by Leraners Themselves • Metacognitive skills in learners, or the capacity to track their own learning levels and misunderstandings, enhances domain-specific expertise • Talent development research in medicine, sports, and music shows that people who excel, dedicate hours of self-motivated practice in a given area after their coaches/mentors point out their weaknesses. (Anderson, 2005; Ericsson, 2004;Pellegrino, Chudowsky, & Glaser, 2001).

  13. A Diagnostic Classroom Assessment Model The Proximal Assessment for Learner Diagnosis (PALD) Model was Designed to Nurture Student Talent/Expertise in Scholastic Domains Application and Results of an Early Empirical Study (Chatterji et al, 2008; Chatterji et al, 2009; Chatterji, in press)

  14. I. Goal-Setting Specify:Culminating performance tasks and domain of embedded concepts and skills Specify:Long-term instructional goals R E V I S E II. PALD Planning • Develop lesson plans linked to domain • Design instructional strategies • Develop ordered diagnostic assessment tasks IV. End of Unit Assessment R E V I S E Make summative decisions on student progress III. PALD Implementation Instruct (New concepts) Embed PALD Cycles to Detect Learner Gaps Assess Analyze Errors/-Diagnoses Mediate Give Practice Make formative decisions The Proximal Assessment for Learner Diagnosis (PALD) Model: How Teachers are Expected to Plan and Work

  15. Definition and Examples … What is a “Domain” in the PALD Model? A defined body of related skills, concept knowledge, higher order capacities or habits/attitudes that enables people to perform particular classes of applied tasks successfully, within a larger subject area or discipline. Two Examples in Vocational Education Competency Domain of English Language: The range of reading, writing, listening comprehension, and speaking skills in English that enable a student to serve as an effective tour guide (Tourism Industry) Competency Domain in Health Information Technology: The complete range of computer literacy, programming, and health science/policy knowledge, skills, and habits needed by a worker in the health care system to meaningfully use information on patients’ electronic health records to improve care (Health Services Sector)

  16. Long Division Domain (and Scoring Rubric) • I CAN: • (1)Say in my own words what the long division problem is asking me to do • (2) Read a story problem and set up the long division and/or other operations needed to solve it • (3)Identify the “dividend” and “divisor” in an long division problem • (4) After the long division problem is set up, know how to start the algorithm (find digit with the highest place value in the dividend) • (5) Recall and use multiplication facts correctly when doing long multiplication and long division • (6) Recall place value concepts and apply place value for digits in whole and decimal numbers, when doing long multiplication and long division • (7) Follow the steps of the long division algorithm correctly • (8) Repeat the long division algorithm until I get a remainder that cannot be further divided (grade 5) • OR • (9) Continue the long division algorithm so I can express remainder as a decimal of the quotient (grade 6) • (10) Check for mistakes in the long division answer with “backwards” operations • (11) Explain what the answer and remainder means in my own words • (12) Think-aloud and write the steps of long division • (13) Think aloud and explain the answer and parts of the answer in a story problem • (14) Keep my scratch work neat (so that I don’t get lost) • (15) Look at my mistakes to help me grow (without fear) • (16) Solve a real-life problem using long division and other operations • (17) Make connections between fractions, decimals and division/ long division operations

  17. Development Continuum in Long Division Domain EASY MORE DIFFICULT MOST COMPLEX ____________ / ____________/ _________________/__ Multiplication Skills Skills Skill Facts 1-3 10-11 16 (Prerequisite) (Culminating Task)

  18. Diagnostic Item (Easy)-Long Division Domain (contd.) • Part 1 • Look at the problem below. 1. The problem is telling me to: (a) Multiply 5 times 175 (b) Find out how many groups of 5 are in 175 (c ) Divide 5 into 175 equal parts (d) Add 5 and 175 (e) Do something else (explain):

  19. Diagnostic Item (Difficult)-Long Division Domain (contd.) • Part 2 I began to do the following problem but know I have made some mistakes. Please find and mark my mistakes. Then, show me how I could fix them by re-doing the problem. • HTO • 610 • _____ • 9) 552 • -54 • 12 • 9 • 3 • -0 • 3R

  20. Most Difficult Item and Target Goal in Grade 5-Long Division Domain • Culminating Assessment Task in Grade 5: Coconut cookies come in packets of 12. I would like to give 1 cookie to each 5th grader at (your school) to celebrate the last day of school (in June). There are 219 5th graders at (school name) Elementary. How many packets of cookies should I buy so that every 5th grader gets at least 1 cookie? Will there be any cookies left over? Show your plan to solve the problem. Then set up the numbers and solve it. Explain all the parts of your answer, and show how you would check if your answer is correct. Show all your work.

  21. Our “Formative” Study on How the PALD Model Works Design: The mixed-methods study, supported by the National Science Foundation, involved a quasi-experiment supported with classroom observations and teacher surveys (40 teachers, 4 schools, about 700 students in grades 5-6 ). Results: Following teacher training and classroom intervention with the PALD approach: • There were no perceptible effects after one year. Growth was uneven in individual teachers and students during the first field trial of the PALD model • BUT, after two years, PALD-trained teachers were using significantly more PALD practices in the classroom during mathematics instruction than Comparison Group teachers (there was more frequent probing, error analysis, mediation, feedback) • Students with two years of exposure showed statistically significant gains (Effect Sizes of .10-.48) on mathematics examinations, conducted internally and externally • Research is continuing with refinements to the PALD model…..

  22. References American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Anderson, J.R. (2005). Cognitive psychology and its implications (Sixth Edition). New York, NY: Worth Publishers. Chatterji, M. (2003). Designing and using tools for educational assessment. Boston, MA: Pearson/Allyn & Bacon *Chatterji, M., Koh, N., Choi, L., & Iyengar, R. (2009). Closing learning gaps proximally with teacher-mediated diagnostic classroom assessment. Research in the Schools, 16(2), 59-75. *Chatterji, M. (in press). Development and validation of indicators of teacher proficiency in diagnostic classroom assessment: A mixed methods study. The International Journal of Educational and Psychological Assessment. Pellegrino, J. W., Chudowsky, N., Glaser, R., & (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, D.C.: National Academies Press. * References not listed but cited are available in these two papers.

  23. Questions and Discussion Applications in: Higher Education ? Vocational Education? China and other nations?

More Related