1 / 20

The Role of Information in Systems for Learning

The Role of Information in Systems for Learning. Paul Nichols Charles DePascale The Center for Assessment. Problem. Recognition that improved student learning requires coordinated assessment, instruction, curricula and professional development

Download Presentation

The Role of Information in Systems for Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The Role of Information in Systems for Learning Paul Nichols Charles DePascale The Center for Assessment

  2. Problem • Recognition that improved student learning requires coordinated assessment, instruction, curricula and professional development • States and districts are being asked to conceptualize and implement assessments as a component of broad and coherent systems • For example, comprehensive assessment systems or systems for learning. • New burdens on district and state staff

  3. Problem • Think beyond a single test or set of tests to coordinated system of assessment, curriculum, pedagogy and professional development • Manage and held accountable for the award, development and implementation of multi-component systems • Cross department boundaries, engage multiple-vendors and involve many times the cost of a testing program

  4. Status Quo • State and district staff already have or can find the following training and tools to help with test development and implementation for individual tests • Currently no theoretical framework and no indices to help state, district and other educational leaders in planning, developing and evaluating systems for learning

  5. Systems Approach • Move beyond hand waving to offer a framework and quantitative tools used by district and state staff to conceptualize and implement systems for learning (SFL) • Note the components that probably should be included in an SFL • Address SFL coherence by looking at communication of information from students to teachers • Present analytic framework allowing district and state staff to predict SFL coherence across different configurations

  6. Defining Systems for Learning • What is a system? From the field of systems thinking: A system is “a collection of parts which interact with each other to function as a whole” • An SFL is a set of components which interact with each other to function as a whole with the intention of improving student learning

  7. Defining Systems for Learning • Paul Black and colleagues describe the components of an SFL:

  8. Defining Systems for Learning • Curriculum consists of the knowledge and skills in subject areas that teachers teach and students learn. • Instruction refers to methods of teaching and the learning activities used to help students master the content and objectives specified by a curriculum. • Assessment is the means used to measure the outcomes of education and the achievement of students with regard to important competencies. • Theory of learning is the model of student cognition and learning in the domain

  9. Defining Systems for Learning • Karin Hess reminds us that professional development is an important missing component:

  10. Defining Systems for Learning • Coherence is provided by the theory of learning • The components must “talk to each other” in a common language • The language of communication is the learning theory

  11. Measuring SFL Coherence • An SFL may be conceptualized as a communication system • Think of a radio or your iPad receiving a transmission • A generic communication system

  12. Measuring SFL Coherence • Signal is generated by an information source • Transmitter codes the signal and passes the coded signal to the receiver • Receiver decodes the signal and passes the reconstructed signal to the destination • Noise is likely to degrade the information in the signal as it is passed from component to component • Information is contaminated by noise so that information received is different from an error-free message

  13. Measuring SFL Coherence • Teacher attempting to understand the information relevant to the construct in student performance

  14. Measuring SFL Coherence • From Measurement Theory to Information Theory • Information is defined as the amount of uncertainty reduced • Student whose answer is right or wrong vs student whose answer reveals stage of learning progression • Bit is measure of amount of information in terms of choice or uncertainty • Learning outcome between two equally probable alternatives is learning one bit of information

  15. Measuring SFL Coherence • Coherence can be defined as the degree to which different components help communication • H(x|y), equivocation, is uncertainty that x was transmitted when y is received • H(x|y) = • Mapping from x to y reduces noise or error the value approaches 0 • Lower values = higher coherence

  16. Measuring SFL Coherence • Components hinder/facilitate communication so decrease/increase coherence • Performance-based assessment where the teacher is looking for formative information • Scaffolding reading or constructed response on the circulatory system • Learning progression with 5 levels • Focus on professional development & instruction

  17. Measuring SFL Coherence • Simulation 1: Professional development addresses basic statistics, reliability, validity • Fails to address learning progressions • Teachers’ scoring shows no relationship to students’ learning progression stage • Students receive instruction unrelated to prior understanding • Coherence low, H(x|y) = 2.32

  18. Measuring SFL Coherence • Simulation 2: Professional development addresses the concept of learning progressions and describes stages in the learning progression • Teachers’ scoring shows large “halo effect” and noise • Students with impoverished understanding fail to receive instruction • Coherence moderate, H(x|y) = 1.49

  19. Measuring SFL Coherence • Simulation 3: Professional development addresses concept of learning progressions, describes stages and in the learning progression, calibrates scoring with exemplars • Teachers’ scoring shows some “halo effect” and impact of noise • Most students receive appropriate instruction • H(x|y) = 0.83

  20. Conclusion • State and district staff are asked to conceptualize and implement SFLs • State and district staff need a theoretical framework and quantitative indices to help plan and evaluate SFLs • Measurement theory focuses on individual tests or groups of tests • Instructional theory focuses on instructional techniques and activities • The framework and index of system coherence are a start…

More Related