1 / 46

Integrated Document Analysis Workshop

Integrated Document Analysis Workshop. Washington, DC May 5-7, 2008. Describing the Content & Language Demands of. Standards, Assessments & Student Work. Peter Worth , Assessment Standards & Development Services, WestEd, CA

alaire
Download Presentation

Integrated Document Analysis Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Integrated Document Analysis Workshop Washington, DC May 5-7, 2008 Describing the Content & Language Demands of Standards, Assessments & Student Work Peter Worth, Assessment Standards & Development Services, WestEd, CA John L. Smithson, Director, Measures of the Enacted Curriculum Wisconsin Center for Education Research, University of Wisconsin-Madison

  2. A neutral linguistic taxonomy A neutral content language Topics by Cognitive Demand Linquistic Complexity by Language Demands Analyzing Alignment The intended curriculum: State content standards—What students should learn The assessed curriculum: State (and other) assessments—tested learning The learned curriculum: Student outcomes based on school learning The enacted curriculum: What teachers teach

  3. Content and Language Coding Form

  4. Content and Language Coding Grade 8 mathematics item

  5. Content and Language Coding Form

  6. K-12 Mathematics Content

  7. K-12 Mathematics Content Basic Algebra

  8. The Goal To render quantitative descriptions of instruction, standards, and assessments using a common language in order to facilitate comparisons and analyses of the three domains of a standards-based approach to education reform and their relationship to one another.

  9. To Facilitate Comparisons Academic Content ELL Lang. Acquisition Assessments Academic Content ELL Lang. Acquisition Curriculum Alignment Academic Content ELL Lang. Acquisition Standards

  10. The Two Dimensions Of Content What students should know [Topics] And… Be Able to Do [Expectations for student performance]

  11. Describing Instructional Content SEC utilizes a two-dimensional taxonomy based on: Topic by Cognitive Demand

  12. The Content Matrix

  13. The Content Matrix

  14. … adding levels of relative emphasis yields a 3-D construct

  15. Describing the Cognitive Domain How Many Categories? 3 SCASS Science 4 DOK (Webb) SEC 5 6 Bloom’s

  16. Dimensions of Knowing & Inquiry Acquire Use Extend (From: Dimensions of Knowing and Inquiring about Science, State Collaborative on Assessments & Student Standards Science Project, Council of Chief State School Officers, 1997)

  17. Depth of Knowledge Level 1 2 3 4 Recall Skill/Concept Strategic Thinking Extended Thinking * Webb, N. 1999. Alignment of Science and Mathematics Standards in Four State. NISE Research Monograph #18. Madison:Wisconsin Center for Education Research.

  18. Exploring Cognitive Demand Acquire Use Extend Recall Skill/Concept Strategic Thinking Extended Thinking

  19. Cognitive Demand (or Expectations for Student Performance) Acquire Use Extend Recall Skill/Concept Strategic Thinking Extended Thinking Memorize Perform Procedures Demonstrate Understanding Conjecture, Generalize Prove Solve non- routine/ make connections Memorize Conduct Investigations Communicate Understanding Analyze Information Apply concepts /make connections Recall Perform Procedures Generate /Demonstrate Analyze/ Investigate Evaluate

  20. Cognitive Demand (or Expectations for Student Performance) Acquire Use Extend Recall Skill/Concept Strategic Thinking Extended Thinking Memorize Perform Procedures Demonstrate Understanding Analyze Information Evaluate/Apply Recall Understanding Application Analyze Evaluate Create

  21. Practice Session #1 Exploring Cognitive Demand But first…. Let’s take a 10 minute break!

  22. Exploring Cognitive Demand CgD Immersion Activity • Organize into Groups/Tables • Each Table w/ CgD Pie • Each Person w/ Cgd Descriptors Step 1: Place CgD cards on Pie Slices face-down Step 2: Turn cards over: ID agreements e.g. 2 cards w/ same descriptor in same slice if Group Agrees ... discuss key words if not … Discuss … operational definition to distinguish Step 3: Discuss disagreements if consensus reached put in envelope / if not, set aside

  23. Content Analysis Materials • Cognitive Demand List • Topics Lists • Comments & Suggestions Worksheet • Coding Forms • Documents to be analyzed

  24. Cognitive Demand Lists • Five categories of cognitive demand • Slightly different for each subject • Each category is defined by a list of descriptors • The list of descriptors are not exhaustive • Each category stands on its own • Each category has an associated letter (B-F)

  25. Dimension A: Content Topics Topics List (In your packet of material) Organized at two levels: Content Areas (16 for Mathematics) (27 for Science) (14 for ELAR) Topics (identified by number) (182 Mathematics Topics) (211 Science Topics) (114 ELAR Topics) Plus: non-specific & other

  26. Topics Lists • Topics Lists • Mathematics • Science • English Language Arts & Reading • Cover grades K-12 • Organized into Content Areas • Topics & Content Areas have an associated #

  27. Comments & Suggestion Worksheet • One for each reviewer - more available • Use to: • Record coding conventions/decision rules • Suggest/identify additional topics not listed • Suggest/identify additional CGD descriptors • Provide other comments & suggestions • Be sure to turn in at end of workshop (and with mail-in materials, as necessary).

  28. The Content Analysis Process ‘Coding’ Teams of 4-5 Content Experts Independent Coding by each Analyst w/ Group Discussion Should not be necessary to discuss every item – select by team Goal for Process: Generalizability not Inter-rater Reliability Pick-up and return documents / coding sheets to Alissa Sign & return to Alissa non-disclosure forms

  29. Practice Exercise #1 (Math)

  30. Practice Exercise #2 (ELAR)

  31. A neutral content language Topics by Cognitive Demand Content Analysis Workshop The intended curriculum: State content standards—What students should learn The assessed curriculum: State (and other) assessments—tested learning The learned curriculum: Student outcomes based on school learning The enacted curriculum: What teachers teach

  32. Coding Forms • Assessment Coding Forms • Standards/Benchmark Coding Forms • Each is used to record content descriptions • Each content description consists of • A topic number • A cognitive demand category letter • The Integrated Analysis Coding Form • Combines Content & Linguistic Information in a single data collection form

  33. Assessment Coding Forms

  34. Standards Coding Forms

  35. Standards Coding Forms

  36. Gr. 5-8 National Science Standards

More Related