Presented by: Lori DeForest firstname.lastname@example.org (315)433-2247 The DATA WISE Process and Data-Driven Dialogue
“Using data effectively does not mean getting good at crunching numbers. It means getting good at working together to gain insights from student-assessment results and to use the insights to improve instruction.”- Kathryn Boudett, Elizabeth City, & Richard Murnane, “When 19 Heads Are Better Than One,” Education Week, December 7, 2005.
Organize for Collaborative Work Prepare • Build Data Teams • Establish team structure to allow for data discussions • Establish norms • Utilize protocols • Complete a Data Inventory
Data Teams Prepare • Raise important questions about student learning and achievement • Assist in organizational aspects of data • Dialogue about multiple data sources • Examine and interpret data • Investigate ways to improve teaching and learning
Curriculum CouncilsInitial Data Teams Prepare October and November - Setting Norms Protocol - Compass Points Protocol
Setting Norms Protocol Prepare What norms do we need? • Brainstorm • Discuss • Synthesize • Build consensus
We need to build emotional safety to reach cognitive complexity.B. Wellman and L. Lipton
Curriculum CouncilsInitial Data Teams October and November - Setting Norms Protocol - Compass Points Protocol December and January - Data Inventory - Data Analysis Tools
Purpose of Data Inventory Prepare • Summarizes all the types of data that are available and helps to determine what other data is needed • Builds assessment literacy • Assists in the planning of using data effectively • Begins conversation about educational questions
Race/Ethnicity • English Proficiency • Attendance Data Inventory Prepare
Data Inventory Prepare
Assessment Literacy Prepare scaled score norm-referenced cohort reliability validity cut score performance levels measurement error raw score grade equivalents sampling standards-referenced percentile rank criterion-referenced
Data Analysis Tools • COGNOS Report Net • COGNOS Power Play Cubes • Data Mentor • nySTART • NYS State Report Card Databases and ELA and Math Media Databases • Student Management System Demonstrate tools at Curriculum Councils, Grade Level and Department Meetings Offer training opportunities
Inquire Data Overview • Determine audience • Decide on educational questions • Create graphic displays of standardized test results • Engage in conversations around initial data set
2006 English Language Arts Performance Data Sources: CNYRIC COGNOS PowerPlay Cubes, NYSED School Report Card and ELA Assessment Databases
2006 Mathematics Performance Data Sources: CNYRIC COGNOS PowerPlay Cubes, NYSED School Report Card and Math Assessment Databases
“Without an investigation of the data, schools risk misdiagnosing the problem.”Data Wise, 2005
Inquire Data Analysis Protocol • Activate and Engage • Set norms • Articulate predictions and assumptions • Explore and Discover • Begin with a single data source • First describe what you see • Ask questions • Identify additional data needs
Data Analysis • Writers Club
Informing Planning for Writers Club Use COGNOS PowerPlay to identify needs of struggling students.
A distracter analysis may help you understand children’s incorrect thought processes.
Data Analysis • Writers Club • Physical Education program • Intervention Analysis
Intervention Analysis 2006 Cohort ELA 8 Performance of students who performed at Level 1 or 2 on ELA 4 and remained in district (n=43) Tracking cohort performance may give you some information about program or student growth.
Data Analysis • Writers Club • Physical Education program • Intervention Analysis • English Language Arts Analyses
English Language Arts Educational question: Is student performance declining in reading comprehension but increasing in listening comprehension? • Utilize COGNOS PowerPlay item analysis and Scoring Key to classify questions by subtest • Compile data using formulas and functions • Study trends over time • Use comparative data to inform analysis
The graph above illustrates the importance of using comparative data to inform analysis.
English Language Arts Analyses COGNOS Report Net now houses analysis reports which can provide you information about student performance.
Sampling Principle “Because a test is not a direct measure of a student’s degree of mastery of an entire domain, any conclusion you reach about proficiency in that domain is based on an inference from proficiency on the smaller sample. …Even a test that provides good support for one inference may provide weak support for another.” Data Wise, 2005
Data Analysis • Writers Club • Physical Education program • Intervention Analysis • English Language Arts Analyses • Math Analyses
Data Overview of NYS Assessment Performance • Considerations when reviewing summary data • Different cohort groups • Different samples of items each year • Different test blueprints • Importance of comparison data sets
Go to www.emsc.nysed.gov/osa and the Report Card link to access data for similar schools.
Enrollment in College Level Math Courses Note: Dual enrollments taken into account for total annual percentage Utilize your own student management system to analyze additional data as well.