1 / 47

Data Analysis

Data Analysis. Based on the work of Vicki Bernhardt ITF• January 27, 2009. Data can help to. Replace hunches with facts concerning changes Facilitate clear understanding of gaps between where school is and where it needs to be, and identify root causes-not treat symptoms

kaveri
Download Presentation

Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Analysis Based on the work of Vicki Bernhardt ITF• January 27, 2009

  2. Data can help to... • Replace hunches with facts concerning changes • Facilitate clear understanding of gaps between where school is and where it needs to be, and identify root causes-not treat symptoms • Provide information to eliminate ineffective practices • Ensure efficient use of $$$ • School if school goals and objectives are being accomplished • Ascertain if school staffs are implementing their visions • Generate answers and effectively educate the community • Predict and prevent failures • Predict and ensure successes • Improve instruction • Provide students with feedback on their performance.

  3. Data can also help to... • Gain understanding of what quality is and how close we are to achieving it • Make sure students are not “falling through the cracks • Get the “root causes” of the problems • Guide curriculum development and revision • Meet state and federal requirements • Promote accountability

  4. Barriers to using data include... • In contrast to business, work culture in education usually doesn’t focus on data • Few are trained and few see it as part of their jobs • Schools don’t have data warehousing solutions that make it easier • Teachers are generally trained to be subject oriented, not data oriented • There’s a perception that data is gathered for other people not mine • The legislature keeps on changing the rules! (e.g. NHEIAP, NECAP)

  5. A better way... • Data analysis should be • Systematic, Systemic • All of the the processes and procedures that contribute to learning-the whole and the interrelationships of the parts of the whole to each other • Continuous • Measuring and evaluating processes on an ongoing basis to identify and implement improvement • “Upstream process improvement not downstream damage control” Teams and Tools (1991)

  6. Using Multiple Measures of Data • One measure gives useful information, but… • Comprehensive, multiple measures give much richer info.

  7. Multiple Measures of Data • Demographics (enrollment, drop out rate, gender, ethnicity) • Perceptions (values/beliefs, attitudes) • Student Learning (standardized tests, authentic assessments, teacher observations) • School Processes (how we teach children to read, our Math scope and sequence through the years)

  8. Various levels of data analysis: Each of the four measures, by themselves, give valuable information. But the deeper we dig and the more levels we utilize, the more effective the resulting information.

  9. Level 1: Snapshots • How many students are enrolled at MSS this year? (Demographics) • How satisfied are parents, students, and/or staff with the learning environment at HMS? (Perceptions) • How did students at the Hopkinton High School score on the SATs this year? (Student Learning) • What new courses are being offered by the high school this year? (School Processes)

  10. Level 2: Measures Over Time • How has enrollment changed throughout the HSD over time? (Demographics) • How have student perceptions of the A.P. program changed over time? (Perceptions) • Are there differences in boy’s 6th grade math scores in the NHEIAP over time? (Student Learning) • What programs have been consistent at MSS over the last five years? (School Processes)

  11. Level 3: Two or More Variables Within Measures Looking at more than one type of data within each of the circles.. • What percentage of students in the HSD are fluent speakers of languages other than English and are there equal numbers of males and females? (Demographics) • Are staff, students, and parent perceptions of the culture within HMS in agreement? (Perceptions) • Are student’s scores on the 6th grade NHEIAP consistent with K-6 assigned grades and performance assessment rubrics? (Student Learning) • What are the major instructional strategies used by the Math and Science departments at the high school? (School Processes)

  12. Level 4: Two or More Variables Within One Type of Measure, Over Time • Similar to Level 3…just does the analysis over time… • For example: How has the enrollment of non-English speaking children changed in the last three years? (Demographic)

  13. Level 5: Intersection of Two Types of Measures • Do students who have perfect attendance perform better on the NHEIAP than students who miss more than five days per month? (Demographics by Student Learning) • Is there a gender difference in students’ perceptions of The Senior Project? (Perceptions by Demographics) • Do students engaged in classrooms utilizing differentiated instruction perform better on the NHEIAP than in those classrooms that do not employ this strategy? (Student Learning by School Processes)

  14. Level 6: Intersection of Two Measures, Over Time Same as Level 6, just over time, for example… • How have students of different ethnicities scored on the SAT over the past three years? (Demographics by Student Learning)

  15. Level 7: Intersection of Three Measures • Is there a difference in students’ reports of what they like most about MSS by whether or not they participate in extracurricular activities? Do these students have higher grades than those who do not participate in extracurricular activities? (Perceptions by Student Learning by School Processes) • What program at the high school is making the biggest difference with respect to student achievement for at-risk students this year and is one group of students responding better to the processes? (School Processes by Student Learning by Demographics)

  16. Level 8: Intersection of Three Measures, Over Time Same as Level 7, but over time, for example… • What programs throughout the district do all types of students like the most every year? (Demographics by Perceptions by School Processes)

  17. Level 9: Intersection of All Four Measures This is the ultimate analysis. It allows us to answer questions such as: • Are there differences in NWEA scores for 8th grade girls and boys who report they like school, by the type of program in which they are enrolled? (Demographics by Perceptions by School Processes by Student Learning)

  18. Level 10: Intersection of All Four Measures, Over Time Same as Level 9, but over time, for example… • Based on whom we have as students, how they prefer to learn, and what programs they are in, are all students learning at the same rate? (Student Learning by Demographics by Perceptions by School Processes)

  19. Each measure in a bit more detail... Demographics, Perceptions, Student Learning, and School Processes

  20. DEMOGRAPHICS

  21. Examples of Demographics • Community • Location, history, population trends… • School District • Description and history, number of schools… • School • Grants and awards received, class sizes, facilities

  22. More examples... • Students Over Time, and by Grade Level • Attendance, gender, race… • Staff Over Time • Years of experience, ethnicity, certifications… • Parents • Educational levels, involvement with child’s learning...

  23. Disaggregation The separation of data into subgroups. Why is it important? • Disaggregation shows if there are differences in subgroups-there should be few differences. • Helps us find subgroups that are not responding to our techniques.

  24. PERCEPTIONS

  25. What are perceptions? • A generally held view. • A belief stronger than impression and less strong than positive knowledge.

  26. Whose perceptions? • Teachers • Parents • Students • Administrators • Community members

  27. How do we change perceptions? • Give parents a chance to experience the approach, e.g. parent night, open house. • Cognitive dissonance; the discomfort one feels when two thoughts, opinions, or ideas are inconsistent.

  28. Assessing Perceptions • Questionnaires • Interviews • Focus Groups (All three were done as part of the SPEDMIP report)

  29. Questionnaires They must be: • Valid • Reliable • Understandable • Quick to complete

  30. Types of Questionnaires • Interview (face to face) • Telephone Interview (person to person) • Mailed • Paper • Online (recommended, e.g. SurveyMonkey.com

  31. STUDENT LEARNING

  32. Ways to measure student learning • Standardized tests • Norm-referenced, criterion referenced • Authentic assessments • Teacher-made test • Teacher-assigned grades • Performance assessments • Standards based assessments • Diagnostic testing

  33. Norm-referenced Tests • Compares test performance of a school, group, or individual with the performance of a norming group (e.g. CAT, Iowa)

  34. Criterion-Referenced tests • Compares individuals performance to a set of standards and not to the performance of other test takers. (e.g. NHEIAP, NECAP)

  35. Diagnostic Tests • Tests given to students to know the nature of a student’s difficulty but not necessary the cause of that difficulty.(e.g. Sped testing)

  36. Performance Assessments • Measures skill and knowledge directly, e.g. if you want students to learn to write, assess it on a writing assignment • Advantages: measures process, can match state standards, is a learning tool in and of itself, provides reflection for students...

  37. What does UbD have to say about Assessment?

  38. 1. Identify desired results 2. Determine acceptable evidence 3. Plan learning experiences & instruction (from UbD PPT) 3 Stages of Design: Stage 2

  39. The big ideafor Stage 2 • The evidence should be credible & helpful. • Implications: the assessments should – • Be grounded in real-world applications, supplemented as needed by more traditional school evidence • Provide useful feedback to the learner, be transparent, and minimize secrecy • Be valid, reliable - aligned with the desired results of Stage 1 (and fair) (from UbD PPT)

  40. Reliability: Snapshot vs. Photo Album • We need patterns that overcome inherent measurement error • Sound assessment (particularly of State Standards) requires multiple evidence over time - a photo album vs. a single snapshot (from UbD PPT)

  41. For Reliability & Sufficiency:Use a Variety of Assessments • Varied types, over time: • authentic tasks and projects • academic exam questions, prompts, and problems • quizzes and test items • informal checks for understanding • student self-assessments (from UbD PPT)

  42. SCHOOL PROCESSES

  43. What are School Processes? • Programs • Reading Recovery, Write Traits, UBD • Practices • HBA, Mini-Mentors, Student Councils • Instructional strategies • Differentiated Instruction, Cooperative Learning

  44. School Level Processes • What teachers and administrators do to achieve the vision of the school • What do we want students to be able to know and do? • How are we enabling students to learn in terms of • Instructional strategies, learning strategies, instructional time, student-teacher ratio, philosophies of classroom management • How will we know if any given approach works? • What will we do with students who don’t learn this way? • How will the parts of the curriculum relate? • What learning strategies do successful learners use?

  45. NOW WHAT?

  46. Some options... • Solid training in UbD • Conducting a School Portfolio • Setting up a systematic system for data collection and analysis • Selecting a data warehouse • Training the staff in general

More Related