Data Analysis for School Improvement

1. Data Analysis for School Improvement EDL 733 Joseph Cannella

2. About data… • It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts. • Sherlock Holmes, from “A Scandal in Bohemia” • Sir Arthur Conan Doyle

3. About data… • If we could first know where we are, and whither we are tending, we could better judge what to do, and how to do it. • “House Divided Speech, June 16, 1858 • Abraham Lincoln

4. About data… • “In God we trust…all others bring data.” • W. Edwards Deming

5. Show me the data!

6. Data-Driven Decision Making • Data-Driven Decision Making • (D3M or D3M or 3DM) • D3M is the process of making choices based on appropriate analysis of relevant information. School district decision makers are using technology and professional expertise to improve instruction and operations.

7. Why use data? • Until recently, decisions made according to • Regulations (We have to…) • Instinct (I think that…) • Tradition (We’ve always done it that way) • Access to better information enables educational professionals to • test their assumptions • identify needs • measure outcomes

8. Why use data? • D3M can help schools • to provide more individualized instruction to students • track professional development resources • identify successful instructional strategies • better allocate scarce resources • communicate better with parents and the community.

9. Why aren’t we doing this? • Lack of training and interoperability are the main barriers to more effective data-driven decision making, according to a survey conducted by Grunwald & Associates on behalf of CoSN in 2004.

10. Why aren’t we doing this? • Lack of training: 50% • Interoperability issues: 42% • Lack of DA know-how: 39% • Absence of priorities re which data: 36% • Lack of uniformity in collection: 35% • Outdated technology: 31% • Data quality/accuracy: 24% • Timing of data collection: 24% • User interface is too complicated: 22%

11. How does a district decide what data to collect? • Most districts are data rich. • They have too much information in too many places to effectively use it. • The challenge is to integrate these disparate systems and make the information available in timely, easy-to-understand reports so that decision makers can affect student achievement.

12. Data rich, information poor • Data is scattered across a variety of disconnected systems • Employees struggle to find a way to access and process data in a sane and orderly fashion. • Prevents schools from producing timely and fact-based information

13. Data rich, information poor • In spite of the computer revolution • In the era of No Child Left Behind and high-stakes accountability, the lack of intelligible, actionable information is no longer tolerable.

14. Data use under NCLB • House Committee on Education and the Workforce • “put an end to education fads that masquerade as sound science” (Guerard, 2002) • Phrases like “evidence-based decisions” and “scientifically based research” occur 111 times in the NCLB act.

15. Knowledge Gap

16. Data Warehouse • A data warehouse is a storage facility • Integrating sources of information about every student and staff member in the school system. • Providing easy access to this data is a crucial element of a data warehousing solution. • At the same time, much of the information is highly confidential. Finding the right balance between access and security, flexibility and control, is an ongoing challenge for K-12 IT departments.

17. Data Warehouse Applications SIS Assessment Finance Instruction Reporting and Analysis ServicesTurning data into useful information Dissemination Sharing data with the community (ie: report cards) Data Warehouse Reports State and Federal Reporting Meeting reporting compliance Training Learning how to use data to make informed decisions. Personalized Instruction Source: US Department of Education, 2003.

18. What a DW is • The data warehouse, by design, contains current as well as historical data. • The DW primary purpose is to enable the user to see a time dimension with regard to data and to enable extraction of trends and time-oriented analysis. • The DW is a snapshot of the enterprise.

19. What a DW is… • Read only repository • Data is current, historical, summarized, static at both a summary and an atomic level • Unidirectional • Integrated • Subject oriented • Time oriented • Analytical • Used to answer comparative and “what if” questions • Reporting friendly • Dynamic • Only as good as the source data

20. What a DW is not… • Operational • Volatile data • Bi Directional • Source of data for operational systems • Omniscient • Due to ever changing requirements, no data warehouse or reporting repository can anticipate every need that will ever exist. • Not a substitute for well defined business processes • A Silver bullet

21. Assessment Assessment Item Assessment Item & Program Assessment & Program Course Attendance Course Enrollment Course Instructor Daily Attendance Finance Location History Program Participation School Days School Enrollment Special Education Events Special Education Services Typical DW Domains

22. Staff Attendance Staff Certifications Staff Demographics Student Demographics Student Demographics with Reporting Districts Student Grades Student Infraction Student Infraction Responses Typical DW Domains

23. Multiple Measures • Victoria Bernhardt, Ph.D., of Education for the Future • Considers intersections of data not commonly considered in school improvement efforts • Demographics, Student Learning, School Processes, Perceptions

24. Multiple MeasuresGraphic

25. Another quote… • 100% of systems are perfectly aligned to get the results they’re getting

26. Enhance Strategic Thinking

27. Additional Resources Data-Driven Decision Making (especially for education) • 3D2Know • 3d2know.cosn.org/index.html • Consortium of School Networking • www.cosn.org • Education for the Future • eff.csuchico.edu/home/

28. Carpe Datum! Seize the data!