Introduction to The Many Uses of Data

1. Introduction to The Many Uses of Data Larry Condelli, Ph.D. Mary Ann Corley, Ph.D. American Institutes for Research Mary Ann Corley American Institutes for Research

2. Data: A Carrot or a Stick? Data Can Be Used. . . • To highlight, clarify, and explain what’s happening in your program, OR • To show what’s not happening in your program. Mary Ann Corley American Institutes for Research

3. How Do You Use Your NRS Data? Do you: • Look at your NRS reports each year; • Make statements about how these tables do notreally reflect what goes on in the classroom; • Attempt to explain them to your publics; • Put results away in your files; and • Hope that the NRS will go away before next year? Mary Ann Corley American Institutes for Research

4. Using Data to Your Advantage (i.e., as a carrot) is up to you. Mary Ann Corley American Institutes for Research

5. Data Tells You • Where you’ve been; • Where you are; • Where you’re going, and • How to get there. Mary Ann Corley American Institutes for Research

6. Data • That which separates successful adult education programs from those that are not successful in their reform efforts! • That which can help you design a quality program to ensure that your learners meet their goals. Mary Ann Corley American Institutes for Research

7. Did You Know That Data Can. . . • Guide you to improveinstruction? • Measure program success and effectiveness? • Tell you if what you are doing is making a difference? • Tell you which programs are getting the results you want—and which are not? • Get to the rootcauses of problems, such as poor retention? • Sell your board, your funders, and your community on the value of your program? Mary Ann Corley American Institutes for Research

8. The same data • Attendance/Enrollment Numbers and Patterns, • Student Test Scores and Learning Gains, • Student Drop-out/Completion Rates, • Teacher Characteristics, • Student Demographics, • Program Spending, can be used for • accountability, • program promotion/marketing, and • program management and improvement. Mary Ann Corley American Institutes for Research

9. Functions of Data • Help us replace hunches and hypotheses with facts concerning the changes that are needed (program management and improvement); • Help us identify root causes of problems (program management and improvement); • Help us identify whether goals are being met (accountability); • Tell our funders, our boards, and our communities about the value of our programs and the return on their investments (marketing). Mary Ann Corley American Institutes for Research

10. Properly Used Data • Can “help identify and uncover powerfulsolutions to your program’s biggest problems.” • But first you need to “take your data and torture it until it confesses.”  (Victoria Bernhardt, Data Analysis for Comprehensive School Improvement, 1998) Mary Ann Corley American Institutes for Research

11. Quote of the Day “People without information cannot act. People with information cannot help but act.” (Ken Blanchard) Mary Ann Corley American Institutes for Research

12. Barriers to Using Data • Your program’s/school’s culture does not focus on data; • Gathering data is perceived to be a waste of time; • Staff lack adequate orientation and training in the value of data collection; • Staff have had negative experiences with data collection; • Staff are not aware of other programs’ successes in using data; • Staff think that data is collected “just for the state or the feds.” Mary Ann Corley American Institutes for Research

13. Focusing the Data If you know why, you can figure out how. . . (W. Edwards Deming) Mary Ann Corley American Institutes for Research

14. Student Ethnicity by Site Mary Ann Corley American Institutes for Research

15. Program Components by Data Function Mary Ann Corley American Institutes for Research

16. A Data Use Model for Program Management and Improvement TwoComponents of the Model: • Data Analysis and • Program Improvement Mary Ann Corley American Institutes for Research

17. 4 Steps to Data Analysis • Identify issues or topics; • Develop questions to address the selected issues or topics; • Plan analyses; and • Analyze and interpret the data. Mary Ann Corley American Institutes for Research

18. 3 Steps to Program Improvement • Develop a plan for initiating change; • Implement the plan; and • Evaluate whether the change has made a difference. Mary Ann Corley American Institutes for Research

19. Model: Using Data for Program Management and Improvement Mary Ann Corley American Institutes for Research

20. Focusing the Question Break the question into inputs and outputs • Inputs (what your program contributes): • Hours of instruction per week • Teacher education, experience, full-time/part-time • Instructional Curriculum • Outputs (outcomes, results, ROI): • Improved test scores • Advances to next educational level • Earned GED credentials • Improved attendance Mary Ann Corley American Institutes for Research

21. Focusing/Refining the Question • Poor Question: • Is my program effective for all students? • Good Question: • Do different types of students in my program achieve their goals? • Better Question: • How do attainment of a GED credential, entry into employment, and educational gain differ by student age and ethnicity? Mary Ann Corley American Institutes for Research

22. Focusing/Refining the Question • Poor Question: • Does my program have good teachers? • Good Question: • Does student learning differ by teacher? • Better Question: • Do students in classes taught by instructors who have more teaching experience have higher test scores than those taught by new teachers? Mary Ann Corley American Institutes for Research

23. Focusing/Refining the Question • Poor Question: • Is my program helping the most needy adult learners? • Good Question: • Are low-literate students learning less in my program than other students? • Better Question: • Are literacy and beginning level ABE students advancing levels at the same rate as students who enter my program at other levels? Mary Ann Corley American Institutes for Research

24. Developing a Data Analysis Plan • What data do you already have that will answer your question? • What additional data, if any, will you need to answer your question? • Where are you going to get the additional data? • What’s your plan for obtaining the data you need—and what’s your timeline? Mary Ann Corley American Institutes for Research

25. Analyzing and Interpreting Your Data • Keep your originalquestion in mind. • Look for patterns and differences. • Use appropriate data and statistics. • Disaggregate the data. • Consider dataquality. • Draw appropriate conclusion(s). • Remember serendipity: be open to the unexpected. Mary Ann Corley American Institutes for Research

26. Averages and Variation • Mean:the average score (add up the scores and divide by the total number of scores) • Median: the score that falls dead-center within the distribution (e.g., half the scores fall above it and half fall below it) • Mode:the score that occurs most frequently in the distribution • Range: the difference between the lowest and highest scores Mary Ann Corley American Institutes for Research

27. Mean, Median, Mode, Range, SD Mean: 92.6 hours Median: 85 hours Mode: 45 hours Range: 178 (24-202) Standard Deviation: 62.0 Mary Ann Corley American Institutes for Research

28. Presenting Your Data Frequency Tables • Show numbers and percentages by category, e.g., ethnicity, gender, age • Simple frequency table versus a two-way table, or cross-tabulation, e.g., ethnicity by age Mary Ann Corley American Institutes for Research

29. Presenting Your Data Graphs and Charts • Bar Chart: • Categories displayed as bars, e.g., enrollees by age • Pie Chart: • Shows a slice of the pie in proportion to the whole, e.g., various ethnicities of total enrolled students • Line Chart: • Data form a continuous measure (not categories), e.g., pre-test and posttest scores Mary Ann Corley American Institutes for Research

30. Presenting Your Data Communication Strategies • Article by Education Reporter in Local Newspaper • Public Meeting or News Conference Presented by Superintendent or Dean • Newsletters • Special Events, e.g., Open House • Web Sites • Annual Report Mary Ann Corley American Institutes for Research