When is Disproportionality Truly Disproportionate?

1. When is Disproportionality Truly Disproportionate? Clifford V. Hatt, Ed.D., ABPP & Jon Thompson, Psy.D. Virginia Beach City Public Schools Virginia Beach, VA

2. About the Presenters • Clifford V. Hatt, Ed.D., ABPP • Coordinator, Psychological Services Virginia Beach City Public Schools • Licensed Clinical Psychologist; Diplomate in School Psychology, American Board of Professional Psychology • Adjunct Professor of Psychology, Norfolk State University & The College of William & Mary • Member, Virginia Department of Education Task Force on Disproportionality

3. About the Presenters • Jon C. Thompson, Psy.D. • Licensed Clinical Psychologist, Virginia Beach City Public Schools • Private Practice, Churchland Psychiatric Associates, Portsmouth, VA • Listed in National Register of Health Service Providers in Psychology • Member, Virginia Task Force on Disproportionality

4. Participants will … • Be able to discuss the three most common methods for determining the presence of disproportionality and understand the relative strengths and weaknesses • Be able to discuss the importance of significance testing when data is analyzed for differences between groups

5. Participants will … • Learn how to apply a formula to convert simple proportions to z-scores for significance testing • Review how the application of a conversion to z-scores would change the interpretation of disproportionality data from Virginia

6. Disproportionality “Disproportionality refers to comparisons made between groups of students by race or ethnicity or language who are identified in special education. Where students from particular ethnic or linguistic groups are identified either at a greater or lesser rate than all other students then that group may be said to be disproportionately represented in special education.” Draft Rubric for Looking at District Practices

7. Disproportionality • How do you measure it? • How should it be measured? • What constitutes disproportionality? • Is there a “correct” or “ideal” proportion?

8. Disproportion vs. Incidence • Do disabilities and illnesses occur in the general population and in subpopulations at the same rate? • If the incidence of mental retardation occurs in 1% of the general school age population (based on USDOE data), can we expect it to occur at that same rate in specific school-aged ethnic groups?

9. 2004: % of Students Age 6-21 Served Under IDEA, Part B

10. Incidence Rates Reported in DSM-IV-TR Learning Disability: Approximately 5% of students in public schools are diagnosed with a learning disability (reading, written language, math) Mental Retardation: Approximately 1% of general population is diagnosed with mental retardation

11. Is Special Education Data Consistent?

12. Consistency of Comparative Data • If Special Education Data is consistent, which incidence rate should be used for comparison? • United States Data • Virginia Data • DSM-IV-TR

13. Current Methods: Which One Do You Use? • Risk Index • Odds/Relative Risk Ratio • Composition Index

14. Proportional Concept Entire Student Population of LEAville 10,000 African/Amer 3,000 (30%) White 6000 (60%) Asian/PIsland 150 (1.5%) Amer Ind 100 (1%) Hispanic 750 (7.5%)

15. Proportional Concept • Assumes same ethnic proportions will/should be reflected in all educational situations • Special Education Classifications • Referrals for educational assistance • Discipline referrals • AP classes • Drop out rates • Seems simple and self-evident

16. For every complex problem there is an answer that is clear, simple, and wrong. H. L. Mencken (1880-1956)

17. LEAville, USA

18. Risk Index Obtained by dividing # of students from a given ethnic background identified with a disability by the total # of students that exist in a given population • Similar to incidence rate • Caveat • Purely descriptive analysis

19. Odds Ratio(Relative Risk Index) • Uses the Risk Index of an ethnic minority as the nominator and the Risk Index of whites as the denominator • Gives simple comparison of groups with whites serving as the control group • Can determine whether a particular ethnic group will be identified as having a disability more or less often compared to whites

20. Odds Ratio • Caveat • No way to evaluate for significance • Assumes whites are the majority and are representative of a disability group as a whole • Assumes all disabilities are equally represented across ethnic groups

21. LEAville, USA

22. Is LEAville Disproportionate?

23. Composition Index • Calculated by dividing the # of students of a racial or ethnic group in a category by the total # of students in that ethnic or racial group in the total student population. • Provides composition for a pre-determined population so that when all compositions of the different ethnic groups are added together, total equals 100% • Composition of the identified category is then compared to composition of the population as a whole

24. Composition Index • Caveat • Once again, no way to evaluate for significance • Highly susceptible to interpretation flaws when population under consideration is small • Assumes all disabilities occur with equal frequency across all populations

25. What About the Composition Index? • Looks at whether the composition of a particular ethnic group in a category is consistent with the composition of that ethnic group in the total student body • Let’s examine African-American for LEAville • Remember that according to the Risk Ratio, African-American’s may be UNDER-represented in Special Education!

26. LEAville Composition Index

27. Arbitrary Numbers • It may be recognized that composition ratios are not very useful when comparing subgroups. • Some type of “statistical” adjustment is sometimes suggested without any clear rationale. • Some states may use “+/- 20 percentage points” to suggest significant disproportionality (Coutinho & Oswald, 2004).

28. Proposal for Statewide Analysis • Use commonly applied analysis techniques, but provide a more sophisticated method to analyze for significance • Look at Virginia data as a whole, using procedure similar to a Chi-Square analysis where we compare the expected composition of data to the obtained composition of data

29. Is Special Education Data Consistent?

30. Preliminary Analysis for Virginia Data

31. Proposed Data Analysis of Individual Districts (Something Old) • Use the commonly applied Risk Index to determine what percentage of an ethnic group is identified with a particular disability • # of students of Hispanic ethnicity that are identified as having a disability divided by the # of students of Hispanic ethnicity in the population • Compare the Risk Index to the percentage of a given ethnic group that makes up the population as a whole • # of students of Hispanic ethnicity divided by the total number of students in a population

32. Proposed Data Analysis of Individual Districts (Something New) • Use a formula that converts proportions to z-scores • Identify those districts that have a z score greater than 2 (i.e., 2 standard deviations above the mean) • z-scores are based upon standard deviation units, with a z-score of 0 = average, +/- 1 = one standard deviation above/below the mean, etc.

33. Standard normal distribution -2 -1 0 1 2

34. Within 2* SDs (* really 1.96) 95% -2 -1 0 1 2

35. Formula Key

36. Formula for z

37. Formula in Special Ed Terms • z = (% min spec ed - % min students)/ sqrt[(x * 1-x)*(1/n1 + 1/n2)] • x = (# min students + # spec ed min students)/(n1 + n2) • n1 = total # minority students • n2 = total # spec ed students

38. Results

39. Implications • Different choices in the use of the comparison group make a difference! • The method you chose to compare data makes a difference! • Disproportionality is a problem regardless of method. • More specific methodology allows for more appropriate, targeted remedies. • Can be easily calculated in an Excel spreadsheet format.

40. Implications • Modified z-score method provides a way to determine whether the differences between predicted and obtained proportions are meaningful. • More appropriately identifies districts that need to have corrective action. • Also provides a way to determine progress or change following interventions. • Accounts for differences in size or number of students under consideration.

41. More on the Method • This type of significance testing is used in public health settings as well. • The University of North Carolina Chapel Hill School of Public Health has a learning module about inferences in the difference in two population proportions at • Preparedness Center Training Site: Comparing Two Proportions

42. A Little Comic Wisdom • Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies. Groucho Marx (1890-1977)

43. Contact Us • Cliff Hatt, Ed.D. ABPP • Clifford.Hatt@vbschools.com • Jon Thompson, Psy.D. • Jon.Thompson@vbschools.com • Virginia Beach City Public Schools Psychological Services 1413 Laskin Road Virginia Beach, VA 23451 Phone: 757-263-2700 Fax: 757-263-2702