1 / 44

Making More Sense of School Data 分析學校自評數據

Making More Sense of School Data 分析學校自評數據. Dr. Soh Kay Cheng 蘇啟禎博士 Consultant (QA) EMB, Hong Kong SAR jensoh@singnet.com.sg. Is a special language Ensures objectivity Avoids misunderstanding Facilitates communication. 它是特殊的語言 它確保客觀 它避免誤解 它促進溝通.

cheng
Download Presentation

Making More Sense of School Data 分析學校自評數據

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making More Sense of School Data分析學校自評數據 Dr. Soh Kay Cheng 蘇啟禎博士 Consultant (QA) EMB, Hong Kong SAR jensoh@singnet.com.sg Soh KC(2006) Making sense of school data

  2. Is a special language Ensures objectivity Avoids misunderstanding Facilitates communication 它是特殊的語言 它確保客觀 它避免誤解 它促進溝通 Why do we need data (statistics)?為何要用數據(統計數字)﹖ Soh KC(2006) Making sense of school data

  3. Describing status e.g., KPM4 Teacher qualification & experience Describing views e.g., KPM 2 Staff’s views on leadership Describing performance e.g,, KPM22 Student attendance Describing trends e.g., KPM9 Students’ reading habits of past 3 years Techniques for comparisons e.g., Compare KPM4 Teacher qualification with Hong Kong’s Reference Data 描述現狀 例如 KPM 4教師的學歷與經驗 描述見解 例如 KPM 2教師對領導的評價 描述表現 例如KPM 22學生出席率 描述趨勢 例如KPM 9過去三年學生閱讀習 比較技術 例如 KPM 4教師學歷與本港參考數據比較 Data for School Self-evaluation學校自評的數據 Soh KC(2006) Making sense of school data

  4. A statistic (e.g., %) standing alone has little or no meaning. One or more points are needed for meaningful interpretation of the given statistic. Reference points may be implicit, assuming the users have common understanding. Implicit reference points need be made explicit. 孤立的數據（如%）沒有多大意義甚至毫無意義。數據要有意義﹐必須另有數據作為參考。 參考數據可能是隱含的﹐假設數據使用者已有共識. 隱含的數據必須明朗化﹐明確指出參考數據的性質。 Describing needs reference data描述需要參考數據 Soh KC(2006) Making sense of school data

  5. Good for a laugh • A professor of statistics meets a colleague on his way to the lecture. The colleague heartily greets him, “Good morning, professor!” and then respectfully asks, “How is your wife?” • And the professor absent-mindedly says, “….?” Soh KC(2006) Making sense of school data

  6. Describing status描述現狀 Soh KC(2006) Making sense of school data

  7. How popular are these ECAs?這些課外活動受歡迎嗎 • The number of groups may not reflect the popularity of each activity. • The nature of the activities may have an impact on their popularity. • In addition to the number of units, it is good to report also the group sizes to reflect their populatiry. • 每種活動的組別數目不一定反映活動受歡迎的程度。 • 活動是否受歡迎﹐和活動的性質有關。 • 除報告組數外﹐同時也報告各組人數﹐更能反映活動受歡迎的程度。 Soh KC(2006) Making sense of school data

  8. EMB issued Reports on Key Performance Measures Reference Data 2003/2004 for both primary and secondary schools. The median is the mid-point of a distribution. 50% of the schools are at or above (below) the median. Top 25% are at or above the 75th percentile. Likewise, bottom 25% are below the 25th percentile. The middle 50% are between the 25th and 75% percentiles. 教統局印發2003/2004年度中小學校表現評量參考數據報告。 中位數將學校分為上下兩個群組﹐各有50%。 最高的25% 在75百分位數或以上。最低的25%在25百分位數之下。中間的50%在兩者之間。 Box-and-whisker plot as reference 盒鬚圖作為參照 Soh KC(2006) Making sense of school data

  9. Use of box-and-whisker plot盒鬚圖的運用 The percentage of subject-trained Chinese and Mathematics teachers of school are both 85%. How would you report on these? For Chinese, the school is at the median of reference data For Mathematics, the school is among the top quarter. 本校已接受專科訓練的中文和數學教師的百分率均為 85%。這情況怎麼報告？ 中文教師方面，本校位列於參考數據的中位數。 數學教師方面，本校位列於最高的四分之一之內。 * * 85% Soh KC(2006) Making sense of school data

  10. Description is not evaluation描述不等於評估 • Is it good that the teachers spent 82.2 hours and HK$127.3 on professional development? What about the principal? • Reference points needed could be Hong Kong norm, school’s past records, or pre-determined targets. • 教師花費82.2小時和港幣127.3元於專業發展，理想嗎？校長方面呢？ • 其他可用的參考資料，如香港常模、本校往年數據、或預定目標。 Soh KC(2006) Making sense of school data

  11. Good to have another laugh • A young man boasts about his wife and says, “My wife has a perfect figure of 100, it’s 38-24-38.” • His middle-age friend, not wanting to lose face, says calmly, “My wife has a perfect figure adding up to 100, too. It is ….” Soh KC(2006) Making sense of school data

  12. Describing views描述見解 Soh KC(2006) Making sense of school data

  13. Scale and dispersion尺度與變異 • 報告採用五度量表。尺度應該說明。假如所用的是十度量表，則各階層領導所得的評估便非常不同。 • 兩個平均數4.01似乎顯示校長與中管理層得到同樣的好評。假如校長的標準差(SD)是0.06，而中管理層的是1.12，應作何解釋？ • Five-point scale was used.This fact is implicit and needs be made explicit. Should it be a 10-point scale, the interpretation would be rather different. • The two means of 4.01 give the impression that the Principal and the Middle Management were equally favorably evaluated.What if the SD for Principal is 0.06 and that for the Middle Management 1.12? Soh KC(2006) Making sense of school data

  14. Poverty is relative A rich American wanted to show his son how rich they were. He took the little boy to a seaside village in the South. When everything was over, the father asked, “Son, what have you learned from this trip?” The boy said, “Oh, yes! We keep one dog and they have four. Sitting at our patio, the view ends at the gate 50 yards away, but at their, there is no end to the horizon. We built walls to protect ourselves, they have friends to protect them….Thank you father, for showing me how….” Soh KC(2006) Making sense of school data

  15. Describing performance描述表現 Soh KC(2006) Making sense of school data

  16. Absolute and relative standards絕對標準與相對標準 Absolute standard: 46.3% borrowed at least once in two weeks. 絕對標準 • 預期目標：每兩週借用資料至少一次的學生有50%或以上。 • 實際上﹐每兩週借用資料至少一次的學生有46.3%﹐接近預期目標50%。。 Relative standard: 36.5% is the mode. 相對標準 • 借用層次有五個。其中最高的是“每月一次”。 • 因此﹐學生借用資料頻數的眾數(36.5%)為“每月一次”。 Soh KC(2006) Making sense of school data

  17. The dangerous average • Mr. Dumb Bell jumped into the sea from a jetty and got a big hump on his forehead, because the sign board says: First 30 meters,average depth 5 meters! Soh KC(2006) Making sense of school data

  18. Describing trends描述趨勢 Soh KC(2006) Making sense of school data

  19. Evaluation varies with reference data評價隨參考數據而變 • Relatively speaking, there was an improvement in 2004 over 2003 in all three subjects, but, there is a retrogression from 2004 to 2005 in the two languages. • Assumption: Papers of the three years are equivalent. • Need HK norms for the three years for proper interpretation. • 相對而言，2004的成績比2003的好，有進步。但是，從2004 到2005, 兩語文科有退步的現象。 • 假設：三年的考卷難度相同。 • 正確的解釋需要三年的香港常模作為參考資料。 Soh KC(2006) Making sense of school data

  20. Effect Size效果强度 *比參考數據 +/-0.1或以上 • Comparison with Reference Data is a good effort. However, +/-0.1 is arbitrary. • If SDs are available, then effect sizes can be calculated for more meaningful interpretation. • 將本校的情況和參考數據比較，是好的做法。 但是﹐以+/-0.1為臨界值﹐似乎武斷。 • 如有標準差﹐可轉化為效果強度(Effect size),更有意義。 Soh KC(2006) Making sense of school data

  21. Conversion to effect size效果強度的轉化 註﹕參考數據是虛擬的。 • With reference to the normal distribution, ES can be used to evaluate differences in percentages. ES = (Mean – Norm) / SD • 如參照常態分佈﹐百分率可轉化為效果強度﹐以便檢定百分率差異的意義。 效果強度 = （平均數 – 常模）/ 標準差 Soh KC(2006) Making sense of school data

  22. Effect size ‘standards’效果強度的‘標準’ Soh KC(2006) Making sense of school data

  23. 用數據溝通，力求客觀共識。 用數據描述現狀﹑見解﹑表現﹑與趨勢。 用參考數據作有意義的詮釋。 用百分比作比較。 用參考數據來評估。 參考的數據可能是常模﹑記錄﹑或目標。 兩個相同的平均數可能有不同意義。注意變數的大小。 用絕對標準或相對標準描述表現。 用曲線圖表達趨勢﹐並注意隱含的假設。 用效果強度的「標準」進行客觀的評估。 Use data to ensure objectivity and common frame of mind. Use data to describe status, views, performance, and trends. Use reference data for meaningful interpretation. Use %’s for comparison. Use reference data for evaluation。 Data for reference may be the norms, past records, or targets. Two means of the same magnitude may have different meaning. Watch out for difference in variability. Use absolute or relative standard to describe performance Use curves to indicate trends and watch out for tacit assumptions. Use effect size ‘standards’ for objective evaluation of effects. Summary綱要 Soh KC(2006) Making sense of school data

  24. Statistics are estimates I asked a statistician for her telephone number and she gave me an estimate. Soh KC(2006) Making sense of school data

  25. Comparison Techniques 比較技術 Soh KC(2006) Making sense of school data

  26. Purpose • School reports always present summary data such as the mean, the SD, etc. • Statistical calculators on the Internet can be used to make some of the needed comparisons. • This part of the seminar introduces such calculators to enable schools to make finer interpretation. Soh KC(2006) Making sense of school data

  27. Comparison with expected performance與預期表現比較 The result of a survey on KPM2 Teachers’ view on the principal’s leadership. If the expected mean is 3.50, was the principal more favorably evaluated than is expected? This calls for a one-sample t-test and the critical value of p is set at 0.05. KPM2 教師對校長領導能力的評估，調查結果如表所示。 如果預期平均數為3.50，校長所得平均數是否較預期的為高？ 這需要用單組t-測加以鑒定，並以0.05為p的臨界值。 Soh KC(2006) Making sense of school data

  28. The result shows that the obtained mean (3.71) is statistically greater than the expected mean (3.50). The principal was evaluated higher than the expected. 統計分析結果顯示，實得的平均數（3.71）的確高於預期的平均數（3.50）。校長所得評估的確比預期的高。 http://glass.ed.asu.edu/stats/analysis/ttest.html Soh KC(2006) Making sense of school data

  29. Comparison of two groups兩群組比較 The table shows the ratings on KPM11 School culture given by teachers and parents. Did the parents evaluate the school more favorably than did the teachers? This calls for an independent t-test and the critical p is set at 0.05 上表顯示教師與家長對KPM11 學校文化的評估。兩者對學校的確有不同評價嗎？ 這必須用獨立t-測加以鑒定，並以0.05為p的臨界值。 Soh KC(2006) Making sense of school data

  30. http://glass.ed.asu.edu/stats/analysis/t2test.html The result shows that the difference (0.23) between the two means is unlikely a chance occurrence. Parents did evaluated the school more favourably than did the teachers. 統計結果顯示，兩平均數之差(0.23) 並非機遇現象。家長對學校的評價的確高於教師的評價。 Soh KC(2006) Making sense of school data

  31. Comparison of many groups多群組比較 The ratings on KPM 11 School culture as given by teachers, students, and parents are shown above. Did the three groups differ in their rating? This calls forone-way ANOVA (analysis of variance) followed by pair-wise comparisons. 針對 KPM 11 學校文化，教師、學生、與家長作以上的評估。三組的評價的確有差異嗎？ 這需要用單向變異分析（one-way analysis of variance）鑒定，並再用配對t-測。 Soh KC(2006) Making sense of school data

  32. http://statpages.org/anova1sm.html The analysis shows that p<0.05; there is at least one significant difference among the three means. 分析結果顯示p<0.05，表示至少有一對的平均數有非機遇的差異。 Soh KC(2006) Making sense of school data

  33. http://glass.ed.asu.edu/stats/analysis/t2test.html As there are three pair-wise tests, Bonferroni adjustment is applied to avoid accumulation of error. For three tests, the p-value should be 0.05/3 or 0.0166. To check on these p-values, see the next slide. 有三對平均數作配對比較，必須作 Bonferroni調整，以避免誤差的累積，而p-值應該是0.05/3 or 0.0166. 要確定這三個p-值，請看下頁。 Soh KC(2006) Making sense of school data

  34. http://department.obg.cuhk.edu.hk/researchsupport/T_Test.asp (65 + 120 -2) For all three t-values (21.76, 43.82, and 27.84), the corresponding p is 0.0001. The differences among the three means are statistically significance; the differences are very unlikely chance occurrences. 三個 t-數(21.76, 43.82, and 27.84)配對比較的相應p-值是0.0001. 它們 之間的差異並非機遇現象；其間的確有差異。 Soh KC(2006) Making sense of school data

  35. Comparison with reference data (1)與參考數據比較（一） For KPM 4, the school has 90% of its 65 teachers professionally trained.Is this percentage significantly lower than the Hong Kong Reference Data? This calls for a chi-square test of goodness of fit. The percentages need be converted into frequencies for analysis. 學校的65位教師，有90%受過專業訓練。這和香港參考資料比較，有差異嗎？ 這需要用卡方測驗來鑒定。並需先將百分數轉為頻數。 Soh KC(2006) Making sense of school data

  36. http://www.georgetown.edu/faculty/ballc/webtools/web_chi.htmlhttp://www.georgetown.edu/faculty/ballc/webtools/web_chi.html Soh KC(2006) Making sense of school data

  37. Soh KC(2006) Making sense of school data

  38. The chi-square test results show p>0.05; the school’s distribution fits the Hong Kong distribution. 卡方測驗結果，p>0.05;顯示學校的分配情況和香港參考數據相配。 Since the chi-square is not significant, the school is not different from the Hong Kong reference data. Soh KC(2006) Making sense of school data

  39. Comparison with reference data (2)與參考數據比較（二） For KPM 4, the school reported the percentages of teachers with different qualification as shown above. Doesthe school’s distribution differ significantly from the Reference Data? This calls for a chi-square test. 學校教師學歷分佈如表所示。與參考數據相同嗎？ 用卡方測驗鑒定。 Soh KC(2006) Making sense of school data

  40. Since the chi-square is not significant (p>0.05), the school’s distribution is not different from the Reference Data. 卡方檢查結果，p>0.05 ; 學校的分佈情況和參考數據的沒有不同。 Soh KC(2006) Making sense of school data

  41. Summary綱要 One-sample t-testfor comparing a observed mean to an expected mean. Two-sample t-testfor comparing the means of two independent groups. One-way ANOVAfor comparing more than two independent means. Significance of t-valuefor checking the probability of an obtained t-value. Chi-square testfor ascertaining association between membership and performance. 單組t-測：比較實際平均數與預期平均數。 兩組t-測：比較兩組的平均數. 單向變異分析：比較多過兩組的差異。 t-值的臨界值：檢查所得t-值是否機遇現象。 卡方測驗：檢定兩個分佈情況之間的差異。 Soh KC(2006) Making sense of school data

  42. Hyperlinks to calculators One-sample t-test http://glass.ed.asu.edu/stats/analysis/ttest.html Two-sample t-test http://glass.ed.asu.edu/stats/analysis/t2test.html One-way ANOVA http://statpages.org/anova1sm.html Significance of t-value http://department.obg.cuhk.edu.hk/researchsupport/T_Test.asp Chi-square test http://www.georgetown.edu/faculty/ballc/webtools/web_chi.html Soh KC(2006) Making sense of school data

  43. He who laughs last, laughs best最後一笑 Q: As a principal, how do you develop your teachers professionally? A: As a responsible leader, I make sure that everyone of them is busy and works hard. Q: What are they working hard on? A: That does not really matter, as long as they are working non-stop. Q: Could you give me an example? A: For instance, …. Soh KC(2006) Making sense of school data

  44. Thank you 謝謝 Soh KC(2006) Making sense of school data

More Related