EDU 5900 REKA BENTUK PENYELIDIKAN

1. EDU 5900 REKA BENTUK PENYELIDIKAN EDU 5900REKA BENTUK PENYELIDIKAN

2. OBJEKTIF • Dapat menjelaskan maksud kesahan dan keboleh percayaan sesuatu alat pengukuran penyelidikan • Dapat menghuraikan jenis-jenis kesahan dan keboleh percayaan alat pengukuran yang digunakan dalam penyelidikan

3. KESAHAN (VALIDITY) DAN KEBOLEHPERCAYAAN (RELIABILITY)

4. Validity refers to the degree in which our test or other measuring device is truly measuring what we intended it to measure. KESAHAN (VALIDITY) Sejauh mana alat mengukur apa yang ia sepatutnya ukur

5. Kesahanialahsejauhmanakahdapatankajianatau data yang dikumpuldapatmemberikanpenjelasan yang bermaknakepadakajian yang dilakukan (ary et. Al (2005) Kesahanbermaksudkebolehanujianmengukurapa yang sepatutnyadiukur, Youngman & Eggleston, 1982; Sax & Newton, 1997) Kesahansesuatualatpengukuranmerujukkepadasejauhmanakahalat yang digunakanmengukur data yang dikehendakiuntukmencapaiobjektifkajian (MohdMajidKonting, 1990)

6. Based on Internal Structure Construct (determination of the significance, meaning, purpose, and use of the scores) Kesahan Gagasan Tiga Aspek Kesahan Based on Relations to Other Variables Based on content Kesahan Kriteria Kesahan Kandungan Criterion-referenced (scores are a predictor of an outcome or criterion they are expected to predict) Content (representative of all possible questions that could be asked) Concurrent Evidence Predictive Evidence Content validation is usually carried out by experts

7. MenurutFraenkel & Wallen (1996) kesahanbolehdibuktikanmelaluibeberapakaedahsepertikesahanisikandungan (content validity), kesahantelahan (predictive validity), kesahanserentak (concurrent validity) dankesahangagasan (construct validity)

8. Evidence Based on Content Kesahan Kandungan(Content Validity) • Sejauhmanaalatmerangkumikandungansesuatubidang. • Matlamatutamaialahuntukmemastikansemuaisidankandunganbidang yang diukurmenggambarkanbidangtersebut. • Berdasarkankepadaskopdanobjektifdankandungansesuatubidang yang dikaji. • Pendapatpakarataupenilailuardiperlukanbagimenilaikesesuaianbutiranbagi domain yang dipilih.

9. Content validity… …is concerned with a test’s ability to include or represent all of the content of a particular construct. The question “1 + 1 = ___” may be a valid basic addition question. Would it represent all of the content that makes up the study of mathematics? It may be included on a scale of intelligence, but does it represent all of intelligence? The answer to these questions is obviously no. To develop a valid test of intelligence, not only must there be questions on math, but also questions on verbal reasoning, analytical ability, and every other aspect of the construct we call intelligence. There is no easy way to determine content validity aside from expert opinion.

10. Content validity… • Do the items appear to represent the thing you are trying to measure? • Does the set of items underrepresented the construct’s content (i.e., have you excluded any important content areas or topics?) • Do any of the items represent something other than what you are trying to measure (i.e., have you included any irrelevant items?)

11. Content validity… Sebelumsesuatuinstrumenitudikatakanmempunyaikesahankandungan, lima syaratiniperludipenuhi: • Bidangkandunganmestilahdinyatakandalambentuktingkahlaku yang secaraumumditerimamaknanya. • Bidangmestilahdihuraikandenganjelas. • Bidangmestilahrelevandengantujuanpenggunaanujian. • Hakim-hakim yang berkelayakanmestilahbersetujubahawabidangtelahdisampelsecaramencukupi.

12. Evidence Based on Internal Structure KesahanGagasan(Construct Validity) • To measure several components or dimensions of a construct. • Use Factor Analysis to analyzes correlations among test items and tells you the number of factors present. Its tell you whether the test is unidimensional or multidimensional. • Unidimensional – all the item measure are single construct. • Multidimensional – different set of item tap different construct or different component of a broader construct.

13. …… Internal Structure • Factor analysis tell you how many dimensions or factors your test items represent. • Also can obtain a measure of test homogeneity (i.e., the degree to which the different items measure the same construct or trait) • Use coefficient alpha (Alpha Cronbach) for the test of homogeneity. • If the alpha is low (e.g., <.70) for the test, then some items might be measuring different constructs or some items might be bad. • Examine the items that are contributing to your low coefficient alpha and consider eliminating or revising them.

14. Based on Relations to Other Variables • Obtained by relating your test scores to a relevant criterion. • A criterion is the standard or benchmark that you want to predict accurately on the basis of scores from your test. • Sejauhmanakaitanantaraalatdengankriterialuaran yang berkecuali (samaada item mengukurkriteria yang hendakdiukur). • Ditentukandengananalisiskorelasiantaradua set markah. • Calculate correlation coefficients for the study of validity – validity coefficients. Kesahan Kriteria (Criterion Validity)

15. Concurrent Validity Concurrent Validity refers to a measurement device’s ability to vary directly with a measure of the same construct or indirectly with a measure of an opposite construct. It allows you to show that your test is valid by comparing it with an already valid test. Administering the focal test and criterion test at approximately the same point in time (i.e., concurrently) and then correlating the two set of scores. If the two sets of scores highly correlated, you have concurrent evidence. e.g. A new test of adult intelligence, for example, would have concurrent validity if it had a high positive correlation with the Wechsler Adult Intelligence Scale since the Wechsler is an accepted measure of the construct we call intelligence. An obvious concern relates to the validity of the test against which you are comparing your test. Some assumptions must be made because there are many who argue the Wechsler scales, for example, are not good measures of intelligence.

16. Predictive Validity • Obtain predictive evidence of validity by measuring your participants at one point in time on your test and then, at a future time, measuring them on the criterion measure. • Take more time and effort than concurrent evidence, but it can provide superior evidence that your test does what you want it to do. In order for a test to be a valid screening device for some future behavior, it must have predictive validity. The SAT is used by college screening committees as one way to predict college grades. The GMAT is used to predict success in business school. And the LSAT is used as a means to predict law school performance. The main concern with these, and many other predictive measures is predictive validity because without it, they would be worthless

17. Reliability is synonymous with the consistency of a test, survey, observation, or other measuring device. Imagine stepping on your bathroom scale and weighing 140 pounds only to find that your weight on the same scale changes to 180 pounds an hour later and 100 pounds an hour after that. Base on the inconsistency of this scale, any research relying on it would certainly be unreliable. Consider an important study on a new diet program that relies on your inconsistent or unreliable bathroom scale as the main way to collect information regarding weight change. Would you consider their results accurate? KEBOLEH PERCAYAAN (RELIABILITY)

18. Kebolehpercayaan • Sejauh mana instrumen mengukur dengan tekal apa yang hendak diukur. • Scores from measuring variables that are stable and consistent

19. Test-retest Reliability Types of Reliability Internal Consistency Reliability Equivalent Forms Reliability

20. Test-Retest Reliability Merujuk kepada ketekalan atau stabiliti markah ujian jika dilakukan pada masa yang berbeza. Contoh: Ujian diberikan kepada 100 individu untuk satu masa dan diulangi pada masa berlainan. Dua set markah ini dikorelasikan. Sekiranya individu memperoleh markah tertinggi dalam ujian 1 juga memperolehi markah tertinggi dalam ujian 2, begitu juga individu yang mendapat markah terendah dalam ujian 1 juga mendapat markah terendah dalam ujian, maka dikatakan mempunyai korelasi yang tinggi. Oleh itu soalan ujian tersebut mempunyai kebolehpercayaan yang tinggi.

21. Equivalent Forms Reliability • Refers to the consistency of a group of individual’s scores on two equivalent forms of a test designed to measure the same characteristic. • Menggunakan satu alat yang dibina dan satu lagi yang piawai. • Ditadbir ke atas subjek yang sama dan pada masa yang sama atau masa yang lain. • Equivalent form means that two tests are constructed so that they are identical in every way except for the specific items asked on the test. • This means that they have the same number of items, the items are the same difficulty level, the item measure the same construct, and the test is administered, scored, and interpreted in the same way. • The two set of scores are than correlated. If this reliability coefficient to be very high and positive, that is the individuals who do well on the first form of the test should also do well on the second form, and individuals who performed poorly on the first form of the test should perform poorly on the second test.

22. Internal Consistency Reliability • Internal consistency refers to how consistently the items on a test measure a single construct or concept. • The test-retest methods of assessing reliability are general methods that can be used with just about any test. • Internal consistency measures are convenient and are very popular with researchers because they require one group of individuals to take the test one time. • Two indexes of internal consistency: • Split half reliability • Coefficient alpha

23. Split-half reliability • Splitting a test into two equivalent halves and then assessing the consistency of the scores across the two halves of the test. • Divide the test into halves and correlate the scores from the two halves. • Compute the correlation between scores on the two halves of the test using Spearman-Brown formula. • The low correlation indicates that the test was unreliable, a high correlation indicates that the test was reliable.

24. Coefficient alpha • Lee Cronbach 1951) developed coefficient alpha.. Alpha Cronbach • Coefficient alpha tells you the degree to which the items are interrelated. • Rule of thumb: • At a minimum, greater than or equal to .07 for research purposes and somewhat greater than that value (e.g. ≥ .09) for clinical testing purposes.

25. Sekiranya pekali kebolehpercayaan rendah???

26. MeningkatkanKebolehpercayaan • Pernyataan item mestilah jelas dan tepat. • Arahan mestilah jelas dan ringkas. • Item hendaklah bentuk sejenis. • Situasi dan masa pengukuran hendaklah piawai, serupa dan terkawal. • Elakkan gangguan ke atas subjek. • Elakkan kebimbangan subjek dengan memberi jaminan keselamatan dan kerahsiaan ke atas maklumat yang diberi.

27. Fasa terakhir tinjauan sebelum pengumpulan data bermula. Ujian Rintis Matlamatnya adalah untuk mencari masalah dalam soal selidik, termasuk soalan yang lemah, arahan yang tidak lengkap dan item yang sukar dijawab.

28. Garis panduan ujian rintis Tidak boleh gunakan kumpulan fokus sebenar. Jumlah responden tidak ditentukan dengan tepat, dicadangkan sekurang-kurangnya 25 orang, lebih baik antara 50 – 75 orang. Untuk kajian baharu, lakukan dua kali ujian rintis.

29. Procedures for Administering the Data Collection Develop standard written procedures for administering an instrument Obtain permission to collect and use public documents Respect individuals and sites during data gathering (ethics) Train researchers to collect observational data

30. What Permissions Are Needed:Obtaining Permission Institutional or organizational (e.g., school district) Site-specific (e.g., secondary school) Individual participants Parents of participants who are not considered adults Campus approval (e.g., university or college) and Institutional Review Board (IRB)