1 / 35

Scientific Methods 1

Scientific Methods 1. ‘Scientific evaluation, experimental design & statistical methods’ COMP80131 Lecture 2: Statistical Methods-Basics. Barry & Goran. www.cs.man.ac.uk/~barry/mydocs/myCOMP80131. Scientific Methods 1.

enrico
Download Presentation

Scientific Methods 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scientific Methods 1 ‘Scientific evaluation, experimental design & statistical methods’ COMP80131 Lecture 2: Statistical Methods-Basics Barry & Goran www.cs.man.ac.uk/~barry/mydocs/myCOMP80131 COMP80131-SEEDSM2

  2. Scientific Methods 1 • Scientific evaluation: derivation of useful & reliable statements about some new or existing scientific idea based on an accumulation of evidence which is often in the form of tables of numerical values. • Experimental design: how to generate the quantifiable outputs, the systematic observation & measurement of these outputs and the recording of the resulting data. The experiments are normally designed to test some theoretical prediction of what the researcher expects to happen – a ‘research hypothesis’ • Statistical methods: the means of deriving the required useful and reliable statements from numerical evidence. COMP80131-SEEDSM2

  3. Scientific Enquiry • It may be argued that: • ‘Scientific researchers propose hypotheses as explanations of phenomena & design experimental studies to test these hypotheses’. • It may also be argued otherwise. • Wider domains of inquiry may combine many independently derived hypotheses. • Or not have hypotheses at all, other than contrived ones such as: • ‘This idea can (not) be implemented’ COMP80131-SEEDSM2

  4. Philosophy of Science • Concerns: the underpinning logic of the scientific method, what separates science from non-science, the ethics implicit in science. • Assumes: reality is objective and consistent, humans have the capacity to perceive reality accurately, rational explanations exist for elements of the real world. • Logical Positivism & other theories claim to have defined the logic of science, but have all been been challenged. • Ludwig Wittgenstein (1889-1951) got his PhD in Manchester COMP80131-SEEDSM2

  5. Objectivity, repeatability & full disclosure • Scientific inquiry is intended to be as objective as possible, to reduce biased interpretations of results. • Procedures must be reproducible (i.e. repeatable) • Researchers should: • document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, giving them the opportunity to verify results by attempting to reproduce them. • This practice is called ‘full disclosure’. • Allows the methodology & the statistical reliability of the data to be verified. COMP80131-SEEDSM2

  6. References on Statistics • DJ Hand ‘Statistics – a very short introduction’ Oxford UP 2008 • Schaum’s Outlines ‘Prob & Stats’ 2009 • WG Hopkins ‘A new View of Statistics’ (Google it) • ‘Why is my evil lecturer forcing me to learn statistics?’ (Google it – forget it!!) COMP80131-SEEDSM2

  7. Tables of Results Engli Maths Phys Chem Hist Fren Music Art Avge 81 67 60 104 89 97 72 30 75.0 91 32 42 34 24 65 81 61 53.8 13 123 45 22 92 61 114 11 60.1 91 65 80 23 95 47 101 33 66.9 63 58 44 6 38 58 36 21 40.5 10 28 69 24 84 91 20 102 53.5 28 20 60 18 46 38 -3 79 35.8 55 0 44 85 35 23 11 112 45.6 96 38 49 17 11 42 45 48 43.3 96 21 48 83 80 27 8 101 58.0 16 68 55 35 69 44 40 55 47.8 97 41 64 13 91 63 -13 33 48.6 96 100 34 19 34 53 81 -10 50.9 49 92 70 17 13 39 63 -19 40.5 80 55 58 3 58 87 68 28 54.6 14 42 45 95 63 30 64 46 49.9 42 82 49 19 88 40 42 16 47.3 92 18 53 80 0 52 -17 108 48.3 79 69 53 29 0 6 59 31 40.8 96 31 62 40 77 23 50 65 55.5 A fictitious set of exam results. A sample of 20 students out of a population of 1000. Complete file is: ExamData.xls or ExamData.dat www.cs.man.ac.uk/~barry COMP80131-SEEDSM2

  8. A bit of MATLAB [Marks,Headings]=xlsread('ExamData.xls'); [nRows,nCols] = size(Marks); Headings(1,1:nCols)) Marks Reads in marks from Excel spreadsheet into an array ‘Marks’. Headings read in separately. Miss out ‘;’ to display. ‘%’ is comment. COMP80131-SEEDSM2

  9. A bit more MATLAB % Row with mean of each column: Me = mean(Marks) % Row with standd deviations of cols: St_devs = std(Marks) % Row with variances of cols: Variances = var(Marks) Statistics printed out: Engli Maths Phys Chem Hist Fren Music Art Avge Means: 52.2 49.2 49.7 49.6 55.7 51.0 48.4 50.7 50.8 Std_devs: 28.2 27.2 10.5 31.5 33.3 28.6 33.4 34.1 8.7 Variances: 795 741 110 990 1109 819 1115 1165 75.5 COMP80131-SEEDSM2

  10. Definitions: mean Here is a col of marks, say for French. The mean is the average. It is about 27. This is a ‘statistic’ which summarizes the column of data. Alternatives exist: e.g. median & mode It allows comparisons to be made. If the average is 31 next year, we can hypothesise that the students are better, better taught or the exam was easier, (or maybe the exam room was warmer). (Is the increase of 4 statistically significant?) 46 8 50 6 99 -42 30 23 16 38 60 -3 45 0 30 COMP80131-SEEDSM2

  11. Definitions: variance 28 26 29 25 30 24 27 26 28 27 28 26 25 29 27 On the right is another column. Mean is also 27. But it is much less ‘spread out’ – its variance is less. All students are getting close to the same mark. Maybe the exam is not well designed to test ability. If there are N marks, subtract the mean from each of them, square them add up the squared values then divide by N-1. 46 8 50 6 99 -42 30 23 16 38 60 -3 45 0 30 Another ‘statistic’: 1068 (left) & 2.86 (right) Measure of ‘spread’ COMP80131-SEEDSM2

  12. Definitions: std_deviation 28 26 29 25 30 24 27 26 28 27 28 26 25 29 27 46 8 50 6 99 -42 30 23 16 38 60 -3 54 0 30 This is the square root of the variance. Also a measure of ‘spread’ Yet another ‘statistic’: 32.7 (left) 1.69 (right) Many alternatives exist COMP80131-SEEDSM2

  13. Population-mean & sample-mean • Simplest statistic is probably the mean or average. • Given a table of 20 marks, average is easily found & understood. • Questions arise if we consider this batch of students to be a ‘sample’ of a much larger ‘population’ of say 1000 students taking exams. • How representative is this batch’s average, called a ‘sample-mean’, likely to be of the mean for the whole population, i.e.the ‘population mean’? • A question that arises all the time in statistical methods. • A 2nd example: if there is a population of 50 million people in the UK, we take a ‘sample’ of 1000 people, measure their heights & compute the average, how close will be this ‘sample mean’ to the true mean for the whole population? • How reliable will sample-mean be as estimate of population-mean? • Same question can be asked about other statistics, e.g.. variance. COMP80131-SEEDSM2

  14. Back to MATLAB • Divide the 1000 marks into batches & compute the sample mean for each batch. True Means: 52.2 49.2 49.7 49.6 55.7 51.0 48.4 50.7 50.8 ------------------------------------------------------------------------------ Means: 50.0 58.7 51.0 46.7 43.7 62.3 61.1 36.9 51.3 52.7 Means: 48.5 51.8 57.8 47.2 45.6 47.7 53.7 50.6 48.0 44.5 Means: 49.5 48.6 30.9 53.9 43.7 53.6 46.6 50.4 56.9 48.4 Means: 44.5 68.2 48.1 55.9 48.0 52.5 54.0 42.2 50.3 56.8 Means: 52.2 39.9 38.1 69.9 50.4 61.9 57.2 50.6 49.5 59.8 Means: 59.0 61.5 39.5 54.9 42.6 44.0 50.6 41.0 62.1 48.9 Means: 44.6 56.1 48.7 49.9 44.3 48.4 39.1 52.4 56.6 43.5 Means: 62.8 49.6 55.7 42.9 48.8 42.1 60.7 66.5 41.8 55.2 Means: 51.7 52.3 53.2 48.2 48.1 69.1 49.8 57.0 50.1 53.4 Means: 49.9 47.4 54.1 50.4 67.2 51.6 42.9 56.1 52.5 44.9 Means: 55.8 46.1 48.5 55.8 54.7 54.5 39.3 49.9 43.8 53.1 Means: 50.4 44.1 55.5 46.6 47.8 41.7 47.9 57.5 53.7 51.5 Means: 52.8 67.2 47.8 46.7 53.3 53.8 46.9 51.3 48.5 58.6 Means: 47.0 48.6 56.4 50.3 50.9 56.4 50.0 52.1 42.5 50.5 Means: 54.2 50.0 52.3 51.0 52.3 50.9 50.8 63.5 48.6 58.6 Means: 56.3 51.1 54.0 53.9 64.0 48.8 50.8 44.3 62.2 61.8 Means: 40.9 53.3 52.8 56.9 51.2 61.1 57.6 56.8 50.1 37.6 Means: 53.0 55.9 38.8 47.2 49.0 62.2 49.1 39.4 54.6 49.5 Means: 47.8 51.4 48.2 45.9 48.2 53.6 54.0 43.6 49.1 48.3 Means: 38.9 51.9 52.0 60.7 44.1 44.2 70.8 51.3 49.9 46.8 Means: 52.6 54.9 54.9 50.8 43.8 53.5 50.9 58.3 40.1 48.9 Means: 52.5 68.1 53.3 46.1 60.1 53.4 52.0 48.3 51.5 55.5 Means: 60.0 45.7 45.5 45.7 50.5 51.8 44.8 50.1 54.2 65.9 Sample means for 50 batches of 20 Look at col 1 (Engl) COMP80131-SEEDSM2

  15. 50 batches of 20 (column 1) Look at spread over all batches for column 1 Remember pop-mean  52.2 Mean (of sample-means) =52.2 Variance = 32 COMP80131-SEEDSM2

  16. 20 batches of 50 (column 1) Variance has reduced. Mean of sample-means = 52.2 Variance = 18.2 COMP80131-SEEDSM2

  17. 10 batches of 100 Mean of sample-means = 52.2 Variance = 7.28 COMP80131-SEEDSM2

  18. Distributions • Histogram divides domain (x-axis) into say 10 or 20 regions & plots the number of marks that fall in each region. • In MATLAB: • figure(1); hist(Marks(:,1),20); • figure(2); hist(Marks(:,2),20); • figure(3); hist(Marks(:,3),20); etc. COMP80131-SEEDSM2

  19. Histogram for col 1 (English) Evenly distributed across the domain. Looks like a ‘uniform’ distribution COMP80131-SEEDSM2

  20. Histogram for col 2 (Maths) Looks a bit ‘Gaussian’ or ‘normal’ Mean  50 COMP80131-SEEDSM2

  21. Histogram for col 3 (Phys) Also looks ‘Gaussian’ Mean  50 with smaller variance COMP80131-SEEDSM2

  22. Histogram for col 4 (Chem) Bi-modal distribution COMP80131-SEEDSM2

  23. Column 5(Hist) A bit strange COMP80131-SEEDSM2

  24. Col 6 (French) Uniform again? COMP80131-SEEDSM2

  25. Column 7 (Music) Gaussian again? COMP80131-SEEDSM2

  26. Col 8 (Art) Gaussian again? COMP80131-SEEDSM2

  27. Col 9 (Average) Gaussian? COMP80131-SEEDSM2

  28. Some questions for you • Analyse the ficticious exam results & comment on features. • Compute means, stds & vars for each subject & histograms for the distributions. • Make observations about performance in each subject & overall • Do marks support the hypothesis that people good at Music are also good at Maths? • Do they support the hypothesis that people good at English are also good at French? • Do they support the hypothesis that people good at Art are also good at Maths? • If you have access to only 50 rows of this data, investigate the same hypotheses • What conclusions could you draw, and with what degree of certainty? COMP80131-SEEDSM2

  29. Correlation • Measure of how two columns are related. • Let cols be x and y: • Correlation coefficient: COMP80131-SEEDSM2

  30. Scatter plot col 1 against col 1 Corr coeff = 1 Positive correlation COMP80131-SEEDSM2

  31. Scatter plot col 1 against -col 1 Corr-coeff = -1 Negative correlation COMP80131-SEEDSM2

  32. Scatter plot col 1(Eng) – col 2(Maths) Corr coeff = 0.04 (close to zero) Very weak or no correlation COMP80131-SEEDSM2

  33. Scatter plot col 2(Maths) – col 7(Mus) Corr coeff = 0.8 (strong +ve corr) COMP80131-SEEDSM2

  34. Scatter plot col 2(Maths) – col 8(Art) Corr coeff = -0.8 Strong –ve correlation COMP80131-SEEDSM2

  35. Correlation In MATLAB: corr(Marks) 1.00 -0.037 -0.029 -0.068 -0.04 0.012 -0.015 0.013 0.34 -0.037 1.00 -0.0014 0.051 -0.033 0.003 0.79 -0.82 0.365 -0.029 -0.0014 1.00 -0.042 0.03 0.009 0.017 0.011 0.15 -0.068 0.051 -0.042 1.00 -0.013 -0.055 0.048 -0.031 0.42 -0.04 -0.033 0.03 -0.013 1.00 -0.053 0.002 -0.006 0.43 0.012 0.003 0.009 -0.055 -0.053 1.00 -0.004 -0.009 0.363 -0.015 0.79 0.017 0.0476 0.0021 -0.004 1.00-0.66 0.48 0.013 -0.82 0.011 -0.031 -0.0061 -0.009 -0.661.00 -0.16 0.34 0.37 0.15 0.42 0.43 0.363 0.48 -0.16 1.00 COMP80131-SEEDSM2

More Related