EDF 5807 week 5

1. EDF 5807 week 5 The many facets of assessment

2. Assessment is a very complex and problematic issue, far more complex and problematic than is often realized. There is a lot of 'science' about assessment.

3. Roles of Assessment • Summative • Occurs at the end of the unit • To produce a final score or grade • Formative • Occurs during the unit • Teacher and students can check ongoing progress and improve identified weaknesses • Diagnostic • Occurs before the unit • Determine prior knowledge/skills • Determine range of students’ ideas and beliefs

4. Shifts in thinking about assessment • Assessment of learning • Assessment for learning • Assessment as learning

5. Behaviourism • Dominant paradigm prior to the 1980s • Curricula organised into specific skills • Each skill was expressed as a behavioural objective • Precise measurement was used to ensure each skill was assessed

6. Assessment OF learning • from a measurement paradigm that is fairly behaviourist • largely summative, diagnostic assessment of entry skills is also assessment of learning • Still frequently needed/appropriate

7. Assessment FOR learning Emerged from the learning paradigm • the assessment process will allow the student to improve their understandings/skills • largely formative assessments, including self/peer assessment • Collecting the range of students ideas and beliefs is assessment for learning • involves consequential validity - by doing the tasks, students learn

8. Assessment AS learning Emerged from the authentic curriculum learning paradigm • New learning is expected as (not after) the ‘assessment’ task is undertaken • Problem Based Learning (PBL) projects and AUSVELS Interdisciplinary Tasks are examples

9. Trustworthiness in evidence of learning • Validity (accuracy & fairness) • Reliability (reproducibility)

10. Assessment is far less valid and far less reliable than is generally realized. • There are often serious problems of validity in large scale standardised tests that cause concerns about the political influence of their results. • The designers of such tests are not incompetent; as assessment moves from 1:1, to whole class, whole year level, state wide, nation wide, and international (and hence cross cultural) it commonly becomes much narrower (often solely based on tests) and increasingly difficult to assess validly.

11. Validity An assessment is valid if … • what it assesses is appropriate. • it provides information which is useful for some valuable purpose. • it is assessed with sufficient accuracy.

12. Construct validity Are you measuring what you say you have been trying to teach? You don’t: • measure the ability to design an effective web page by testing using individual bits of the software design package • measure ability to play a piano by a written test on music theory • measure the ability to analyse an argumentative piece of writing by asking students to reproduce your analysis • measure understanding of a complex idea like energy by calling for a ‘correct’ definition

13. Reliability • The reliability of an assessment task is the consistency with which it assesses whatever it assesses. • A task could be very reliable, but not valid I could reliably measure the heights of a netball team, but this would not be a valid measure of their value to the team.

14. Reliability -student How confident can you be that judgements that you have made are not significantly affected by chance factors such as: • how the student was feeling on the day? • luck in being assessed on some things rather than others? • luck in that the mode of assessment suited the student particularly well (or badly?)

15. Reliability -assessor • If another assessor judged the assessment task, would the student be awarded the same result? (inter-rater reliability) • If the same assessor judged the work on another day would the result be the same? (intra-rater reliability). • This type of reliability is particularly important with criterion-referenced assessment.

16. The big picture • Assessment, even at the classroom level requires careful thinking about what you have been trying to teach and (multiple) ways that you could assess for learning. There are a range of issues associated with both setting and interpreting the outcomes of assessment. • Skilful assessment is a crucial feature of good classrooms

17. Assessment and learning • Assessment has a major influence on how students learn • Therefore it is important to use tasks that stimulate/require quality learning

18. Returning to the discussion Four useful statements For each of these statements, explain why Vera or Joanne said it and then discuss to what extent you now agree or disagree with it and why • Vera said Sugar is bad for you • Jo-Anne said No, it is not, it gives you energy • Jo-Anne later said That’s why we eat food [to get energy] • Vera then said If that was true then you should be able to live on water and sugar alone, and I don’t think you could

19. Four aspects of quality learning • Checks personal understanding • Links school to outside life • Retrieves and links schoolwork to relevant prior views • Restructures existing ideas as needed Sean in Using Concept maps as a before and after

22. Metacognition –a key aspect of quality learning • Cognition is knowing, metacognition is knowing about your own knowing • Metacognitive learners have : -knowledge of what good learning involves -awareness of their own learning and so can -control their own learning

23. The Non metacognitive learner • Sees school as a series of unrelated tasks • Does not recognise that there are key ideas and skills that provide purposes for tasks or that it is worth thinking about this • Sees assessment as something teachers do to students • Hence the need to Move assessment into the process of learning

24. Using summative assessment formatively This sounds straightforward, but is often one of the most difficult lessons to try and run: you are asking students to revisit and wallow in what they see as pain and shame You need students having a metacognitive understanding of how this can be useful

25. Aspects of quality learning that can be stimulated or required by assessment Processing • Looks for key ideas • Checks personal understanding • Builds the richest possible meaning for a piece of content

27. Aspects of quality learning that can be stimulated or required by assessment Planning/Preparing • Plans or reflects on the overall strategy –does not just dive in • Checks the purpose of the task • Considers the reasons behind suggested actions • Considers alternative approaches

28. Procedure F25 Preliminary thinking sheets Variety matters; do not ask students the same sorts of questions each time. Look for opportunities to get students to work out part or all of the method, what they need to measure accurately, what are possible results, how data will be treated, links to relevant theory, to earlier practical activities or to outside life, what conclusions could be drawn from particular results, however do not ask all of these every time.

29. Aspects of quality learning that can be stimulated or required by assessment Decision making • About what to do or explore including the appropriate level of challenge • About how to manage their time • About how to present work

30. A smorgasbord

31. A one point question

32. A three point question

33. Aspects of quality learning that can be stimulated or required by assessment Risk taking • Takes risks in creative tasks • Makes a prediction or informed guess Working collaboratively • Engages in collaborative planning and working in small groups

34. Creative writing in science Paula Protein, Vera Vitamin, Carol Carbohydrate, Fanny Fat and Fred Fibre are all in a ham sandwich that you eat. Describe what happens to each of these characters in a creative story that moves from the mouth to the anus • Splash! The gastric juices came squirting in. Paula started screaming in pain - they were acid and were attacking and dissolving her. The others watched in horror, they didn’t seem so affected but Paula was breaking up into tiny pieces before their eyes. ... • ...Fred lay alone, bruised and battered on the floor of the intestine - oh no! It was happening again! The walls closed in on him, squeezing and crushing him and pushing him further along. "I hate peristalsis," he groaned.

35. Big Ideas • Assessment has a major influence on how students learn • Therefore it is important to use tasks that stimulate/require quality learning

36. A few comments on skills of assessment

37. One common flaw The task can be done/item responded to in ways different from what you intended/wanted but which are completely defensible in terms of the way the task is worded

39. Aboriginals are ___________ [nomads]

40. Write brief notes on… -The synthesis of ammonia -Mendeleev and the development of the periodic table

41. Better The synthesis of ammonia involves a trade off between two chemical principles. Discuss

42. Better How would the design of the industrial production of ammonia change if the reaction between nitrogen and hydrogen was endothermic [this reverses one of the two competing principles so both now operate in the same direction]

43. Spot the error: Who were the first explorers of Australia? A. the Dutch B. Dampier C. the French D. British

44. Item with more than one correct answer The Prime Minister of Australia is A. elected directly by the electorate. B. elected by the Members of Parliament in the governing party or parties. C. elected by the Parliament. D. appointed by the Governor-General.

45. Misconceptions about multiple-choice tests Misconception Multiple-choice tests can only measure low-level outcomes such as knowledge of facts, names and places Fact In the hands of a skilled writer, multiple-choice tests can assess complex reasoning efficiently and well

46. Bloom application level

47. Bloom analysis level

48. Big idea There is a rich range of skills in designing assessment that is valid and not technically flawed