Oxford Brookes University EdD Colloquium 29 th June 2013

1. Creativity, Design and the Role of Cognitive Ability Tests: a small scale studyAdrian Twissell MA Oxford Brookes University EdD Colloquium 29th June 2013

2. Abstract Abstract This paper develops previous work by the author surrounding the efficacy of cognitive ability tests (CATs) in the identification of subject specific abilities in Technology education. It records the results of a small scale study which aimed to evaluate the utility of the component parts of the Yellis (Year Eleven Information System) test as an indicator of creativity and design ability within the context of electronics education. The study compares three Year 10 electronic products groups from three years (Class of 2012, 2013 and 2014), their Yellis scores and performance on a Design and Technology specific design question similar to that employed in many public GCSE Design and Technology examination papers. Data comparison uses Pearson’s ‘r’ product moment correlation coefficient. The main findings support previous research which questions the predictive validity of CAT test scores in revealing subject specific abilities and suggests, alternatively, that they reflect an individual’s general intelligence (‘g’).Results also indicate a significant correlation between performance on the Patterns test component (Yellis) and drawing ability and similarly between the Patterns component and creativity. The paper concludes that CAT tests and subject specific outcomes reflect different ability measures and do not necessarily allow a one-to-one correspondence, or predict success for some subjects. Key Terms: creativity, design, cognitive ability, YELLIS test, technology education

3. Cognitive Ability Tests (CAT) • Vocabulary Score The word that means about the same as ‘hat’ is? book  cap  pencil  road  • Maths Score Work out the following: (2x10)+1= • Patterns Score Look at the symbols or pattern on the left Circle the matching symbols or pattern on the right Defty(2008)

4. Yellis Test • Yellis (Year Eleven Information System) taken at age 15 (Year 10) • Used to set performance targets across the subjects in schools • Targets set for students and teachers

5. The Context • Previous research found some correlation between CAT scores (Yellis) and D&T specific abilities, probably in relation to ‘test-taking’ or ’lesson-learning’ abilities (Renzulli and Reis, 2008: 16),rather than subject ability. • Previous research found some correlation between CAT scores (MidYIS Non-Verbal) and D&T gifted/talented students • Negligible predictive validity found by others between CAT scores (NFER, 1986 CAT test) & non-core subjects (Moody, 2001)

6. Previous Research Twissell (2011)

7. Previous Research Twissell (2011)

8. Design & Creativity: Definitions • Creativity: The work is novel ‘Creative work is original and somehow distinctive with respect to the work with which it is compared’ Sternberg and Kaufman (2010, 467) • Drawing ability Line work is accurate and may follow a recognisable standard (i.e. isometric, planometric, orthographic) • Technological Literacy Technical terms describe drawn elements/design decisions/materials/manufacturing processes

9. Do High Yellis Scores=Good Design/Creative Ability? Top 28% of 36 GCSE Electronics Students based on Yellis Score 100=national average Max 5 points

10. Research Questions • What is the predictive validity of the Yellis test score and/or its components, for Year 10 students’ success in completing an Electronics specific design question? • What is the predictive validity of the Patterns score (Yellis), in determining specific Design & Technology abilities (e.g. creativity, drawing ability and technological literacy)?

11. The Design Question You have been asked to design the casing for a digital dice The specification for the casing for the digital dice is that it must: • include an LED display for 6 (or 7) LEDs • be capable of switching on/off • include a ‘roll’ function • have clearly labelled functions • Include ‘graphics’ clearly indicating the product function • use components commonly found in the school store • use assembly techniques available in the school electronics lab • be capable of replacing the batteries easily In the spaces opposite, use sketches and, where appropriate, brief notes to show twodifferent design ideas for the digital dice that meet the specification points above. Total 16 Marks (2x8).

12. Design Question:Adequately Drawn/Limited Creativity Total for Design Question=16 Creativity, drawing ability and technical terms also assessed

13. Design Question:Well Drawn/Limited Creativity

14. Good Use of Technical Terms

15. Design Question:Creative Example

16. Methodology/Method • Non-experimental fixed design ‘correlational study’ (Robson, 2011) • Use of exam board’s assessment criteria for design question scores (max 16) • Multiple years’ (2011-2013) group (Y10) comparison of design question scores using Pearson’s ‘r’ product moment correlation coefficient • Additional ‘drawing ability’, ‘creativity’ and ‘technical terms’ analysis using a 5 point rating scale (5=highest score) for each

17. Results

18. Yellis Scores/Design Question Do Yellis scores predict success on the design question? Correlation negligible Good correlation (p>0.05)

19. Yellis Scores/Drawing Accuracy What is the role of drawing accuracy? Very good correlation with Patterns Score (p=0.05)

20. Yellis Scores/Technical Terms Is there a link between Yellis scores and the use of technical terms? Some correlation with Maths Score (p<0.05)

21. Yellis Scores/Creativity Is there a link between Yellis scores and creativity? Poor correlation Very good correlation (p=0.05) Inverse relationship

22. Discussion Q1 • What is the predictive validity of the Yellis test score and/or its components, for Year 10 students’ success in completing an Electronics specific design question? Difficult to correlate Yellis Scores & Design Question scores Suggests disconnect between Yellis test/test components and design question outcome measures Yellis test components also negligible

23. Discussion Q2 • What is the predictive validity of the Patterns score (Yellis), in determining specific Design & Technology abilities (e.g. creativity, drawing ability and technological literacy)? Patterns score/design question score r= -0.042(i.e. inverse relationship) Patterns score/accuracy of drawing/creativity in direction of a reasonable correlation Suggests closer alignment between spatial awareness and drawing/creativity tasks

24. Conclusion • Yellis test and design question measure different abilities • Previous studies have indicated that CAT style tests generally reflect a student’s ‘general intelligence’ or ‘g’ (Frey and Detterman, 2004) • Previous research has indicated that other attributes, such as ‘love of work’, ‘purpose in life’ and ‘high energy levels’ affect an individual’s likelihood of creative achievements(Torrance, 2004)

25. References Defty, N. (2008) Introduction to the Yellis Project, accessed at http://www.docstoc.com/docs/2684558/introduction-to-the-yellis-project on 17th June 2013. Frey, M.C. and Detterman, K.D. (2004) Scholastic Assessment or g?: The Relationship Between the Scholastic Assessment Test and General Cognitive Ability, Psychological Science, Vol.15, No.6. Moody, I. (2001) A case-study of the predictive validity and reliability of Key Stage 2 test results, and teacher assessments, as baseline data for target-setting and value-added at Key Stage 3, The Curriculum Journal, Vol. 12, No. 1, pp81-101. Renzulli, J. and Reis, S. (2008) Enriching Curriculum For All Students, Thousand Oaks, California: Corwin. Robson, C. (2011) Real World Research (3rded), Chichester: Wiley. Sternberg, R.J. and Kaufman, J.C. (2010) Constraints on Creativity: Obvious and Not So Obvious. In Sternberg, R.J. and Kaufman, J.C. (eds) The Cambridge Handbook of Creativity, New York: Cambridge University Press, Chap 24, pp467-482. Torrance (2004) Great Expectations: Creative Achievements of the Sociometric Stars in a 30-Year Study, The Journal of Secondary Gifted Education,Vol. 16, No.1, pp5-13. Twissell, A. (2011) An investigation into the use of cognitive ability tests in the identification of gifted students in design and technology, Design and Technology Education: An International Journal, Vol. 16, No. 2, pp20-32.