Applying Science Education Research to Computer Science Instruction

1. Applying Science Education Research to Computer Science Instruction Clement, J. M. (2004). A call for action (research): Applying science education research to computer science instruction. Computer Science Education, 14(4), 343-364.

2. Outline • Introduction • Relevant science education research • Piagetian thinking levels • The Piagetian tasks • Socratic dialog • Resistant misconceptions • Cycle theories • Standard tests of student understanding • Applications to computer science • The computer science curriculum at St. Pius • General principles • Specific examples of instructional activities • Evaluation of outcomes • Future possibilities • Conclusions

3. IntroductionAction research • Action research is carried out in the context of an ongoing class. • The researcher is the classroom teacher, who may have no opportunity to set up true experiments that include both a control and a treatment group. • All teachers do action research as they explore strategies and discard ideas that do not seem to work, but true action research requires that teachers be aware that they are doing research and consciously analyze results with the intention of making adjustments as the courses proceeds.

4. IntroductionTeaching methods • This paper focus on teaching methods that have merged from research in science education. • Scientific teaching must involve active learning strategies, so that students are engaged in the scientific process. • Handelsman(2004) recommend that reform must be based on the idea of “scientific teaching”. • Scientific teaching must use teaching methods that have been systematically tested and shown to reach diverse students.

5. IntroductionPER • Physics education research (PER) has enjoyed particularly impressive success in promoting student understanding of physics concepts. • This success has been built on careful testing, research, and curriculum planning by physics educators.

6. IntroductionResearch question • How results from science education research (particularly from PER) apply to computer science? • In general, science is the search for models. • Computer science has much in common with physics. • The hardware of computer science deals with physics laws. • Programming deals with constructs invented by humans. • The research in science education shows that students perform better if they act like scientists in their classes. • They must discover the rules and with the teacher’s help, create coherent mental models of physical systems.

7. Relevant Science Education ResearchPiagetian thinking levels • Children develop their intelligence through a series of qualitatively differentiated stages. • Sensorimotor, preoperaltional, concrete operational, and formal operational periods. • Lawson(1995) has proposed a new category, transitional, that describes students who use only some of the formal operational patterns of thinking. • Adults: 30%, high school students: 20% at the formal operational level. • The concrete operational students tended to use just one line of reasoning and could handle one variable thinking. • The formal operational students could handle multiple variables and equations involving three variables (Adey & Shayer, 2002).

8. Relevant Science Education ResearchThe Piagetian tasks • Piagetian tasks can be considered a measurement of the readiness of students to understand science and mathematics (Arrons, 1997). • Concrete operational thinking • Conservation (e.g. a ball of clay) • Control of variables (e.g. the long and short strings with identical weight bobs) • Formal operational thinking • Proportional reasoning (e.g. water is measured in a fat cylinder and predict the level when it poured into a thin one) • Two variable reasoning (e.g. four measurements and figure out which of two variables is important) • Probabilistic thinking (e.g. probability) • Sequential reasoning (e.g. count possible combinations of four buttons that must be pushed to light a bulb) • Correlational reasoning (e.g. the size of fish and the width of its stripes) • Lawson Classroom Test of Scientific Reasoning (Lawson, 1995) • Concrete, transitional, and formal level. • Give the correct answer and explain it properly.

9. Relevant Science Education ResearchScoratic Dialog • In Scoratic teaching, the instructor focuses on giving students questions rather than answers. • In physics, Hake’s(1992) Socratic Dialog-Inducing (SDI) labs have been demonstrated very effective in studies based on rigorous pre-post testing.

10. Relevant Science Education ResearchResistant Misconceptions • Some misconceptions come from students’ interactions with the world around them, while others are influenced by students’ previous learning in school. • E.g. Newton’s third law: forces exerted by two colliding objects must be equal. • Instruction must either change the student’s paradigm or convert the student’s way of thinking. • Adey and Shayer(2002) report empirical evidence that techniques designed to promote accommodation are effective in treating resistant Piagetian task misconceptions. • PS: In assimilation, the learner fits information into an existing framework, while in accommodation the learner modifies thinking patterns to make sense of discrepant information.

11. Relevant Science Education ResearchCycle Theories • Two cycle theories have come into widespread use in PER. • Karplus learning cycle (Rober Karplus) • Three phases: exploration, term definition and application. • Karplus cycle can improve content comprehension and raise student thinking level (Lawson, 2001). • The order is vital in order to optimize results with concrete operational students (Renner, Abraham, & Birnie, 1988). • Predict-confront-resolve cycle (McDermott) • Predict the outcome of experiments, then experience the results and finally resolve any discrepancies. • The Interactive Lecture Demonstrations curriculum( ILD) has very strong evidence for the pedagogical effectiveness (Thornton & Sokoloff, 1998). • Modeling cycle (Wells, Hestenes, & Swackhamner) • A variant of the Karplus learning cycle • Breaks the curriculum up into specific models and concentrates on helping students understand just one model at a time.

12. Relevant Science Education ResearchStandard Tests of Student Understanding • The biggest changes in physics education resulted from the introduction of standard tests of student understanding. • Force Concept Inventory ( Hestenes, 1992) • Normalized gain depends only on the teaching style and not on student’s previous knowledge.

13. Application to Computer ScienceThe computer science curriculum at St. Pius • At St. Pius, all computer science coursed involve teaching programming. • The school is switching from C++ to Java. • The challenge is to help students think logically and understand the basic ideas. • Once they have learned the basic ideas, students easily can master the details. • To accomplish this, the early part of the curriculum has been stripped of most details.

14. Application to Computer ScienceThe computer science curriculum at St. Pius • The first semester curriculum covers the fundamentals of variables, assignment and simple calculations, if-then-else statements, loops, and use of functions. • To avoid distracting students from the main ideas, they learn only enough formatting to print out the results of their calculations. • The second semester is a separate course taken by about 5-15% of the first-semester students. • By the end of the second course, students should be able to write programs and make sense of the textbook. • Instruction in these courses concentrate on getting students to figure out the answers themselves. • The Karplus cycle is the heart of this approach. • Enter and compile a piece code, answer questions about the code, and verify their answers by experimenting. • Students either read the relevant explanation in the text or listen to a brief lecture by the instructor. • Students design programs that use the new concept. • Because students work more or less independently, the teacher can engage them in Socratic dialog and readily assess which students most need help.

15. Application to Computer ScienceGeneral Principles • Pre-college students tended to have more difficulty with the concepts of propositional logic than with other computer science concepts (Lawson, Karplus, & Adi, 1978;Almstrum,1996). • Karplus should be helpful in teaching propositional logic. • Explicit direct conceptual instruction occurs during the second Karplus phase. • A good alternative to lectures is to have students exchange answers on individual white boards and do most of the questioning (Wells, 1995). • Students are required to read the book during the second phase of the Karplus learning cycle, after they have already done sufficient exploration that the book will be understandable (Leonard et al., 1999).

16. Application to Computer ScienceGeneral Principles • Socratic dialog allows an instructor to become very aware of students’ problems and misconceptions. • One of the most visible problems is the inability of concrete operational students to engage in metacognition. • They may be able to solve many simple problems, but they cannot express how they found the solution. • Once they can explain the sequence of operations, they find it easy to code the problem. • The instructor must ask students to sketch the problem on paper, and it can be helpful for students to do the calculation with a calculator, then write down what they did. • Experience with research-based teaching in science can sensitize the instructor to the fact that the students need to use multiple representations to form their ideas (Wells, 1995).

17. Application to Computer ScienceGeneral Principles • Along with the inability to engage in metacognition comes the inability to decode text. • The instructor must guide the students’ reading with Socratic dialog.

18. Application to Computer ScienceGeneral Principles • Science research experience is also helpful when tackling complex problems, because the instructor is able to devise manipulatives that help students understand the concepts. • For example, students are capable of devising sorting algorithms by themselves if the problem is presentedusing manipulatives. • One can create a set of small numbered pieces of paper and a labeled representation of the array elements (either another sheet of paper or a set of boxes). • As they sort, students must realize that once a piece of paper is put on top of another piece, the bottom piece has been discarded. • By using Socratic dialog and these manipulatives, students readily devise a simple and workable strategy.

19. Application to Computer ScienceSpecific Examples of Instructional Activities • Teach the binary system and variables. • The programme at St. Pius introduces the general idea of counting through the use of manipulatives, simple counters that consist of three wheels labeled with the relevant set of digits around the edges. • Students use the base-10 counter (labeled with the digits 0–9 on each wheel) to count and describe the steps they follow in turning the wheels to count. • The instructor collects the students’ instructions and follows the steps to count on the wheels. When the instructions are incorrect or incomplete, the instructor innocently and cheerfully follows the wrong steps as described, which helps students better understand the full extent of each step. For example, students often fail to convey that the next higher digit can be advanced only when the lower one ‘‘goes to zero’’. • Next, students count using a similar set of octal counters (labeled with the digits 0–7 on each wheel).

20. Application to Computer ScienceSpecific Examples of Instructional Activities • The students must create a table of equivalents between octal and decimal numbers. Along the way they must predict various equivalents and then count up to that number in order to verify the prediction. For example, after counting up to 50 on the decimal counter, they must then predict the decimal equivalent to octal 100. Students invariably answer 80 and are surprised to discover that the correct answer is 64. • Next, students make rules for octal counting and compare these to the rules they have already developed for decimal counting. • In the concluding phase, students go through the same activities using a binary counter (digits 0–1 on each wheel). • After their initial explorations, students must develop and then apply rules for converting from binary to decimal. • This sequence, using the Karplus learning cycle, bridges from ideas that students already know to less familiar situations, allowing them to see the ideas in very concrete contexts. • Challenge students to go beyond their comfort zones.

21. Application to Computer ScienceSpecific Examples of Instructional Activities • Once students have completed the binary exploration, they are ready to learn about limitations on variables. • Define a variable, read a number from the keyboard, and then output the number. • Find the maximum, minimum, and smallest non-zero number that can be entered, as well as whether or not the variable can handle decimal points. • Do this for various types of variables and discover the limitations. • Connect some of these limitations to the number of bits in a binary representation. • Socratic teaching and the predict-confront-resolve cycle can be used together as students try to discover the largest and smallest numbers that can be represented on their computers. • For many students, this is a very difficult task because they lack number sense.

22. Application to Computer ScienceSpecific Examples of Instructional Activities • The predict-confront-resolve cycle can be used productively at all stages of computer science. • An activity to guide students in discovering the meaning of all of the standard operations in C++ proceeds as follows, guided by a prepared worksheet: • Create a square table showing the results of A <=B for all values of A or B from 0 to 4. Look at the results and try to figure out what the operation is doing. • Repeat for other operations (>=,==, <, >, etc.). • The most challenging of the operations is generally the modulus operator. • The instructor must lead the students to try a whole variety of numbers in some organized fashion. Some students will quickly discover that it is the remainder, but concrete operational students usually take much longer. • Term definition stage: students read the section of the text that explains the terms. • Later, when the logical operators are introduced in a similar worksheet, the tables become vital to building an understanding of the logic. • Throughout this process, the teacher uses Socratic dialog and reminds students to write their observations out on paper and make their thinking visible.

23. Evaluation of Outcomes • Anecdotal evidence • Shortly after this programme was first implemented one student commented, ‘‘It is seventh period so I have to start thinking again.’’ When I asked what he meant, he said that he did intense thinking during my first period physics class, but did not have to think much until he returned to my classroom for his seventh period computer science class. That student is now pursuing a degree in computer science. • Several students have commented that while taking the SAT math test for the second time they thought about the ways we solved computer science problems. They reported that they were able to solve many more problems and increased their scores by up to 100 points (a significant improvement). • Other students have commented that they feel they can think better after the course, indicating that their thinking ability has been sharpened.

24. Evaluation of Outcomes • Beyond anecdotal evidence, the work that students do on their own shows evidence of improved thinking. • At the beginning of the first course some students attempt to carry out calculations before getting input and are usually surprised when they get wrong answers. After questioning and role playing, most of these students come to realize that operations must be done in the correct order. • The final assignment, in which they use functions from the Microsoft Foundation Class Library to draw animated pictures, strengthens their understanding of the importance of carrying out operations in the correct order. Students quickly discover that each component must be drawn in the correct order or details will disappear behind other objects. • A side-benefit of this assignment is that the required use of X–Y coordinates helps their understanding of coordinates in math.

25. Future Possibilities • Beyond the fairly large ideas addressed here, many other promising ideas may be adaptable for use in teaching computing, for example, the need for multiple representations, rich context problem-solving, ranking tasks, and whiteboarding. These ideas all have been used in various science curricula and are associated with greater student understanding of concepts and improved problem-solving ability. • The computer science education community could develop one or more standard tests, a computer science concept inventory, to evaluate student understanding of basic concepts. • Is it possible to develop a concept inventory test in a field like computer science? • The real challenge would be to identify the important concepts independently and then devise appropriate tests. • A possible starting point for a computer science concept inventory (CSCI) is the Propositional Logic Test (PLT) (Almstrum, 1999). • Both hardware and software aspects of computer science rely on students’ ability to understand propositional logic. • The CSCI must be worded using everyday examples. • Specific computer terminology must be avoided whenever possible.

26. Future Possibilities • The available textbooks do not provide an appropriate level of support. • Generally written by individuals who are formal operational, textbooks tend to begin with formal definitions of terms and ideas • Muscheno and Lawson (1999) have shown that prose written in the style of the Karplus learning cycle can significantly improve comprehension. • In their study, students read a text passage written either as a Karplus learning cycle or in the traditional style. • Both passages used the same words and had the same sentence structure. • The only differences were in the order of the information and the addition of a question at the beginning of the Karplus learning cycle version. • Concrete operational students scored 69% on the traditional text, but 88% after the Karplus learning cycle text, and scores for formal operational students rose from 78% to 92%. • Such textbooks also should use multiple representations and suggest manipulatives as appropriate to help students learn the concepts.

27. Conclusions • What are the goals of a good high school computer science course? • The Advanced Placement progamme has tended to drive American high school computer science courses toward a career preparation model, despite evidence that most students who take high school computer science will never write another program. • A much better goal in the general computing course is to improve students’ thinking ability, with only enough career preparation to allow some students to continue their studies in the field.

28. Conclusions • I have observed the following outcomes for students enrolled in computer science at St. Pius. • Improved ability to read expository text. • Better understanding of propositional logic. • Better understanding of recursion and it applications. • Improved understanding of X–Y coordinates and two-variable logic. • Better planning in the process of general problem solving. • Improved understanding of the concept of variable. • Better sequential thinking.

29. Conclusions • The ideas and teaching concepts presented in this paper may be controversial among computer science teachers. • The recent article Scientific Teaching by Handelsman and others (2004) spotlights some of this controversy and points out that ‘‘Many scientists are still unaware of the data and analyses that demonstrate the effectiveness of active learning techniques’’. • While the reformed methods have not been universally accepted by all educators in the physics or science communities, they are accepted and promoted by major organizations such as the American Association for the Advancement of Science (www.aaas.org/) and the American Association of Physics Teachers (www.aapt.org/).

30. Conclusions • Wycoff (2000) points out that there are currently no research centers for computer science education. • Perhaps some of the success in science community building can be duplicated in computer science, and both can learn from each other (e.g. Arizona ACEPT project). • A decade is a reasonable estimate of the minimum time required to change the undergraduate science teaching culture in a university. • Teaching by lecture rather than interactive engagement may be among the significant factors limiting the quality of science education in this nation.

31. Thank you Have a nice day.