Test Learn 2 ways of helping Learning

1. Test & Learn � 2 ways of helping Learning Jon Sims Williams Mike Barry Engineering Mathematics University of BRISTOL ,UK. Apologies if I appear to be teaching you professional teachers how to do your job � what I am really trying to do is to show you some tools that will help you.Apologies if I appear to be teaching you professional teachers how to do your job � what I am really trying to do is to show you some tools that will help you.

2. A Learning Cycle � Test & Learn - TAL Attend Lecture take test Or Read Book revise Take specialised test This diagram is intended to show a model of how students can be helped to learn by the use of tests. The tests show up weeknesses in student understanding, they revise or are helped with understanidng how to do that sort of question and then need to re-inforce that sort of work by a specialised test on that sub-topic. This process is repeated for each topic that the initial test showed up as a weekness. As you can see this requires lots and lots of tests. We start off with a diagnostic test when the students first arrive. What you really need to help you build a learning environment like this is a database of questions so you can easily generate small tests on any topicThis diagram is intended to show a model of how students can be helped to learn by the use of tests. The tests show up weeknesses in student understanding, they revise or are helped with understanidng how to do that sort of question and then need to re-inforce that sort of work by a specialised test on that sub-topic. This process is repeated for each topic that the initial test showed up as a weekness. As you can see this requires lots and lots of tests. We start off with a diagnostic test when the students first arrive. What you really need to help you build a learning environment like this is a database of questions so you can easily generate small tests on any topic

3. TAL � Test & Learn - www.tal.bris.ac.uk TAL is a Computer Assisted Assessment System The Core element is an Oracle Database The database stores questions and student data about 2000 questions on Mathematics Both staff and students can talk to TAL from a web browser from home or work Say the name TAL so that they recognise it on the next slide Explain that databases are wonderful for storing lots of information but you can also lose information in them unless it is well organised and the data is CLASSIFIEDSay the name TAL so that they recognise it on the next slide Explain that databases are wonderful for storing lots of information but you can also lose information in them unless it is well organised and the data is CLASSIFIED

4. Delivery of Support through Tests Tests provide students with encouragement Tests tell students where they do not understand Tests can provide feedback on questions to help students who don�t understand TAL will build sets of tests that are all on the same topics; take about the same length of time to do and are about of equal difficulty We have said that students need to be encouraged to know what they don�t understand and to have help when they cannot understand. Tests can help students with all three but they are more sceptical about the help that the tests give them. Note that the test tells students WHERE they don�t understand but not WHAT they don�t understand, nevertheless it is a motivator to do something about improving their understanding. We give them very few marks for doing well in these tests. 1% if they score above 60% on the best test in a set of tests, but the benchmark is there that they should try to score 60%. Very important to go over this slide again summarising that we support students by providing sets of tests that they can test themselves against. This is in contrast to the thesis that we could provide students with the means to test themselves ie generate their own tests. We have said that students need to be encouraged to know what they don�t understand and to have help when they cannot understand. Tests can help students with all three but they are more sceptical about the help that the tests give them. Note that the test tells students WHERE they don�t understand but not WHAT they don�t understand, nevertheless it is a motivator to do something about improving their understanding. We give them very few marks for doing well in these tests. 1% if they score above 60% on the best test in a set of tests, but the benchmark is there that they should try to score 60%. Very important to go over this slide again summarising that we support students by providing sets of tests that they can test themselves against. This is in contrast to the thesis that we could provide students with the means to test themselves ie generate their own tests.

5. Example Multiple Choice Question Question : Given x, y, z are real quantities and a, b integer powers, show that the indices in the identity (xayz-1)b(2yz2)2(x/y)a = 4(x6z)2 must be one of the following. Specify which is correct. Answer1: a = 4, b = 2 Feedback: Correct. Answer 2: a = 2, b = 4 Feedback: No. Multiply out and then equate powers of each variable x , y and z in turn Answer 3: a = 3, b = 2 �� TAL Classification: mathematics/Algebra/ Equating powers in identities Facility: 50% Time-to-do: 90 seconds Theory: 3 Function Type: Simple algebraic functions Evaluationtype: Numerical This question illustrates the idea of classifying questions in the database Each student group has its own facility and time-to-do We can also point out that the feedback for the wrong answer should help the student � although they complain that the feedbacks are not good. To improve the feedback we can provide links to web-based support materials that are classified in the same way as the question. Explain the the time-to-do and facility can be evaluated from data on how the students actually do on each question and this data can be different for each class using the system.This question illustrates the idea of classifying questions in the database Each student group has its own facility and time-to-do We can also point out that the feedback for the wrong answer should help the student � although they complain that the feedbacks are not good. To improve the feedback we can provide links to web-based support materials that are classified in the same way as the question. Explain the the time-to-do and facility can be evaluated from data on how the students actually do on each question and this data can be different for each class using the system.

6. Part of Subject Classification Algebra Arithmetic Complex Algebra Differentiation Chain; product; ratio rules Differentiability Equations of lines Function of function rule �.. Fourier Series Functions Etc. MathematicsWeb [3] , Zentralblatt MATH [4] and the 2000 Mathematics Subject Classification from the American Mathematical Society [5] etc are published classifications but none of them are suitable because they do not give the level of detail required by teachers. Questions hang from points on the classification tree � so it is easy to find all the questions about say �differentiability� If you choose a high point in the tree you get all the questions at any point in the sub-classification as well. So Differentiation will give you questions involving anything under Differentiation. This classification is only two level but you have said that the published 3 level systems are inadequate.MathematicsWeb [3] , Zentralblatt MATH [4] and the 2000 Mathematics Subject Classification from the American Mathematical Society [5] etc are published classifications but none of them are suitable because they do not give the level of detail required by teachers. Questions hang from points on the classification tree � so it is easy to find all the questions about say �differentiability� If you choose a high point in the tree you get all the questions at any point in the sub-classification as well. So Differentiation will give you questions involving anything under Differentiation. This classification is only two level but you have said that the published 3 level systems are inadequate.

7. TAL and Maths Classification Weissteins World of Maths � http://mathworld.wolfram.com/ The European Society for Engineering Education, SEFI syllabus for Core Zero http://learn.lboro.ac.uk/mwg/core.html When we started TAL we generated our own classification � but this is a huge job; so we have adopted two published clasification schemes Weisstein�s World of Maths is to support a dictionary of maths � not the same as database of questions SEFI is a syllabus definition which is appropriate at the school/University interface where most of our work takes place. People have suggested that an Alevel syllabus would be a good scheme to use for classification and there is not reason why all three schemes should not be used in the same database so that different sorts of people can use the scheme that helps them best. The same question can have multiple classifications under the different schemes.When we started TAL we generated our own classification � but this is a huge job; so we have adopted two published clasification schemes Weisstein�s World of Maths is to support a dictionary of maths � not the same as database of questions SEFI is a syllabus definition which is appropriate at the school/University interface where most of our work takes place. People have suggested that an Alevel syllabus would be a good scheme to use for classification and there is not reason why all three schemes should not be used in the same database so that different sorts of people can use the scheme that helps them best. The same question can have multiple classifications under the different schemes.

8. A small sample from the SEFI syllabus Analysis and Calculus Rates of change and differentiation Average & instantaneous rates of change Definition of derivative at point Derivative as instantaneous change rate Derivative as gradient at a point Difference between derivative & derived function Use notations: dy/dx, f(x), y etc. Note this is a hierarchical classification It also gives the sort of detail that a syllabus and a teacher will needNote this is a hierarchical classification It also gives the sort of detail that a syllabus and a teacher will need

9. A Theory of Successful Learning successful learning = Ease_of_Learning * Motivation

10. A Learning Cycle � Test & Learn - TAL Attend Lecture take test Or Read Book revise Take specialised test What you really need to help you build a learning environment like this is a database of questions so you can easily generate small tests on any topicWhat you really need to help you build a learning environment like this is a database of questions so you can easily generate small tests on any topic

11. Examples from web-link database

12. An Example MCQ

14. A Theory of Successful Learning � revisited successful learning = Motivation* Ease_of_Learning

15. Motivation for Histogram

16. The Camera�s Histogram records the number of pixels with each of 255 illumination levels, grouped as shown.

17. How does the camera display the histogram For each of the 255 illumination levels, cj that the camera can recognise it will do a test to see if the illumination level, I satisfies: cj < I < cj+1 count the number of pixels that satisfy this constraint. This then forms the height of the histogram at the point corresponding to cj So we are doing arithmetic, handling inequalities as part of the drawing the histogram.So we are doing arithmetic, handling inequalities as part of the drawing the histogram.

18. A better distribution of brightness So the motivation is to understand what histograms mean so that one can improve one�s photography getting histograms and photos nearer to this example So the motivation is to understand what histograms mean so that one can improve one�s photography getting histograms and photos nearer to this example

19. Pedagogy Support 2 Example of Motivation but suppose the students did not understand support webpage So we need to find out why. Pedagogy Support 2 is a trace of the syllabus back from any point. So we start from Histograms.

20. Trace back through syllabus Interpret data presented in the form of histograms Requires: � understand the Cartesian co-ordinate system � plot points on a graph using Cartesian co-ordinates

21. plot points on a graph using Cartesian co-ordinates requires: Algebraic expressions and formulae � add and subtract algebraic expressions and simplify the result � understand the terms direct proportion, inverse proportion and joint proportion � solve simple problems involving proportion � interpret simple inequalities in terms of intervals on the real line This is just an example of the syllabus tracer that allows one to select any topic in the syllabus and find out all the prior knowledge, defined in syllabus terms that is required.This is just an example of the syllabus tracer that allows one to select any topic in the syllabus and find out all the prior knowledge, defined in syllabus terms that is required.

22. Summary � 2 ways to support learning Automatic feedback from the question topic to a website to help the student Trace back through syllabus to find source of problem Contact: Jon.sims.williams@bris.ac.uk to use TAL Mike.Barry@bris.ac.uk for syllabus tracking The feedback may not be adequate so we need to be able to tell the student what he needs to understand before he can approach the topic. So the motivation is still available to help students learn the background materials. The feedback may not be adequate so we need to be able to tell the student what he needs to understand before he can approach the topic. So the motivation is still available to help students learn the background materials.