Download
1 / 63
630 likes | 782 Views
History, Current Practice and Perspectives, and Predictions for the Future of Computer Based Assessment in K-12 Education. John Poggio University of Kansas. (Ancient) History circa… the 1970s, ’80s & ’90s….
E N D
History, Current Practice and Perspectives, and Predictions for the Future of Computer Based Assessment in K-12 Education John Poggio University of Kansas
(Ancient) History circa… the 1970s, ’80s & ’90s… • Impersonal, frightens students, computer anxiety, lack of familiarity, equivalence to P&P at best, • “a dramatically different experience that guarantees the two modalities are not equivalent” • “CAT will restrict problem solving and only support solutions in a strict linear manner” … • However, study as early as 1972 through the turn of the century research found no differences…
Advantages • Immediacy of results Among the most notable features associated with computerized testing is the immediacy of results. In most systems as soon as a student finishes taking the test, scores are ready for display. Delays and slowness in reporting results can be a concern of the past.
Advantages • Cost efficiency Moving tests to computer delivery will signal a reduction in costs. While upon first consideration this may not seem obvious, production of test booklets and answer sheets are extremely large expenditure items that can be set aside.
Advantages • Accuracy in data and data collection Capturing information from students on answer sheets is among the biggest NEGATIVES of the testing process. A casual review of information gathered on answer sheets reveals the considerable number of errors that students make. Answers to questions and the information provided, such as gender, race, age, and even name are frequently miscoded and mis-marked by students. Computer-based assessment all but eliminates this error in the measurement process.
Advantages • Student motivation A number of researchers and field practitioners have commented on heightened level of student motivation when assessments are taken online. This in turn translates into higher performance. Whether such motivation will be seen over time is not clear. However, at this time, students appear more attentive, comfortable, and responsive to computer rather than paper and pencil assessment.
Advantages • Adaptive alternative Moving traditional tests to computer administration has opened the door to computer adaptive testing. This approach has the advantage of reducing traditional tests by half to as much as 75% of the original length. In this methodology, while we have considerable reduction in test length, there is relatively little loss in test properties as validity and reliability (…and, assessments may be fairer!)
Advantages • Reduced administrator and instructor effort As the tests will be administered online, the traditional activities by administrators and teachers receiving, unpacking, securing, counting, sorting, and distributing test booklets and answer sheets is eliminated. This management effort that requires considerable commitment of time and attention is all but eliminated when we go to computerized testing.
Advantages • Meeting the needs of special populations Technological advances used with computerized testing signals the dawn of a new era in testing for students with special learning needs and other populations (non native speakers). Largely unstandardized accommodations of the past can now be transformed into common and fixed procedures, for example, increasing text size with the click of a button, transforming text to speech, zooming, colorizing objects, English/native language translations, ASL avatars for the hearing impaired, etc.
Advantages • Ready support for data based decision making The speed with which data can come available to educators allows rapid analysis and evaluation of performance. Whereas in the past it could take three months for data to be returned to administrators and teachers, today’s computerized tests allow educators to review results and while the information is fresh and current make instructional and evaluative decisions on behalf of students.
Advantages • Store information, ready access to results, and create archives As students take a test on a computer, the storage and retrieval of information is efficient and speedy. In the past reliance on answer sheets meant data cleanup, scanning and processing, scoring and check up, and eventual readying of information for storage and retrieval. But with computer-based testing, the storage of information is part of the process and access of that information is efficient and accurate.
Advantages • Ability to modify tests as necessary In the past, when an error was discovered in a test booklet, the educator had little recourse but to stand by and watch the mistake time after time during administration. In today's computerized tests when an error is found it can be corrected immediately. In this way, test scores are improved for many students.
Advantages • Improved security … some risks As tests are delivered online, traditional handling of test materials is eliminated. Breaches of test security are fewer. There are threats related to hacking that are not avoided. However, up to this time there have been few reports of security issues because of computerized administration.
Advantages • Increased opportunity for collection of supplemental information or data Computerized testing allows for and supports the collection of additional information. Students can be quickly and easily surveyed or additional data can be gathered. Research and evaluation can have a place in assessment once more!
Advantages • Equal (..Stronger?) Performance Research of the past 25 years is clear and certain: paper and pencil testing yields scores equivalent to computer-based testing. We should continue to study these effects. But, the real effect to watch is, with student gaining CBT experience, will CBT scores become stronger than P&P performance?
Disadvantages:Implementation Barriers • Local resistance to change Change can come very hard or very slow for some. This new technology brings with it a demand for new skills and understandings that must be mastered. In addition, some will see no need to change from the way that testing has been done. Careful, thoughtful steps must be planned to assist individuals make the transition. Without assistance and help, change will be very slow for a few. Individuals must be brought into the new technology and made to feel comfortable and be supported.
Disadvantages • Local capacity A school or school district may not have the wherewithal to move to online testing. Equipment may not be current or nonexistent, or local staff may not have the expertise to venture into this arena. The inability to move forward with the innovation will in its own way limit adoption and understanding. Fear of this unknown needs your deliberate attention.
Disadvantages • Risk – Not changing horses… These are high stakes tests and failure may not be tolerated. Careers are on the line and shifting to a new mode of testing may not be seen as wise. Again, awareness and sensitivity are important considerations; plans should be prepared with these factors in mind.
Disadvantages • Minimize or eliminate obstacles Perceived impediments to implementation of computerized testing will add to frustration and dissatisfaction. When systems are new to the teacher and testing is pressing forward with the threat of accountability, tolerance for the unexpected will be very low. Make the systems: work, understandable and forgiving.
Keys to Future Success • Involve the (state, district and school) technology staff Schools must support such personnel, and they must become involved from the outset. See to it that they work closely with educators, and that educators are comfortable reaching out to them for assistance and support. Remember: involve and train all!
Keys to Future Success • Training workshops Plan training sessions periodically, but frequently, during the semester - as you have staff, money and time. Get teachers working at the same grade and across grades solving problems together with technology staff – together! The sessions can be brief - limited to one or two hours each.
What of Staffing? • Change in roles and needs... • From P&P staff to... • Programmers and web designers, and • Office staff who are computer literate and savvy, ...and who have patience and understanding of others who are lost, truly lost...
Keys to Future Success • Training materials Development and utilize student, parent, and teacher/administrator tutorials, student training and practice tests, and implementation guides for teachers and administrators.
Keys to Success • Communication Frequent, brief and effective communication is essential. Share information about the program and its activities. Constant communication (weekly newsletters, announcements, sharing and postings) ...email and the web are your friends! Build a community of knowledgeable and involved educators, and share the successes and failures encountered.
About Cost... • Running “Dual Programs” is very, very expensive... about twice the cost initially! • Support of the infrastructure to do online testing must not be shortsighted... It must be planned for…What you once spent on paper, the attention of staff, storage and mailing/ shipping, you will now spend on servers and programmers!
And (of course): Computerized Assessments and Learning (CAL)…http://cal-demo.caltesting.org/
