1 / 31

Performance-Based Assessments

Performance-Based Assessments. Jennifer Weber Arizona Department of Agriculture. Presentation Overview. Overview of standardized multiple-choice exam and performance based exam formats Activity involving each type of exam Compare and contrast formats based on your test results

garran
Download Presentation

Performance-Based Assessments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance-Based Assessments Jennifer Weber Arizona Department of Agriculture

  2. Presentation Overview • Overview of standardized multiple-choice exam and performance based exam formats • Activity involving each type of exam • Compare and contrast formats based on your test results • Describe characteristics of pesticide handlers • Draw conclusions about effectiveness of these two types of exams for the agricultural community

  3. Assessing Knowledge. . . through a hands-on activity through multiple-choice questions on a written test

  4. Assessing Knowledge. . . Hmm, now what is wrong with this picture? Hmm, now which of these is the correct answer?

  5. Assessing Knowledge. . . Individually answering questions about how to respond to pesticide-related situations Working together in small groups to demonstrate how to respond to a pesticide-related situation

  6. O.K. class, let’s begin.

  7. Correct Answers Your Results?

  8. Multiple-Choice Exam Activity • You will receive a 5 question, written, multiple-choice exam • Please read each question thoroughly and circle the correct answer • You will have 3 minutes to complete the exam • Do not discuss questions or answer options with others in the room • I will provide you with the correct “letter” but will not explain the answer itself. • You may begin your test

  9. Performance-Based Exam Activity Do we have a volunteer?

  10. Test Format Preference

  11. Discussion • How many people in this session preferred the multiple-choice exam format? • Why? • How many people in this session preferred the performance-based exam format? • Why?

  12. Test Format Preference Quote: • “Personally I like multiple choice as it is not open to interpretation and judgment of the person administering the exam and embarrassment, should I not have a clue about the answer. I have a 25% chance of getting the correct answer with multiple choice.” (Jack Peterson, personal communication, April 11, 2007)

  13. Advantages of Multiple-Choice Exams Over Performance-Based Exams • Easier to administer • Less time consuming for department staff • Can be scored quickly and easily if a Scantron form and scoring machine are used • More economical • Fewer logistics • Can test more individuals in a short amount of time

  14. Multiple-Choice Exams for Pesticide Applicators • May be the ideal format for a fluent English-reader who grew up on a farm, has completed high school in the United States and is accustomed to taking multiple-choice exams • Anyone without the aforementioned characteristics could struggle with the multiple-choice exam format. (Brenda Ball and Jack Peterson, personal communication, April 11, 2007)

  15. Diversity of Agricultural Community • Language skills • Literacy levels • Test taking experience • Learning styles • Educational background • Work experience • Age

  16. What is an Effective Way to Measure Knowledge and Skills? The story of the three brothers

  17. The story continues. . .

  18. Language and Literacy • The following table is from the 2005 NAWS report and displays the participants’ English speaking and reading abilities by place of birth and ethnicity data. English Speaking and Reading Ability, by Place of Birth and Ethnicity

  19. Language and Literacy • Transferring knowledge and terminology from one’s native language to English can be difficult, especially when the same knowledge was acquired in a different context. In this type of situation, standardized exams can become tests of English reading and comprehension abilities and not of content knowledge and skills. (Carlon Ami, Relevance of Assessment and Culture in Evaluation (RACE) Conference. Tempe, AZ 2004)

  20. Language and Literacy • “Thinking processes aside, non-English speakers often must learn additional words. For example, while all of them know that they must protect their eyes and skin during pesticide application, ‘ocular’ and ‘dermal’ are two more words that have to be learned to pass many exams.” (Gerald Kinro, personal communication, May 6, 2007)

  21. Multiple-Choice Exams - Spanish Private applicator licensing exams are offered in Spanish in Oregon, Washington, and California. • Results: • Oregon: 55% pass rate among pesticide applicators who took the Private Applicator’s exam in English and a 3-5% pass rate among those who take the same exam in Spanish (Myron Shenk, personal communication, 2001) • Monterey County, California: 80% of the English test takers and 15% of the Spanish test takers passing the Private Applicator exam on their first attempt (Maria Vidauri, personal communication, May 6, 2004)

  22. Learning Styles and Test Taking Experience • Native American students are holistic, visual learners, a contrast to the more logical and linear thinking skills that are often needed to excel on multiple-choice exams. • They possess unique intelligences and a vast amount of knowledge and skills that cannot be captured through standardized exams. (Carlon Ami, RACE Conference. Tempe, AZ 2004)

  23. Work Experience and Knowledge • Becky Hines provides a test preparation course at WSU for landscapers • Most have been working in the industry for several years and have vast amount of knowledge about pesticides and pest control • Excel in group work, individual tasks, and practice quizzes during the three day test preparation course

  24. Years of Experience Handling Pesticides • Very few have passed the pesticide applicator’s licensing exam that is administered immediately following the course • “It’s so frustrating! They’re not stupid. They know the information but they just can’t seem to pass these tests.” (Becky Hines, personal communication, March 30, 2007).

  25. Equity Concerns • The problem arises when these single scores are low despite one’s experience and expertise in the area of study. • Pesticide handlers who have limited English language and/or test taking abilities may struggle with the traditional multiple-choice format and be unable to display their true knowledge on the subject.

  26. Educational Background and Test Taking Experience • “The multiple-choice tests are hard for the Spanish-speaking growers from Mexico. Where I grew up in Mexico, we didn’t have multiple-choice exams or the Scantron form. It’s hard to adjust to these types of tests.” (Maria Vidauri, personal communication, 2004)

  27. Possible Solutions • One might think that a simple solution would be to present a course on multiple-choice test taking skills, to teach people how to use process of elimination tactics, pace themselves on timed tests, and fill in Scantron “bubbles” clearly and correctly • While such a session might be helpful for some, efforts should really be made on finding the right tool or tools to assess what pesticide handlers already know or need to learn about pesticide safety

  28. Alternative Form of Assessment • The provision of an alternative form of assessment, such as performance-based exam, would allow test takers who might have difficulties passing a multiple-choice exam with an opportunity to demonstrate their knowledge and skills through authentic, hands-on activities, case studies, and question and answer sessions designed to reflect actual workplace situations.

  29. Examples of Performance-Based Exam Programs • Crane operator’s exam • Business school admission requirement • Driver’s license driving demonstration • Computer technician “laboratory” • Clinical skills for nurses and veterinarians • Air traffic controller’s communication skills test

  30. Additional Advantages of Performance-Based Exams • A performance-based exam provides an opportunity to exchange safety information as well as assess individual knowledge and skill. • In the long run, this assessment process has the potential to prevent hazardous situations at the work site.

  31. Conclusion and Suggestions • Develop and offer two testing formats: • a multiple-choice exam, which may be the desired format for people who have attended school in the U.S. and are accustomed to this testing format • a performance-based exam that would enable test takers from various educational backgrounds with diverse language and test taking skills equal opportunity to demonstrate their knowledge on the safe and effective use of pesticides

More Related