Surviving (and Thriving) in a “Perfect Storm” Environment Ray T. Clifford 22 February 2007
The Perfect Storm • Too little time. • Limited funding. • Not enough qualified teachers. • Rising expectations.
Some Recent Headlines • Court Creates Demand for Translators • Home Invasions Show Need for Translators • On the Table: A Cure for Medicare Language Gap • Built to be Bilingual: Researchers Pinpoint How the Brain Switches Between Languages
Some Recent Headlines • Untying U.S. Tongues • FBI Agents Still Lacking Arabic Skills • U.S. has a Strategic Need for Multilingual Citizens • Learning Second Language Changes the Brain: Bilingual People have More Grey Matter in Key Region
Some Recent Headlines • Iraq Panel: Put Forward Officials who Speak to Arabs in Arabic • First Ears, Then Hearts and Minds: Facing Shortage of Arabic Interpreters, Pentagon Seeks a Technological Solution
High Demand, Low Supply • More than 80 federal agencies employ individuals with skills in more than 100 languages. • Degrees granted in 2002: • 2, 396 French • 13 Arabic • Only one certified Arabic teacher in U.S. public schools.
And Yet… • Our nation suffers from delusions of linguistic adequacy!
Overview • What do Americans “know” about languages? • Why aren’t foreign languages required subjects in U.S. schools? • How can the U.S. respond to changing national language needs?
Limited Experience Leads to Naiveté • The general public believes: • Every English word has an exact match in the second language. • Second language learning is simply a re-lexification process. • High levels of language competence can be attained in two years of school classes…or perhaps in a few weeks with a computer program.
Limited Experience Leads to Naiveté • Even teachers often believe: • Students’ failure to attain meaningful levels of proficiency is primarily the result of poor teaching. • There is a single, best teaching method.
Needs + Naiveté = Nonsense • The Reality of FL Learning • “Time on Task,” not teaching method, is the best predictor of success in second language learning. • Of course, how that time is spent does make a difference. • Nothing clarifies the debate about teaching methods like a definition of goals.
ILR Proficiency Level Summary [With language “text type” highlighted in red] LEVEL FUNCTION/TASKS CONTEXT/TOPICS ACCURACY All expected of an educated NS [Books] Accepted as an educated NS All subjects 5 Tailor language, counsel, motivate, persuade, negotiate [Chapters] Wide range of professional needs Extensive, precise, and appropriate 4 Errors never interfere with communication & rarely disturb Support opinions, hypothesize, explain, deal with unfamiliar topics [Multiple pages] Practical, abstract, special interests 3 Concrete, real-world, factual Intelligible even if not used to dealing with non-NS Narrate, describe, give directions [Multiple paragraphs] 2 Q & A, create with the language [Multiple sentences] Intelligible with effort or practice 1 Everydaysurvival 0 Memorized [Words and Phrases] Random Unintelligible
The ACTFL and ILR Scales Represent Significant, Substantive Levels of Accomplishment • Language is the most complex of human behaviors. • Language learning is an educational process which slowly, but inexorably reveals the limitations of any monolingual view of the world.
Language Teachers are Professional Educators. • The study of second language acquisition is a professional discipline. • Graduate degrees are offered in the field. • The discipline has professional journals, organizations, and conferences. • The field conducts an ever-expanding body of research. • Being a “native speaker” is not enough.
Bloom’s Taxonomy • Evaluation and persuasion through refined use of professional, literary, and rhetorical skills. • Synthesis of concepts to produce and comprehend abstract ideas and hypothetical situations. • Analysis and definition of factual relationships in paragraph length communications. • Application of skills to create and understand new communications. • Comprehension and use of words and phrases. • Memorization of facts.
Language and Cognition • Levels 4 & 5, evaluation and persuasion through refined use of professional rhetorical skills. • Level 3, synthesis of ideas to produce and comprehend abstract comments and hypothetical situations expressed in essays, chapters, etc. • Level 2, analysis and definition of relationships expressed in multiple interrelated paragraphs. • Level 1, application of skills to create and understand sentence length communications. • Level 0+, comprehension and use of words and phrases. • Level 0, memorization of facts.
Language and Perceived Intelligence It is good [for students] to become acquainted with languages, for they may have to go abroad, and should be able to talk to people, and not look like fools. I care not how much intelligence you have, if you cannot exhibit it, you look like an ignoramus. John Taylor, 1852
A Continuing Problem • 1975: The International Association for the Evaluation of Educational Achievement published the results of a study titled, The Teaching of French as a Foreign Language in Eight Countries. • In the U.S., the researchers could not find enough 12th grade students with four years of language study to complete the study as designed. • The U.S. students ranked last in competence. • The study found that “…the primary factor in the attainment of proficiency in …any foreign language… is the amount of instructional time provided.”
A Continuing Problem • 1979: The President’s commission on Foreign Language and International Studies reported, “Americans’ incompetence in Foreign Languages is nothing short of scandalous, and it is becoming worse.”
A Continuing Problem • 1983: The Commission on Excellence in Education • Heard testimony that in the U.S., FL instruction had yet to attain mediocrity. • Published the report A Nation at Risk, which recommended longer course sequences for foreign language programs.
A Continuing Problem • 2004: The Modern Language Association survey of foreign language enrollments in higher education reported: • Total foreign language enrollments have increased slightly. • However, as a percentage of total enrollments, foreign language enrollments are decreasing. • In 1960 they were 16.1% of total enrollments. • In 2002 they were only 8.3% of total enrollments.
And Now… • The original 2007 federal budget figures from the Office of Management and Budget showed the following planned reductions: Defense – 8.7% Education – 24.4% • The forecast: A competitive fiscal environment.
How can language programsbecome competitive? • Assess the current status of our programs. • Tell the real story. • Support our story with test results.
What do tests have to do with Teaching and Learning? • It helps to begin with the end in mind. • You can’t teach skills you can’t measure. (Even if you could, how could you prove you’ve done it?) • Education is undergoing a paradigm shift from instructional objectives to learner outcomes.
Changes in Accreditation • There is an unprecedented move to replace process reviews with outcome reviews. • Accreditation and Student Learning Outcomes: A proposed Point of Departure • Knowledge outcomes • Skills outcomes • Affective outcomes • Abilities (the integration of KSA outcomes) Peter T. Ewell , Council for Higher Education Accreditation, September 2001
What will be the effect of these accreditation requirements? • More testing will take place. • Some beneficial • Some detrimental • These tests will influence learning. • The “Will that be on the test?” phenomenon. • The temptation to “teach the test” instead of teaching the skills necessary to pass the test. • Every testing decision creates a washback effect.
“Washback” Effects • Testing has a negative impact when: • Educational goals are reduced to those that are most easily measured. • Testing procedures do not reflect course goals, for instance… • Giving multiple choice tests in speaking classes. • Using grammar tests as a measure of general proficiency. • The test results don’t provide useful information.
The National Debate onSchool Testing • One formula for evaluating school performance School score = (((((X23*100)*Y23) + ((X24*100)*Y24) + ((X25*100)*Y25) + ((X26*100)*Y26) + ((X27*100)*Y27) + ((X28*100)*Y28) + ((X29*100)*Y29) + ((X30*100)*Y30) / ((X23 + X24 + X25 + X26 + X27 + X28 +X29 + X30)*100)) + ((((Z23*100)*Y23) + ((Z24*100)*Y24) + ((Z25*100)*Y25) + ((Z26*100)*Y26) + ((Z27*100)*Y27) + ((Z28*100)*Y28) + ((Z29*100)*Y29) + ((Z30*100)*Y30)) /((Z23 + Z24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)) / ((Z23 + Z 24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)*100))) / 2 The Wall Street Journal, May 21, 2001, page A24
The National Debate onSchool Testing • What would be the washback effect of this evaluation formula? • Perhaps Confusion? • Perhaps Frustration? • Perhaps “teaching the test” in a desperate attempt to improve results?
Washback Effects of Tests • Testing has a positive impact when: • Tests reinforce course objectives. (For students, nothing defines course objectives like a test.) • Tests can act as change agents for improving teaching and learning.
Language Testing and Motivation • Appropriate tests can motivate learners to improve their skills. • Appropriate tests can motivate teachers to refine their teaching to match their students’ needs.
“Some considered projections” from THE NATIONAL INTEREST AND FOREIGN LANGUAGES Eight- and ten-year sequences of foreign language study will become common in the public schools. The better colleges and universities will require demonstrated proficiency (not high school “units”) in a foreign language for entrance, and demonstrated proficiency in a second foreign language (often non-Western) for graduation. WILLIAM RILEY PARKER for THE U.S. NATIONAL COMMISSION FOR UNESCO, DEPARTMENT OF STATE, 1961.
The National Debate onSchool Testing • What might have been the washback effect if students’ language abilities had been used to evaluate program effectiveness in 1961? • Perhaps clearer statements of objectives? • Perhaps more realistic learner expectations? • Perhaps more accurate course descriptions? • Perhaps qualitative feedback that would have justified longer course sequences?
After four decades, why are Parker’s projections not yet a reality? • Lasting improvements are only sustainable if there is a recognition of need for them. • Norm-referenced grading gave little evidence of real-world ability gaps. • Accreditation was focused on process rather than on outcomes, and provided no incentive to change. • Administrators didn’t read the research results showing that no improvements in teaching methods or curricula can compensate for the current lack of “time-on-task” in our educational system.
International Research on Learning French • John B. Carroll, The Teaching of French in Eight Countries (International Studies in Evaluation V) John Wiley & Sons, New York, 1975. • “… the primary factor in the attainment of proficiency in French (and presumably, any foreign language) is the amount of instructional time provided.” [Page 276]
International Research on Learning English • G. Bonnet, et al. The Assessment of Pupils’ skills in English in Eight European Countries: 2002. European Network of Policy makers for the Evaluation of Education Systems, 2004. • Students from those nations where there is more contact with (and more time spent using) English have higher levels of competence in English. • In a language-rich environment, time spent using the language is more important than the teaching methods used in the classroom.
Assessing FL Learner Outcomes • FL profession has made much progress in the area of assessment and standards. • Our ability to test students’ language proficiency puts us ahead of many other disciplines. • But proficiency tests are not the only way to measure learner outcomes. • Which test is “best” will depend on: • The purpose of the test. • Each program’s instructional goals.
Matching Tests with Your Instructional Purposes • Some typical FL testing purposes. • Assigning course grades in a beginning class. • Placement of students into a sequence of courses. • Certifying a general level of ability for teachers. • Screening of job applicants. • Some general instructional purposes. • Achievement – knowledge of contributing language elements found in a specific curriculum. • Performance – ability to communicate in specific settings. • Proficiency – demonstration of unrehearsed ability.
If Tests Are to bePositive Motivators • We have to select the right type of test for each testing purpose.
The 3 Major Types of Tests • Achievement = Memorized responses using the content of a specific textbook or curriculum. • Performance =Rehearsed ability to communicate in specific, familiar settings. • Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.
Some CommonTesting Purposes • Assigning grades in a class. • Placing students into a sequence of courses. • Selecting an applicant for a job with limited, static language requirements. • Screening employees for future jobs with broad, general language requirements.
Aligning testing programs with our testing purposes requires knowing what makes proficiency tests different fromperformance testsand achievement tests.
The ILR/ACTFL Proficiency Scales are Different, Because They: • Represent a geometric progression. • Use threshold rating criteria. • Do not use compensatory rating procedures. • Are a multidimensional measure of ability. • Apply Guttmann scaling criteria. • Are based on a wide range of “real world” communication needs. • Assess unrehearsed language abilities.
1. The ILR/ACTFL scales represent a geometric progression. • Advancing from one level to the next becomes increasingly difficult as one moves up the scale. • For instance, moving from 0 to 1 requires less time and effort than moving from 2 to 3.
2. The ILR/ACTFL scales use “threshold” rating criteria. • Candidates must meet all of the stated criteria for a given level. • If they meet most, but not all of the criteria, some award a “plus” designation to the next lower level. • For example, someone who meets the criteria for Level 2 almost all of the time, could be rated 1+.
3. The ILR/ACTFL Ratings Are Not Based On Total, Average, or Other “Compensatory” Scores. • Ratings may be limited by one or more of the task, context, and accuracy requirements. • For example, a large vocabulary does not compensate for inaccurate usage.
The ILR/ACTFL scales representa complex set of abilities. • The ILR/ACTFL scale is a multi-dimensional scale. • The scale is not uni-dimensional like a rope that has been cut into segments. • It is multi-dimensional, because each level of the scale represents a different constellation of supporting and enabling skills. • The ILR/ACTFL scale (and other complex scales) cannot be represented by a one-dimensional statistical model.