Advertisement
1 / 276

Surviving (and Thriving) in a “Perfect Storm” Environment PowerPoint PPT Presentation


Surviving (and Thriving) in a “Perfect Storm” Environment Ray T. Clifford 22 February 2007 The Perfect Storm Too little time. Limited funding. Not enough qualified teachers. Rising expectations. Some Recent Headlines Court Creates Demand for Translators - PowerPoint PPT Presentation

Download Presentation

Surviving (and Thriving) in a “Perfect Storm” Environment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Surviving and thriving in a perfect storm environment l.jpg

Surviving (and Thriving) in a “Perfect Storm” Environment

Ray T. Clifford

22 February 2007


The perfect storm l.jpg

The Perfect Storm

  • Too little time.

  • Limited funding.

  • Not enough qualified teachers.

  • Rising expectations.


Some recent headlines l.jpg

Some Recent Headlines

  • Court Creates Demand for Translators

  • Home Invasions Show Need for Translators

  • On the Table: A Cure for Medicare Language Gap

  • Built to be Bilingual: Researchers Pinpoint How the Brain Switches Between Languages


Some recent headlines4 l.jpg

Some Recent Headlines

  • Untying U.S. Tongues

  • FBI Agents Still Lacking Arabic Skills

  • U.S. has a Strategic Need for Multilingual Citizens

  • Learning Second Language Changes the Brain: Bilingual People have More Grey Matter in Key Region


Some recent headlines5 l.jpg

Some Recent Headlines

  • Iraq Panel: Put Forward Officials who Speak to Arabs in Arabic

  • First Ears, Then Hearts and Minds: Facing Shortage of Arabic Interpreters, Pentagon Seeks a Technological Solution


High demand low supply l.jpg

High Demand, Low Supply

  • More than 80 federal agencies employ individuals with skills in more than 100 languages.

  • Degrees granted in 2002:

    • 2, 396 French

    • 13 Arabic

  • Only one certified Arabic teacher in U.S. public schools.


And yet l.jpg

And Yet…

  • Our nation suffers from delusions of linguistic adequacy!


Overview l.jpg

Overview

  • What do Americans “know” about languages?

  • Why aren’t foreign languages required subjects in U.S. schools?

  • How can the U.S. respond to changing national language needs?


Limited experience leads to naivet l.jpg

Limited Experience Leads to Naiveté

  • The general public believes:

    • Every English word has an exact match in the second language.

    • Second language learning is simply a

      re-lexification process.

    • High levels of language competence can be attained in two years of school classes…or perhaps in a few weeks with a computer program.


Limited experience leads to naivet10 l.jpg

Limited Experience Leads to Naiveté

  • Even teachers often believe:

    • Students’ failure to attain meaningful levels of proficiency is primarily the result of poor teaching.

    • There is a single, best teaching method.


Needs naivet nonsense l.jpg

Needs + Naiveté = Nonsense

  • The Reality of FL Learning

    • “Time on Task,” not teaching method, is the best predictor of success in second language learning.

    • Of course, how that time is spent does make a difference.

  • Nothing clarifies the debate about teaching methods like a definition of goals.


The major actfl proficiency levels l.jpg

The Major ACTFL Proficiency Levels


Slide13 l.jpg

ILR Proficiency Level Summary

[With language “text type” highlighted in red]

LEVEL

FUNCTION/TASKS

CONTEXT/TOPICS

ACCURACY

All expected of an educated NS [Books]

Accepted as an educated NS

All subjects

5

Tailor language, counsel, motivate, persuade, negotiate [Chapters]

Wide range of professional needs

Extensive, precise, and appropriate

4

Errors never interfere with communication & rarely disturb

Support opinions, hypothesize, explain, deal with unfamiliar topics [Multiple pages]

Practical, abstract, special interests

3

Concrete, real-world, factual

Intelligible even if not used to dealing with non-NS

Narrate, describe, give directions [Multiple paragraphs]

2

Q & A, create with the language [Multiple sentences]

Intelligible with effort or practice

1

Everydaysurvival

0

Memorized [Words and Phrases]

Random

Unintelligible


The actfl and ilr scales represent significant substantive levels of accomplishment l.jpg

The ACTFL and ILR Scales Represent Significant, Substantive Levels of Accomplishment

  • Language is the most complex of human behaviors.

  • Language learning is an educational process which slowly, but inexorably reveals the limitations of any monolingual view of the world.


Language teachers are professional educators l.jpg

Language Teachers are Professional Educators.

  • The study of second language acquisition is a professional discipline.

    • Graduate degrees are offered in the field.

    • The discipline has professional journals, organizations, and conferences.

    • The field conducts an ever-expanding body of research.

  • Being a “native speaker” is not enough.


Bloom s taxonomy l.jpg

Bloom’s Taxonomy

  • Evaluation and persuasion through refined use of professional, literary, and rhetorical skills.

  • Synthesis of concepts to produce and comprehend abstract ideas and hypothetical situations.

  • Analysis and definition of factual relationships in paragraph length communications.

  • Application of skills to create and understand new communications.

  • Comprehension and use of words and phrases.

  • Memorization of facts.


Language and cognition l.jpg

Language and Cognition

  • Levels 4 & 5, evaluation and persuasion through refined use of professional rhetorical skills.

  • Level 3, synthesis of ideas to produce and comprehend abstract comments and hypothetical situations expressed in essays, chapters, etc.

  • Level 2, analysis and definition of relationships expressed in multiple interrelated paragraphs.

  • Level 1, application of skills to create and understand sentence length communications.

  • Level 0+, comprehension and use of words and phrases.

  • Level 0, memorization of facts.


Language and perceived intelligence l.jpg

Language and Perceived Intelligence

It is good [for students] to become acquainted with languages, for they may have to go abroad, and should be able to talk to people, and not look like fools. I care not how much intelligence you have, if you cannot exhibit it, you look like an ignoramus.

John Taylor, 1852


A continuing problem l.jpg

A Continuing Problem

  • 1975: The International Association for the Evaluation of Educational Achievement published the results of a study titled, The Teaching of French as a Foreign Language in Eight Countries.

    • In the U.S., the researchers could not find enough 12th grade students with four years of language study to complete the study as designed.

    • The U.S. students ranked last in competence.

    • The study found that “…the primary factor in the attainment of proficiency in …any foreign language… is the amount of instructional time provided.”


A continuing problem20 l.jpg

A Continuing Problem

  • 1979: The President’s commission on Foreign Language and International Studies reported,

    “Americans’ incompetence in Foreign Languages is nothing short of scandalous, and it is becoming worse.”


A continuing problem21 l.jpg

A Continuing Problem

  • 1983: The Commission on Excellence in Education

    • Heard testimony that in the U.S., FL instruction had yet to attain mediocrity.

    • Published the report A Nation at Risk, which recommended longer course sequences for foreign language programs.


A continuing problem22 l.jpg

A Continuing Problem

  • 2004: The Modern Language Association survey of foreign language enrollments in higher education reported:

    • Total foreign language enrollments have increased slightly.

    • However, as a percentage of total enrollments, foreign language enrollments are decreasing.

      • In 1960 they were 16.1% of total enrollments.

      • In 2002 they were only 8.3% of total enrollments.


And now l.jpg

And Now…

  • The original 2007 federal budget figures from the Office of Management and Budget showed the following planned reductions:

    Defense – 8.7%

    Education – 24.4%

  • The forecast: A competitive fiscal environment.


How can language programs become competitive l.jpg

How can language programsbecome competitive?

  • Assess the current status of our programs.

  • Tell the real story.

  • Support our story with test results.


What do tests have to do with teaching and learning l.jpg

What do tests have to do with Teaching and Learning?

  • It helps to begin with the end in mind.

  • You can’t teach skills you can’t measure.

    (Even if you could, how could you prove you’ve done it?)

  • Education is undergoing a paradigm shift from instructional objectives to learner outcomes.


Changes in accreditation l.jpg

Changes in Accreditation

  • There is an unprecedented move to replace process reviews with outcome reviews.

  • Accreditation and Student Learning Outcomes: A proposed Point of Departure

    • Knowledge outcomes

    • Skills outcomes

    • Affective outcomes

    • Abilities (the integration of KSA outcomes)

      Peter T. Ewell , Council for Higher Education Accreditation, September 2001


What will be the effect of these accreditation requirements l.jpg

What will be the effect of these accreditation requirements?

  • More testing will take place.

    • Some beneficial

    • Some detrimental

  • These tests will influence learning.

    • The “Will that be on the test?” phenomenon.

    • The temptation to “teach the test” instead of teaching the skills necessary to pass the test.

    • Every testing decision creates a washback effect.


Washback effects l.jpg

“Washback” Effects

  • Testing has a negative impact when:

    • Educational goals are reduced to those that are most easily measured.

    • Testing procedures do not reflect course goals, for instance…

      • Giving multiple choice tests in speaking classes.

      • Using grammar tests as a measure of general proficiency.

    • The test results don’t provide useful information.


The national debate on school testing l.jpg

The National Debate onSchool Testing

  • One formula for evaluating school performance

    School score = (((((X23*100)*Y23) + ((X24*100)*Y24) + ((X25*100)*Y25) + ((X26*100)*Y26) + ((X27*100)*Y27) + ((X28*100)*Y28) + ((X29*100)*Y29) + ((X30*100)*Y30) / ((X23 + X24 + X25 + X26 + X27 + X28 +X29 + X30)*100)) + ((((Z23*100)*Y23) + ((Z24*100)*Y24) + ((Z25*100)*Y25) + ((Z26*100)*Y26) + ((Z27*100)*Y27) + ((Z28*100)*Y28) + ((Z29*100)*Y29) + ((Z30*100)*Y30)) /((Z23 + Z24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)) / ((Z23 + Z24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)*100))) / 2

    The Wall Street Journal, May 21, 2001, page A24


The national debate on school testing30 l.jpg

The National Debate onSchool Testing

  • What would be the washback effect of this evaluation formula?

    • Perhaps Confusion?

    • Perhaps Frustration?

    • Perhaps “teaching the test” in a desperate attempt to improve results?


Washback effects of tests l.jpg

Washback Effects of Tests

  • Testing has a positive impact when:

    • Tests reinforce course objectives.

      (For students, nothing defines course objectives like a test.)

    • Tests can act as change agents for improving teaching and learning.


Language testing and motivation l.jpg

Language Testing and Motivation

  • Appropriate tests can motivate learners to improve their skills.

  • Appropriate tests can motivate teachers to refine their teaching to match their students’ needs.


Slide33 l.jpg

“Some considered projections” from

THE NATIONAL INTEREST AND FOREIGN LANGUAGES

Eight- and ten-year sequences of foreign language study will become common in the public schools.

The better colleges and universities will require demonstrated proficiency (not high school “units”) in a foreign language for entrance, and demonstrated proficiency in a second foreign language (often non-Western) for graduation.

WILLIAM RILEY PARKER for

THE U.S. NATIONAL COMMISSION FOR UNESCO, DEPARTMENT OF STATE, 1961.


The national debate on school testing34 l.jpg

The National Debate onSchool Testing

  • What might have been the washback effect if students’ language abilities had been used to evaluate program effectiveness in 1961?

    • Perhaps clearer statements of objectives?

    • Perhaps more realistic learner expectations?

    • Perhaps more accurate course descriptions?

    • Perhaps qualitative feedback that would have justified longer course sequences?


After four decades why are parker s projections not yet a reality l.jpg

After four decades, why are Parker’s projections not yet a reality?

  • Lasting improvements are only sustainable if there is a recognition of need for them.

  • Norm-referenced grading gave little evidence of real-world ability gaps.

  • Accreditation was focused on process rather than on outcomes, and provided no incentive to change.

  • Administrators didn’t read the research results showing that no improvements in teaching methods or curricula can compensate for the current lack of “time-on-task” in our educational system.


International research on learning french l.jpg

International Research on Learning French

  • John B. Carroll, The Teaching of French in Eight Countries (International Studies in Evaluation V) John Wiley & Sons, New York, 1975.

    • “… the primary factor in the attainment of proficiency in French (and presumably, any foreign language) is the amount of instructional time provided.” [Page 276]


International research on learning english l.jpg

International Research on Learning English

  • G. Bonnet, et al. The Assessment of Pupils’ skills in English in Eight European Countries: 2002. European Network of Policy makers for the Evaluation of Education Systems, 2004.

    • Students from those nations where there is more contact with (and more time spent using) English have higher levels of competence in English.

    • In a language-rich environment, time spent using the language is more important than the teaching methods used in the classroom.


Assessing fl learner outcomes l.jpg

Assessing FL Learner Outcomes

  • FL profession has made much progress in the area of assessment and standards.

  • Our ability to test students’ language proficiency puts us ahead of many other disciplines.

  • But proficiency tests are not the only way to measure learner outcomes.

  • Which test is “best” will depend on:

    • The purpose of the test.

    • Each program’s instructional goals.


Matching tests with your instructional purposes l.jpg

Matching Tests with Your Instructional Purposes

  • Some typical FL testing purposes.

    • Assigning course grades in a beginning class.

    • Placement of students into a sequence of courses.

    • Certifying a general level of ability for teachers.

    • Screening of job applicants.

  • Some general instructional purposes.

    • Achievement – knowledge of contributing language elements found in a specific curriculum.

    • Performance – ability to communicate in specific settings.

    • Proficiency – demonstration of unrehearsed ability.


If tests are to be positive motivators l.jpg

If Tests Are to bePositive Motivators

  • We have to select the right type of test for each testing purpose.


The 3 major types of tests l.jpg

The 3 Major Types of Tests

  • Achievement = Memorized responses using the content of a specific textbook or curriculum.

  • Performance =Rehearsed ability to communicate in specific, familiar settings.

  • Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.


Some common testing purposes l.jpg

Some CommonTesting Purposes

  • Assigning grades in a class.

  • Placing students into a sequence of courses.

  • Selecting an applicant for a job with limited, static language requirements.

  • Screening employees for future jobs with broad, general language requirements.


Slide43 l.jpg

Aligning testing programs with our testing purposes requires knowing what makes proficiency tests different fromperformance testsand achievement tests.


The ilr actfl proficiency scales are different because they l.jpg

The ILR/ACTFL Proficiency Scales are Different, Because They:

  • Represent a geometric progression.

  • Use threshold rating criteria.

  • Do not use compensatory rating procedures.

  • Are a multidimensional measure of ability.

  • Apply Guttmann scaling criteria.

  • Are based on a wide range of “real world” communication needs.

  • Assess unrehearsed language abilities.


1 the ilr actfl scales represent a geometric progression l.jpg

1. The ILR/ACTFL scales represent a geometric progression.

  • Advancing from one level to the next becomes increasingly difficult as one moves up the scale.

  • For instance, moving from 0 to 1 requires less time and effort than moving from 2 to 3.


What are the major levels how are they defined l.jpg

What are the Major Levels?How are They Defined?


2 the ilr actfl scales use threshold rating criteria l.jpg

2. The ILR/ACTFL scales use “threshold” rating criteria.

  • Candidates must meet all of the stated criteria for a given level.

  • If they meet most, but not all of the criteria, some award a “plus” designation to the next lower level.

  • For example, someone who meets the criteria for Level 2 almost all of the time, could be rated 1+.


3 the ilr actfl ratings are not based on total average or other compensatory scores l.jpg

3. The ILR/ACTFL Ratings Are Not Based On Total, Average, or Other “Compensatory” Scores.

  • Ratings may be limited by one or more of the task, context, and accuracy requirements.

  • For example, a large vocabulary does not compensate for inaccurate usage.


Implications of points 2 and 3 assessment criteria speaking l.jpg

Implications of Points 2 and 3: Assessment Criteria-Speaking


The ilr actfl scales represent a complex set of abilities l.jpg

The ILR/ACTFL scales representa complex set of abilities.

  • The ILR/ACTFL scale is a multi-dimensional scale.

    • The scale is not uni-dimensional like a rope that has been cut into segments.

    • It is multi-dimensional, because each level of the scale represents a different constellation of supporting and enabling skills.

  • The ILR/ACTFL scale (and other complex scales) cannot be represented by a one-dimensional statistical model.


Bloom s cognitive taxonomy l.jpg

Bloom’s Cognitive Taxonomy

  • Evaluationand persuasion through refined use of professional, literary, and rhetorical skills.

  • Synthesisof concepts to produce and comprehend abstract ideas and hypothetical situations.

  • Analysisand comparisons of factual relationships in complex situations.

  • Applicationof skills to apply facts and understand new applications of those facts.

  • Comprehensionand facts learned.

  • Memorizationof facts.


Cognition and language taxonomy l.jpg

Cognition and Language Taxonomy

  • Levels 4 & 5, evaluation and persuasionthrough refined use of professional rhetorical skills.

  • Level 3, synthesisof concepts to produce and comprehend abstract ideas and hypothetical situations.

  • Level 2, analysisand description of factual relationships in paragraph length communications.

  • Level 1, application of skills to create and understand new communications. (Performance)

  • Level 0+, comprehension and useof words and phrases. (Achievement / Performance)

  • Level 0, memorization of facts. (Achievement)


5 ilr actfl scale ratings apply guttmann scaling criteria l.jpg

5. ILR/ACTFL Scale Ratings Apply Guttmann Scaling Criteria.

  • To qualify for any given rating, a person must meet the criteria for that level and also satisfy the criteria for all lower ratings.

  • For instance, a Level 2 (Advanced) speaker must also be able to satisfy Level 1 (Intermediate) task, topical, and accuracy communication requirements.


6 the ilr actfl ratings are based on a range of real world communication needs l.jpg

6. The ILR/ACTFL ratings are based on a range of “real world” communication needs.

  • The candidate must demonstrate sustained ability across a wide range of topics and real world settings.

  • For instance, proficiency ratings can not be assigned based on one’s:

    • knowledge of a textbook.

    • performance of a specific job task or set of tasks.


7 the ilr actfl scales assess unrehearsed abilities l.jpg

7. The ILR/ACTFL scalesassess unrehearsed abilities.

  • Broad elicitation techniques are used to verify sustainable ability versus limited performance.

  • For instance,

    • At level 1 (Intermediate), test takers must create new utterances, not just recite dialogs.

    • At level 2 (Advanced), they must handle new work requirements, not just routine communications.

    • At level 3 (Superior), they must hypothesize and defend opinions in subjects beyond their personal interests and areas of specialization.

  • “Proficiency” indicates transferable skills – not just rehearsed performance in familiar areas.


A summary of the contrasts between proficiency and rehearsed performance l.jpg

A Summary of the Contrasts Between Proficiency And Rehearsed Performance

Unrehearsed Rehearsed

ProficiencyPerformance

Task:A wide range of abilities Specific Job Skills

Context:Broad, in-depth, variable Focused, restricted

Accuracy: Ascending expectations Situation dependent


So what l.jpg

So what?


Review of major test types l.jpg

Review of Major Test Types

  • Achievement = Memorized responses using the content of a specific textbook or curriculum.

  • Performance =Rehearsed ability to communicate in specific, familiar settings.

  • Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.


A self assessment quiz l.jpg

A Self Assessment Quiz

  • You will be asked about 6 different testing purposes.

  • For each of those test purposes, which type of test would you choose?a. Achievement.

    b. Performance.

    c. Proficiency.


Which type of test would you choose achievement performance or proficiency l.jpg

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?


Which type of test would you choose achievement performance or proficiency61 l.jpg

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?


Which type of test would you choose achievement performance or proficiency62 l.jpg

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?


Which type of test would you choose achievement performance or proficiency63 l.jpg

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?


Which test type would you choose achievement performance or proficiency l.jpg

Which test type would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

  • To select someone to be the NATO spokesperson on a news show with a “hostile” moderator?


Which test type would you choose achievement performance or proficiency65 l.jpg

Which test type would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

  • To select someone to be the NATO spokesperson on a news show with a “hostile” moderator?

  • To document a person’s language ability in that person’s official personnel file?


Achievement performance or proficiency which test type did you choose l.jpg

Achievement, Performance, or ProficiencyWhich test type did you choose?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

  • To select someone to be the NATO spokesperson on a “hostile” news show?

  • To document a person’s language ability in that person’s official personnel file?


Solving testing problems l.jpg

Solving Testing Problems

  • “The solutions to our problems should be as simple as possible, but no simpler.”

    Albert Einstein

  • There is no answer for the overly simple question of “Which test is best?”

  • There is an answer to the question, “Which type of test is best for a given purpose?”


Which test is best l.jpg

Which test is best?

  • The test that matches the purpose for which the results will be used.

    • Use achievement tests for testing mastery of lessons in a textbook.

    • Use performance tests for checking rehearsed abilities within specific contexts.

    • Use proficiency tests for determining general, unrehearsed ability in real-world situations.


Why are programs not assessed l.jpg

Why are programs not assessed?

  • It would be too much work.

  • It is not “my thing.”

  • We’d rather not know the results.


Why assess our programs l.jpg

Why assess our programs?

  • In today’s environment – if you don’t measure it, you can’t get money for it.

  • Assessment will help us articulate our objectives.

    • For ourselves.

    • For the students.

    • For program sponsors.

  • Formative assessment allows informed decision making.


The improvement process l.jpg

The Improvement Process

  • Set goals.

  • Decide how to measure progress toward those goals.

  • Collect data.

  • Analyze the results.

  • Make informed changes.

  • Repeat.


It won t be easy but if we are willing to work together and learn from the past l.jpg

It won’t be easy, but if we are willing to work together and learn from the past…

then William Riley Parker’s vision of second language programs in the U.S. might yet become a reality!


In the field of second language teaching the u s has a long history of missed opportunities l.jpg

In the field of second language teaching, the U.S. has a long history of missed opportunities.

  • Only if we are willing to document the status of our programs, can we hope to improve them.

  • With limited resources, collaboration will be more productive than competition.


Conclusion l.jpg

Conclusion

  • The past has not been perfect,

  • and the future is conditional

  • on whether the present is progressive.