Surviving (and Thriving) in a “Perfect Storm” Environment
Download

Surviving (and Thriving) in a “Perfect Storm” Environment







Advertisement
/ 276 []
Download Presentation
Comments
arleen
From:
|  
(3759) |   (0) |   (0)
Views: 118 | Added: 13-12-2010
Rate Presentation: 0 0
Description:
Surviving (and Thriving) in a “Perfect Storm” Environment Ray T. Clifford 22 February 2007 The Perfect Storm Too little time. Limited funding. Not enough qualified teachers. Rising expectations. Some Recent Headlines Court Creates Demand for Translators
Surviving (and Thriving) in a “Perfect Storm” Environment

An Image/Link below is provided (as is) to

Download Policy: Content on the Website is provided to you AS IS for your information and personal use only and may not be sold or licensed nor shared on other sites. SlideServe reserves the right to change this policy at anytime. While downloading, If for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.











- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -




Slide 1

Surviving (and Thriving) in a “Perfect Storm” Environment

Ray T. Clifford

22 February 2007

Slide 2

The Perfect Storm

  • Too little time.

  • Limited funding.

  • Not enough qualified teachers.

  • Rising expectations.

Slide 3

Some Recent Headlines

  • Court Creates Demand for Translators

  • Home Invasions Show Need for Translators

  • On the Table: A Cure for Medicare Language Gap

  • Built to be Bilingual: Researchers Pinpoint How the Brain Switches Between Languages

Slide 4

Some Recent Headlines

  • Untying U.S. Tongues

  • FBI Agents Still Lacking Arabic Skills

  • U.S. has a Strategic Need for Multilingual Citizens

  • Learning Second Language Changes the Brain: Bilingual People have More Grey Matter in Key Region

Slide 5

Some Recent Headlines

  • Iraq Panel: Put Forward Officials who Speak to Arabs in Arabic

  • First Ears, Then Hearts and Minds: Facing Shortage of Arabic Interpreters, Pentagon Seeks a Technological Solution

Slide 6

High Demand, Low Supply

  • More than 80 federal agencies employ individuals with skills in more than 100 languages.

  • Degrees granted in 2002:

    • 2, 396 French

    • 13 Arabic

  • Only one certified Arabic teacher in U.S. public schools.

Slide 7

And Yet…

  • Our nation suffers from delusions of linguistic adequacy!

Slide 8

Overview

  • What do Americans “know” about languages?

  • Why aren’t foreign languages required subjects in U.S. schools?

  • How can the U.S. respond to changing national language needs?

Slide 9

Limited Experience Leads to Naiveté

  • The general public believes:

    • Every English word has an exact match in the second language.

    • Second language learning is simply a

      re-lexification process.

    • High levels of language competence can be attained in two years of school classes…or perhaps in a few weeks with a computer program.

Slide 10

Limited Experience Leads to Naiveté

  • Even teachers often believe:

    • Students’ failure to attain meaningful levels of proficiency is primarily the result of poor teaching.

    • There is a single, best teaching method.

Slide 11

Needs + Naiveté = Nonsense

  • The Reality of FL Learning

    • “Time on Task,” not teaching method, is the best predictor of success in second language learning.

    • Of course, how that time is spent does make a difference.

  • Nothing clarifies the debate about teaching methods like a definition of goals.

Slide 12

The Major ACTFL Proficiency Levels

Slide 13

ILR Proficiency Level Summary

[With language “text type” highlighted in red]

LEVEL

FUNCTION/TASKS

CONTEXT/TOPICS

ACCURACY

All expected of an educated NS [Books]

Accepted as an educated NS

All subjects

5

Tailor language, counsel, motivate, persuade, negotiate [Chapters]

Wide range of professional needs

Extensive, precise, and appropriate

4

Errors never interfere with communication & rarely disturb

Support opinions, hypothesize, explain, deal with unfamiliar topics [Multiple pages]

Practical, abstract, special interests

3

Concrete, real-world, factual

Intelligible even if not used to dealing with non-NS

Narrate, describe, give directions [Multiple paragraphs]

2

Q & A, create with the language [Multiple sentences]

Intelligible with effort or practice

1

Everydaysurvival

0

Memorized [Words and Phrases]

Random

Unintelligible

Slide 14

The ACTFL and ILR Scales Represent Significant, Substantive Levels of Accomplishment

  • Language is the most complex of human behaviors.

  • Language learning is an educational process which slowly, but inexorably reveals the limitations of any monolingual view of the world.

Slide 15

Language Teachers are Professional Educators.

  • The study of second language acquisition is a professional discipline.

    • Graduate degrees are offered in the field.

    • The discipline has professional journals, organizations, and conferences.

    • The field conducts an ever-expanding body of research.

  • Being a “native speaker” is not enough.

Slide 16

Bloom’s Taxonomy

  • Evaluation and persuasion through refined use of professional, literary, and rhetorical skills.

  • Synthesis of concepts to produce and comprehend abstract ideas and hypothetical situations.

  • Analysis and definition of factual relationships in paragraph length communications.

  • Application of skills to create and understand new communications.

  • Comprehension and use of words and phrases.

  • Memorization of facts.

Slide 17

Language and Cognition

  • Levels 4 & 5, evaluation and persuasion through refined use of professional rhetorical skills.

  • Level 3, synthesis of ideas to produce and comprehend abstract comments and hypothetical situations expressed in essays, chapters, etc.

  • Level 2, analysis and definition of relationships expressed in multiple interrelated paragraphs.

  • Level 1, application of skills to create and understand sentence length communications.

  • Level 0+, comprehension and use of words and phrases.

  • Level 0, memorization of facts.

Slide 18

Language and Perceived Intelligence

It is good [for students] to become acquainted with languages, for they may have to go abroad, and should be able to talk to people, and not look like fools. I care not how much intelligence you have, if you cannot exhibit it, you look like an ignoramus.

John Taylor, 1852

Slide 19

A Continuing Problem

  • 1975: The International Association for the Evaluation of Educational Achievement published the results of a study titled, The Teaching of French as a Foreign Language in Eight Countries.

    • In the U.S., the researchers could not find enough 12th grade students with four years of language study to complete the study as designed.

    • The U.S. students ranked last in competence.

    • The study found that “…the primary factor in the attainment of proficiency in …any foreign language… is the amount of instructional time provided.”

Slide 20

A Continuing Problem

  • 1979: The President’s commission on Foreign Language and International Studies reported,

    “Americans’ incompetence in Foreign Languages is nothing short of scandalous, and it is becoming worse.”

Slide 21

A Continuing Problem

  • 1983: The Commission on Excellence in Education

    • Heard testimony that in the U.S., FL instruction had yet to attain mediocrity.

    • Published the report A Nation at Risk, which recommended longer course sequences for foreign language programs.

Slide 22

A Continuing Problem

  • 2004: The Modern Language Association survey of foreign language enrollments in higher education reported:

    • Total foreign language enrollments have increased slightly.

    • However, as a percentage of total enrollments, foreign language enrollments are decreasing.

      • In 1960 they were 16.1% of total enrollments.

      • In 2002 they were only 8.3% of total enrollments.

Slide 23

And Now…

  • The original 2007 federal budget figures from the Office of Management and Budget showed the following planned reductions:

    Defense – 8.7%

    Education – 24.4%

  • The forecast: A competitive fiscal environment.

Slide 24

How can language programsbecome competitive?

  • Assess the current status of our programs.

  • Tell the real story.

  • Support our story with test results.

Slide 25

What do tests have to do with Teaching and Learning?

  • It helps to begin with the end in mind.

  • You can’t teach skills you can’t measure.

    (Even if you could, how could you prove you’ve done it?)

  • Education is undergoing a paradigm shift from instructional objectives to learner outcomes.

Slide 26

Changes in Accreditation

  • There is an unprecedented move to replace process reviews with outcome reviews.

  • Accreditation and Student Learning Outcomes: A proposed Point of Departure

    • Knowledge outcomes

    • Skills outcomes

    • Affective outcomes

    • Abilities (the integration of KSA outcomes)

      Peter T. Ewell , Council for Higher Education Accreditation, September 2001

Slide 27

What will be the effect of these accreditation requirements?

  • More testing will take place.

    • Some beneficial

    • Some detrimental

  • These tests will influence learning.

    • The “Will that be on the test?” phenomenon.

    • The temptation to “teach the test” instead of teaching the skills necessary to pass the test.

    • Every testing decision creates a washback effect.

Slide 28

“Washback” Effects

  • Testing has a negative impact when:

    • Educational goals are reduced to those that are most easily measured.

    • Testing procedures do not reflect course goals, for instance…

      • Giving multiple choice tests in speaking classes.

      • Using grammar tests as a measure of general proficiency.

    • The test results don’t provide useful information.

Slide 29

The National Debate onSchool Testing

  • One formula for evaluating school performance

    School score = (((((X23*100)*Y23) + ((X24*100)*Y24) + ((X25*100)*Y25) + ((X26*100)*Y26) + ((X27*100)*Y27) + ((X28*100)*Y28) + ((X29*100)*Y29) + ((X30*100)*Y30) / ((X23 + X24 + X25 + X26 + X27 + X28 +X29 + X30)*100)) + ((((Z23*100)*Y23) + ((Z24*100)*Y24) + ((Z25*100)*Y25) + ((Z26*100)*Y26) + ((Z27*100)*Y27) + ((Z28*100)*Y28) + ((Z29*100)*Y29) + ((Z30*100)*Y30)) /((Z23 + Z24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)) / ((Z23 + Z 24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)*100))) / 2

    The Wall Street Journal, May 21, 2001, page A24

Slide 30

The National Debate onSchool Testing

  • What would be the washback effect of this evaluation formula?

    • Perhaps Confusion?

    • Perhaps Frustration?

    • Perhaps “teaching the test” in a desperate attempt to improve results?

Slide 31

Washback Effects of Tests

  • Testing has a positive impact when:

    • Tests reinforce course objectives.

      (For students, nothing defines course objectives like a test.)

    • Tests can act as change agents for improving teaching and learning.

Slide 32

Language Testing and Motivation

  • Appropriate tests can motivate learners to improve their skills.

  • Appropriate tests can motivate teachers to refine their teaching to match their students’ needs.

Slide 33

“Some considered projections” from

THE NATIONAL INTEREST AND FOREIGN LANGUAGES

Eight- and ten-year sequences of foreign language study will become common in the public schools.

The better colleges and universities will require demonstrated proficiency (not high school “units”) in a foreign language for entrance, and demonstrated proficiency in a second foreign language (often non-Western) for graduation.

WILLIAM RILEY PARKER for

THE U.S. NATIONAL COMMISSION FOR UNESCO, DEPARTMENT OF STATE, 1961.

Slide 34

The National Debate onSchool Testing

  • What might have been the washback effect if students’ language abilities had been used to evaluate program effectiveness in 1961?

    • Perhaps clearer statements of objectives?

    • Perhaps more realistic learner expectations?

    • Perhaps more accurate course descriptions?

    • Perhaps qualitative feedback that would have justified longer course sequences?

Slide 35

After four decades, why are Parker’s projections not yet a reality?

  • Lasting improvements are only sustainable if there is a recognition of need for them.

  • Norm-referenced grading gave little evidence of real-world ability gaps.

  • Accreditation was focused on process rather than on outcomes, and provided no incentive to change.

  • Administrators didn’t read the research results showing that no improvements in teaching methods or curricula can compensate for the current lack of “time-on-task” in our educational system.

Slide 36

International Research on Learning French

  • John B. Carroll, The Teaching of French in Eight Countries (International Studies in Evaluation V) John Wiley & Sons, New York, 1975.

    • “… the primary factor in the attainment of proficiency in French (and presumably, any foreign language) is the amount of instructional time provided.” [Page 276]

Slide 37

International Research on Learning English

  • G. Bonnet, et al. The Assessment of Pupils’ skills in English in Eight European Countries: 2002. European Network of Policy makers for the Evaluation of Education Systems, 2004.

    • Students from those nations where there is more contact with (and more time spent using) English have higher levels of competence in English.

    • In a language-rich environment, time spent using the language is more important than the teaching methods used in the classroom.

Slide 38

Assessing FL Learner Outcomes

  • FL profession has made much progress in the area of assessment and standards.

  • Our ability to test students’ language proficiency puts us ahead of many other disciplines.

  • But proficiency tests are not the only way to measure learner outcomes.

  • Which test is “best” will depend on:

    • The purpose of the test.

    • Each program’s instructional goals.

Slide 39

Matching Tests with Your Instructional Purposes

  • Some typical FL testing purposes.

    • Assigning course grades in a beginning class.

    • Placement of students into a sequence of courses.

    • Certifying a general level of ability for teachers.

    • Screening of job applicants.

  • Some general instructional purposes.

    • Achievement – knowledge of contributing language elements found in a specific curriculum.

    • Performance – ability to communicate in specific settings.

    • Proficiency – demonstration of unrehearsed ability.

Slide 40

If Tests Are to bePositive Motivators

  • We have to select the right type of test for each testing purpose.

Slide 41

The 3 Major Types of Tests

  • Achievement = Memorized responses using the content of a specific textbook or curriculum.

  • Performance =Rehearsed ability to communicate in specific, familiar settings.

  • Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.

Slide 42

Some CommonTesting Purposes

  • Assigning grades in a class.

  • Placing students into a sequence of courses.

  • Selecting an applicant for a job with limited, static language requirements.

  • Screening employees for future jobs with broad, general language requirements.

Slide 43

Aligning testing programs with our testing purposes requires knowing what makes proficiency tests different fromperformance testsand achievement tests.

Slide 44

The ILR/ACTFL Proficiency Scales are Different, Because They:

  • Represent a geometric progression.

  • Use threshold rating criteria.

  • Do not use compensatory rating procedures.

  • Are a multidimensional measure of ability.

  • Apply Guttmann scaling criteria.

  • Are based on a wide range of “real world” communication needs.

  • Assess unrehearsed language abilities.

Slide 45

1. The ILR/ACTFL scales represent a geometric progression.

  • Advancing from one level to the next becomes increasingly difficult as one moves up the scale.

  • For instance, moving from 0 to 1 requires less time and effort than moving from 2 to 3.

Slide 46

What are the Major Levels?How are They Defined?

Slide 47

2. The ILR/ACTFL scales use “threshold” rating criteria.

  • Candidates must meet all of the stated criteria for a given level.

  • If they meet most, but not all of the criteria, some award a “plus” designation to the next lower level.

  • For example, someone who meets the criteria for Level 2 almost all of the time, could be rated 1+.

Slide 48

3. The ILR/ACTFL Ratings Are Not Based On Total, Average, or Other “Compensatory” Scores.

  • Ratings may be limited by one or more of the task, context, and accuracy requirements.

  • For example, a large vocabulary does not compensate for inaccurate usage.

Slide 49

Implications of Points 2 and 3: Assessment Criteria-Speaking

Slide 50

The ILR/ACTFL scales representa complex set of abilities.

  • The ILR/ACTFL scale is a multi-dimensional scale.

    • The scale is not uni-dimensional like a rope that has been cut into segments.

    • It is multi-dimensional, because each level of the scale represents a different constellation of supporting and enabling skills.

  • The ILR/ACTFL scale (and other complex scales) cannot be represented by a one-dimensional statistical model.

Slide 51

Bloom’s Cognitive Taxonomy

  • Evaluationand persuasion through refined use of professional, literary, and rhetorical skills.

  • Synthesisof concepts to produce and comprehend abstract ideas and hypothetical situations.

  • Analysisand comparisons of factual relationships in complex situations.

  • Applicationof skills to apply facts and understand new applications of those facts.

  • Comprehensionand facts learned.

  • Memorizationof facts.

Slide 52

Cognition and Language Taxonomy

  • Levels 4 & 5, evaluation and persuasionthrough refined use of professional rhetorical skills.

  • Level 3, synthesisof concepts to produce and comprehend abstract ideas and hypothetical situations.

  • Level 2, analysisand description of factual relationships in paragraph length communications.

  • Level 1, application of skills to create and understand new communications. (Performance)

  • Level 0+, comprehension and useof words and phrases. (Achievement / Performance)

  • Level 0, memorization of facts. (Achievement)

Slide 53

5. ILR/ACTFL Scale Ratings Apply Guttmann Scaling Criteria.

  • To qualify for any given rating, a person must meet the criteria for that level and also satisfy the criteria for all lower ratings.

  • For instance, a Level 2 (Advanced) speaker must also be able to satisfy Level 1 (Intermediate) task, topical, and accuracy communication requirements.

Slide 54

6. The ILR/ACTFL ratings are based on a range of “real world” communication needs.

  • The candidate must demonstrate sustained ability across a wide range of topics and real world settings.

  • For instance, proficiency ratings can not be assigned based on one’s:

    • knowledge of a textbook.

    • performance of a specific job task or set of tasks.

Slide 55

7. The ILR/ACTFL scalesassess unrehearsed abilities.

  • Broad elicitation techniques are used to verify sustainable ability versus limited performance.

  • For instance,

    • At level 1 (Intermediate), test takers must create new utterances, not just recite dialogs.

    • At level 2 (Advanced), they must handle new work requirements, not just routine communications.

    • At level 3 (Superior), they must hypothesize and defend opinions in subjects beyond their personal interests and areas of specialization.

  • “Proficiency” indicates transferable skills – not just rehearsed performance in familiar areas.

Slide 56

A Summary of the Contrasts Between Proficiency And Rehearsed Performance

Unrehearsed Rehearsed

ProficiencyPerformance

Task: A wide range of abilities Specific Job Skills

Context: Broad, in-depth, variable Focused, restricted

Accuracy: Ascending expectations Situation dependent

Slide 57

So what?

Slide 58

Review of Major Test Types

  • Achievement = Memorized responses using the content of a specific textbook or curriculum.

  • Performance =Rehearsed ability to communicate in specific, familiar settings.

  • Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.

Slide 59

A Self Assessment Quiz

  • You will be asked about 6 different testing purposes.

  • For each of those test purposes, which type of test would you choose? a. Achievement.

    b. Performance.

    c. Proficiency.

Slide 60

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

Slide 61

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

Slide 62

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

Slide 63

Which type of test would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

Slide 64

Which test type would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

  • To select someone to be the NATO spokesperson on a news show with a “hostile” moderator?

Slide 65

Which test type would you choose:Achievement, Performance, or Proficiency?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

  • To select someone to be the NATO spokesperson on a news show with a “hostile” moderator?

  • To document a person’s language ability in that person’s official personnel file?

Slide 66

Achievement, Performance, or ProficiencyWhich test type did you choose?

  • To assess students’ language learning after Chapter 3 of a beginning language course?

  • To place students into a university’s sequence of courses?

  • To test students completing a year-long, intensive language course?

  • To screen job applicants for a specific job with well-defined, repetitive tasks?

  • To select someone to be the NATO spokesperson on a “hostile” news show?

  • To document a person’s language ability in that person’s official personnel file?

Slide 67

Solving Testing Problems

  • “The solutions to our problems should be as simple as possible, but no simpler.”

    Albert Einstein

  • There is no answer for the overly simple question of “Which test is best?”

  • There is an answer to the question, “Which type of test is best for a given purpose?”

Slide 68

Which test is best?

  • The test that matches the purpose for which the results will be used.

    • Use achievement tests for testing mastery of lessons in a textbook.

    • Use performance tests for checking rehearsed abilities within specific contexts.

    • Use proficiency tests for determining general, unrehearsed ability in real-world situations.

Slide 270

Why are programs not assessed?

  • It would be too much work.

  • It is not “my thing.”

  • We’d rather not know the results.

Slide 271

Why assess our programs?

  • In today’s environment – if you don’t measure it, you can’t get money for it.

  • Assessment will help us articulate our objectives.

    • For ourselves.

    • For the students.

    • For program sponsors.

  • Formative assessment allows informed decision making.

Slide 272

The Improvement Process

  • Set goals.

  • Decide how to measure progress toward those goals.

  • Collect data.

  • Analyze the results.

  • Make informed changes.

  • Repeat.

Slide 273

It won’t be easy, but if we are willing to work together and learn from the past…

then William Riley Parker’s vision of second language programs in the U.S. might yet become a reality!

Slide 274

In the field of second language teaching, the U.S. has a long history of missed opportunities.

  • Only if we are willing to document the status of our programs, can we hope to improve them.

  • With limited resources, collaboration will be more productive than competition.

Slide 275

Conclusion

  • The past has not been perfect,

  • and the future is conditional

  • on whether the present is progressive.


Copyright © 2014 SlideServe. All rights reserved | Powered By DigitalOfficePro