Russian Proficiency Assessment at Indiana University SWSEEL

1. Russian Proficiency Assessment at Indiana University SWSEEL Mark Trotter Indiana University Russian and East European Institute

2. Indiana University Summer Workshop in Slavic, East European, and Central Asian Languages (SWSEEL) • 15-20 languages offered each summer, including most of the major national languages spoken between Slovenia/Czech Republic and Western China/Kazakhstan • http://www.iub.edu/~swseel/program/languages.shtml • 8-week course of study (equivalent to 2-semesters of AY study) • 20-23 classroom hours/week • extensive supplementary cultural program • Approximately 200 students enroll each summer • Student funding available: FLAS, Title VIII, ACLS

3. RUSSIAN AT SWSEEL • Roughly 50% of SWSEEL students • Offered at 9 different levels (Years 1-6) • For most levels, students attend: • Core class in 4 skills plus grammar/lexicon (15 hours/week, 3 hours/day) • Conversation practice (4 hours/week) • Phonetics (2 hours/week) • Listening comprehension (2 hours/week) • Cultural program includes: • Weekly Russian-language lecture for all students • Weekly Russian-language film • Russian language lunches (2/week) • Opportunities to perform in Russian choir or drama group, write for weekly Russian newspaper

4. STANDARDIZED ASSESSMENT 1. SWSEEL placement test 2. ACTFL Oral Proficiency Interview (OPI) 3. IEPS/IRIS Instructor Foreign Language Evaluation

5. SWSEEL PLACEMENT TEST: CONTENT • Grammar: sentence completion task, sentence construction task, multiple choice questions (60%) • Reading comprehension: texts and multiple choice comprehension questions (20%) • Written production task with visual prompt, students choose task on basis of desired level (20%)

6. SWSEEL PLACEMENT TEST: ADMINISTRATION • Required for all students • Given prior to beginning of program, either at IU or through proctor on other campuses • Results determine initial placement of students into SWSEEL Russian levels • All students take same test at end of program

7. ACTFL ORAL PROFICIENCY INTERVIEW: CONTENT • Structured interview that determines patterns of strengths and weaknesses to establish candidate’s consistent functional ability in spoken use of the language • Interview consists of specific language tasks (sometimes w role-play) ; performance on each is compared with criteria for each of the ten proficiency levels described in the ACTFL Proficiency Guidelines-Speaking • Testers must undergo extensive special training in order to receive certification from ACTFL

8. ACTFL ORAL PROFICIENCY INTERVIEW: ADMINISTRATION • Required of all FLAS awardees • OPIs conducted during initial and final week of classes (pre- and post-test) • ACTFL-certified testers recruited by REEI, do NOT teach in the SWSEEL program

9. IEPS/IRIS INSTRUCTOR FOREIGN LANGUAGE EVALUATION: CONTENT • 35-item questionnaire composed of statements about spoken proficiency of testee; evaluators mark yes/no for each question • Items focus on microskills that characterize functional command of the language at various levels (novice to near-native): The student can order a simple meal The student can describe in detail a place that is familiar to him/her The student can naturally integrate appropriate cultural and historical references in their speech The student can eloquently represent a point of view other than their own

10. IEPS/IRIS INSTRUCTOR FOREIGN LANGUAGE EVALUATION: ADMINISTRATION • Required of all FLAS awardees (IEPS mandate) • In most cases, supplied by Conversation instructor and on basis of in-class observation • Pre-program (submitted in wk 1 or 2) and post-program (submitted in wk 8 or post-program) evaluations

11. ACTFL OPI: RESULTS (SWSEEL RUSSIAN/2007-2010) • 120 OPI-tested students • 87/120 improved over summer by at least one level w/in a given range • 20/26 moved from Novice range into Intermediate range • 14/83 moved from Intermediate range into Advanced range

12. NEW DIRECTIONS FOR 2011-2014: CURRENT ASSESSMENT TOOLS  • Deeper and broader analysis of test data to: • determine correlations between performances on different tests • refine curricula of SWSEEL Russian courses so as to incorporate feasible realistic proficiency goals • draw up profiles of habits, aptitudes, backgrounds that differentiate learners who start program at same proficiency range/level and finish at different range/level (how does an Intermediate to Intermediate learner differ from an Intermediate to Advanced learner)

13. NEW DIRECTIONS FOR 2011-2014: OTHER ASSESSMENT TOOLS • Test of Russian as a Foreign Language • Development of Russian c-test

14. SAMPLE C-TEST (English) Throughout early childhood training we are taught that our attitudes, behavior and opinions must be consistent. That is, things we do must make sense. We exp___ our fri____ not t_ do thi___ that a__ unpleasant f__ us, a__ we a__ suspicious i_ a per___ we dis____ does some_____ nice f__ us. W_ do n__ expect peo___ who d_ not bel____ in G__ to g_ to church unless there is a good reason, such as liking the music or wanting to get to know people in the community. We are usually rather surprised if people do or say things that do not fit in with what we already know about them. http://www.c-test.de/deutsch/index.php?lang=de&content=online_test_english&section=ctest