Peggy Garza Associate BILC Secretary For Testing Programs

1. BILC Standardization Initiatives and Conference Objectives Peggy Garza Associate BILC Secretary For Testing Programs

2. Standardization Efforts - Language Testing Seminars - Tailored Workshops - Toolbox for Language Testers - STANAG 6001 Testing Conferences Conference Topics and Goals Outline

3. Language Testing Seminar (LTS) Advanced Language Testing Seminar (ALTS) Language Standards and Assessment Seminar (LSAS) BILC Testing Seminars

4. LANGUAGE TESTING SEMINAR (LTS)

5. Language Testing Seminar (LTS) Developed and conducted by BILC testing experts Background: A BILC initiative aimed at promoting a common interpretation of STANAG 6001 and consistency in language proficiency testing Specific objective: To give NATO/PfP language teaching professionals the opportunity to practice developing test items based on STANAG 6001

6. Language Testing Seminar (LTS) Held since Autumn 2000 at the Marshall Center, Garmisch, Germany Two-week graduate-level seminar Offered twice times a year (February and September) More than 350 graduates from NATO/PfP/MD nations

7. ADVANCED LANGUAGE TESTING SEMINAR (ALTS)

8. Advanced Language Testing Seminar (ALTS) • Need for continuing professional development for test developers identified • Stakeholders’ meeting in September 2008 • Result: Advanced Language Testing Seminar (ALTS) • Validated in 2009

9. Target audience: LTS graduates who continue to work on STANAG 6001 testing - STANAG 6001 Edition 4 standardization sessions - Focus on Level 3 for practical exercises - Opportunity to exchange best practices Modular approach to seminar Advanced Language Testing Seminar (ALTS)

10. Pre-seminar assignments - Approximately 40 hours Three-week seminar - Module 1: Receptive Skills: Listening and Reading (1 week) - Module 2: Productive Skills: Speaking and Writing (1 week) - Module 3: Test Analysis (3 days) - Module 4: Managerial Issues (2 days) Advanced Language Testing Seminar (ALTS)

11. NEW LANGUAGE STANDARDS AND ASSESSMENT SEMINAR (LSAS)

12. Language Standards and Assessment Seminar (LSAS) 3-11 July 2013 For language teachers and program managers Objectives STANAG 6001 familiarization Guidelines for aligning instruction to standards Test and measurement principles Classroom assessment and practical exercises

13. Testing Seminar Schedule in 2014 Language Testing Seminar (LTS) 10-21 February 15-26 September Advanced Language Testing Seminar (ALTS) 2- 20 June Language Standards and Assessment Seminar (LSAS) 23 Apr – 1 May 3-11 December

14. Recommended by ALTS participants Samples of listening and reading items at levels 1-3 Samples of speaking and writing tests at levels 1-3, including plus levels Frequently Asked Questions (FAQs) Self-Assessment Outline for STANAG 6001 Testing Programs STANAG 6001 Toolbox for Testers

15. BILC Website www.natobilc.org

18. Tailored Workshops for National Testing Programs

19. Ukraine November 2011, 2012 Moldova May 2012 Russia June 2013 Tailored Workshops

20. Zagreb, 2009 Focus on standardization, moderation, and validation Sarajevo, 2010 Dates: 31 August – 2 September Focus on testing speaking Stockholm, 2011 Dates: 6-8 September Focus on testing writing Copenhagen, 2012 Dates: 3-6 September, 2012 Focus on testing listening STANAG 6001 Testing Conferences

21. 2013 STANAG 6001 Testing Conference Some of your questions: Our stakeholders have questions. Is there advice for how best to communicate with the stakeholders? Our stakeholders are asking us to test more and more people. What should we tell them? We have developed items. What’s next? How do we know if our tests are good? We need to hire new testers. Is there a position description for testers? How can we recruit for testers? How can we keep professionally current ?

22. 2013 STANAG 6001 Testing Conference Conference format Presentations Guest speakers from NATO and SHAPE Panel discussions Resolving challenges Study Groups meetings and reports Making a validity argument Washback Action plans Renorming and standardizing testers Piloting items and prompts

23. Specific Conference Goals Review and clarify the entire test construction process Identify action plans for test development and administration Provide ideas for selecting the appropriate testing techniques Discuss piloting and pretesting Learn more about determining test validity Identify practicality issues and discuss lessons learned Discuss best practices in renorming and standardizing testers Learn about innovations in testing in the BILC community Computer-delivered testing Proficiency and specialist testing Testing at Level 4

24. Test Life Cycle Diagram

25. Communicating with the Stakeholders • Identify the stakeholders • Coordinate with the stakeholders during the needs analysis and planning stages • Stakeholders approve personnel and resources • Close the loop with the stakeholders during the Test Life Cycle • Action plans • Technical reports • Updates • Washback

26. Test Construction NeedsAnalysis Stage 1 Planning Stage 2 Design Stage 3 Test Specifications Revision Development Stage 4 Pre-testing / Analysis Standard Setting Evaluation/review Administration Stage 5 Monitoring Stage 6 Evaluation

27. Development Stage Part 1 Text selection Itemwriting Item review I (moderating) Piloting on a small scale • Retain • Revise • Reject Item review II (editing) More items needed? Assembly of trial test

28. Development StagePart 2 Pre-testing/trialling on a larger scale Statistical Analyses • Retain • Revise • Reject Item revision Standard Setting Item banking Assembly of final version of test

29. 2013 STANAG 6001 Testing Conference Expectations Active participation and interaction Frank and open discussions Provide answers to the difficult questions Create a forum for future networking and cooperation BILC appreciates your participation and we look forward to a very productive conference!