The Summary of Answers and Skills (SOAS) Tutorial

1. The Summary of Answers and Skills (SOAS) Tutorial

2. SOAS Introduction Audience • Administrators • Department heads • Curriculum specialists • Teachers • Other staff interested in students’ college preparatory skills Note: the more inclusive the testing within each grade, the more meaningful and applicable the results.

3. Logging In Repeat Users www.collegeboard.com/results If you’ve accessed SOAS reports before, just enter your username and password.

4. Logging In New Users If it’s your first time, you’ll need your school’s SOAS access code. Access codes can be found on your Roster of Student Scores and Plans.

5. Your Reports • Links to your reports will be available in the box (see below) once you are logged in. • Click on the plus sign to show all the reports available to you. Each link is a separate report for an individual grade level. • Reports are provided if your school tested 25 or more students in a grade.

6. Report Orientation Table of Contents Page(s) • Title Page • Performance Overview (pg 1) • Skills Analysis (pgs 2, 6-7, 11) • Question Analysis (3-4, 8-9, 12-13) • Comparable Group Analysis (5, 10, 14) Skills Analysis, Question Analysis, and Comparable Group Analysis pages are repeated for each section of the test: Critical Reading, Math, and Writing.

7. Performance Overview Page 1 • Number of students in report • Mean scores and score distribution • College Readiness Benchmarks

8. Skills Analysis Pages: 2, 6-7, 11 • Skills • Triangles show a comparison to the state and nation • Each test question is linked to a skill

9. Question Analysis Pages: 3-4, 8-9, 12-13 • A visual display of student responses and answer patterns • Correct answers are shown in the gray portion of the bar chart • Comparisons to the state and nation are provided

10. Question Analysis Tips: Look for questions your students answered incorrectly that thestudents in the state and nation answered correctly. Look for commonly wrong answers.

11. Comparable Group Analysis Pages: 5, 10, 14 Darker blue = significantly below the comparable group Lighter blue = significantly above the comparable group

12. SOAS Comparable Group What is a comparable group? • A useful statistical model. • A statistically created group (virtual group). • Mirrors your group’s performance profile. • This creates an expected performance indicator for your group on each question. • Provides more “actionable” feedback than state or national averages on questions/skills.

13. Supporting Resources Resources • Several resources are available to help you optimize use and understanding of SOAS reports. • Free Webinars • Tutorials and a video • Skills aligned to state standards • Test Question and Answer Explanations (see next slide)

14. Supporting Resources Test Questions and Answer Explanations 2. View full test question details 1. Choose test questions

15. How does it all add up? Step 1 of 4: Identify a question for review

16. How does it all add up? Step 2 of 4: Find the skill measured by this question

17. How does it all add up? Step 3 of 4: View the question details

18. How does it all add up? Step 4 of 4: See your state standards aligned to this question/skill

19. Other Relevant Resources:Before, During, and After SOAS review

20. Understanding PSAT/NMSQT Scores • Score related FAQ’s • Information about percentiles and mean scores • Score conversions This publication is mailed with score reports in December and is also available online at www.collegeboard.com.

21. PSAT/NMSQT Summary Reports – State Level Reports are available at www.collegeboard.com

22. PSAT/NMSQT Summary Reports – School Level Summary Reports are provided to all schools that tested at least 50 juniors or sophomores. State level summary reports are also available (College Bound Juniors/Sophomores reports) www.collegeboard.com/reporting. Reports contain: • number of test-takers • ethnic background • participation in Student Search Service • self-reported GPA • performance by score range • college major

23. SOAS Activities for Educators

24. Activity AReview Skills Analysis Instructions • Work within your own school group/academic area, using your school’s SOAS report. • Identify the skill that your students had the lowest performance on. • Using the Test Question and Answer Explanations, look at each question aligned to that skill. • Things to look for: Skills furthest below the state/national average. Skills that are furthest to the left (i.e. those that are the weakest for the group). Skills that came to teachers’ attention after review of Question Analysis.

25. Activity A Review Skills Analysis Questions for Consideration Within your group, answer the following questions: • Is there a pattern in the skills you identified as problematic? • Are there actions or strategies that might help address the issues?

26. Activity BReview Question Analysis Instructions • Work within your own school group/academic area, using your school’s SOAS report. • Refer to the Test Questions and Answer Explanations to make note of problem questions and how they might be addressed. • Things to look for: Easier questions (difficulty level 1-3) where less than a 2/3 of the students answered correctly. A larger proportion of your students answered incorrectly or omitted an answer in comparison to the state/nation. Commonly wrong answers. Review any questions that generated teacher interest when you took the test in Activity A.

27. Activity B Review Question Analysis Questions for Consideration Within your group, answer the following questions: • Are there any patterns in the questions you identified as problematic? • Are there actions or strategies that might help address the issues? • Summarize your findings for sharing.

28. Activity C Comparable Group Analysis Instructions • Work within your own school group/academic area, using your school’s SOAS report. • Using the Test Question and Answer Explanations, look at each question. • Things to look for: Questions that fall into the dark blue area, where your students performed below expectation. Questions that fall into the light blue area, where your students performed higher than expectation.

29. Activity C Comparable Group Analysis Questions for Consideration Within your group, answer the following questions: • Is there a pattern in the skills aligned with the test questions your students performed below expectation on? • Is there a pattern in the skills aligned with the test questions your students performed expectation on? • Are there actions or strategies that might help address the issues?

30. For Group Discussion To Help Students and Parents 1. How does the school educate parents about the wealth of information on Score Report Plus? 2. What plan does the school have in place to make sure students receive and understand their scores?

31. For Group Discussion Using Your SOAS Data • Identify the skills associated with the questions students answered incorrectly. • Make inferences about what led to errors. • Identify where in the curriculum the skills are (or are not) taught. • Work with departments and classrooms to review common concerns. • What recommendations can you make to your school for increased student success?

32. Looking Ahead:2011 PSAT/NMSQTWednesday, October 12thSaturday, October 15th