Measuring Student Learning at the Classroom Level: A Longitudinal Study

1. Measuring Student Learning at the Classroom Level: A Longitudinal Study Aleidine Moeller Leyla Masmaliyeva Dallas Malhiwsky University of Nebraska-Lincoln

2. Research Focus A longitudinal study at the classroom level that attempts to more accurately depict students’ acquisition of language proficiency that ultimately could contribute to a more accurate and smooth articulation tool for documenting language progress.

3. The Need: Clearly Articulated Descriptors • What students know and are able to do at the end of each year of classroom language study • Accurate and smooth articulation requires empirically tested common yardstick at the classroom level that defines language progression from one level to the next in a cohesive manner

4. Smooth Articulation • Accurate and smooth articulation requires an empirically tested common yardstick at the classroom levelthat defines language progression from one level to the next in a cohesive manner.

5. ACTFL constructed a chart approximating what students should know and be able to do given specific years of language study, however, this chart does not reflect classroom based research, but rather is based on proficiency interviews with individual language students and “on information gathered from foreign language professionals representing a variety of program models and articulation sequences” (ACTFL Performance Guidelines for K-12 Learners, 1998).

6. Classroom Based Evidence • There is a dearth of classroom tested studies that have investigated the level and rate at which language learners achieve proficiency after a sequenced course of language study. • Classroom-based evidence has been limited to a few studies restricted to one or two classrooms that do not allow for generalizable results.

7. Subjects 7-12 Teachers of Spanish 2 week Spanish Immersion funded by Teacher Quality Grants • Language, literature, culture pedagogy, technology, LinguaFolio • Teacher participants implemented LF (120 teachers) • Selected 17 • Followed teachers & students for 5 years

8. Context for the Study: LinguaFolio • Modeled after European Language Portfolio (ELP) of the Council of Europe (COE) • Adapted by National Council of State Supervisors of Foreign Languages (NCSSFL) • Promote reflective, autonomous language learning and cultural interactions through a self-assessment tool • http://linguafolio.unl.edu

9. LinguaFolio Nebraska • seeks to inform students about themselves and their own learning, showing them exactly where they are in relation to learning goals • with the help of their teachers, knowing which steps to take to achieve these goals. • through systematic self-assessment, goal setting, feedback and reflective learning tools, the LinguaFolio guides students to monitor and map their own learning.

10. Students must make what they learn part of themselves (Chickering & Gamson, 1987, p. 3) • Talk about what they are learning, write about it, relate to past experiences, apply in their daily lives • Too often we see students as subjects of assessment, not users of assessments. The student must be considered the primary user of assessment information.

11. Promote Self-regulation • How do we position students to think about their own learning and make connections inside and outside the classroom to monitor their learning development?

14. Pilot Test: Lessons Learned 10 teachers of Spanish 7-12 Tested & revised instruments teacher observation form teacher beliefs & attitudes Goals: needed training Development of Teachers Handbook Online LinguaFolio

15. Why goal setting? “Goal setting influences learning and motivation by providing a target and information about how well one is doing.” (Alderman, 2005, pg. 105)

16. Goal setting → increased self-efficacy→ increased intrinsic motivation Goal setting: • Monitor progress Increased self-efficacy • provides expectation for success and increases the likelihood of engaging in future tasks without external incentives Increased intrinsic motivation

17. Characteristics of goals Performance vs Learning based Goals Performance- focus on comparing oneself to others Learning- focus on gaining new knowledge and skills

18. Characteristics of goals Short vs long term • Near future versus distant future • Both needed

19. Characteristics of goals Easy vs. difficult Analysis of 110 studies published from 1969 to 1980 demonstrated that 99 studies showed that difficult learning goals produced better performance than medium, easy, do-best, or no goals at all (Griffee and Templi,1997).

20. Characteristics of goals Specific vs. Broad Goals • Students setting too specific goals may focus their attention to such a degree that they will ignore “information not specifically related to their goal” • (Marzano et al, 2001, p. 94). • Broad goals are vague and have a nonspecific outcome

21. Characteristics of goals Assigned versus Self-chosen Goals • Assigned goals foster motivation because they imply that the individual is capable of achieving the goal or task • Self-chosen- power of ownership • Crucial factor in assigned goals Is acceptance- once involved in a goal, the goal itself becomes more important than how it was set or whether it was imposed

22. Simply goal setting is enough? Many educators overestimate their students’ ability to set high quality goals (Boekaerts, 2002). “A perceived discrepancy between performance and the goal may create dissatisfaction” (Schunk, 2003, p. 163).

23. Examples of students’ poor goals: • “speak fluently” • “understand everything” • “get an A (grade)”

24. For goals to enhance student achievement goals should meet specific criteria (Schunk, 2003).

25. SMART Goals • Specific • Measurable • Agreed upon • Realistic (challenging yet realistic) • Time-based

29. SMART+ER =SMARTER Return to Your Goals • What evidence do you have that indicates whether you met your goals? • What could you have done to better meet your goals? • How could you have restated your goals in order for them to be more appropriate? • Are you satisfied with your progress today? Did you put forth your “best effort” to achieve your goals?

33. Formula for Success: Setting goals +documenting progress +self-assessing learning =student achievement

34. Results of Goal Setting on Student Achievement Correlation between goal setting and student achievement at .01 level of significance in reading, speaking and writing

37. Assessment for Learning • Students engage in assessment for learning when they use assessment information to learn how to judge the quality of their own work and set goals for their own improvement.

39. This interactive LF NE website • students log in and establish base line knowledge and skills from which to build long term goals • Self-assessment of competency chart: identify what they are able to do in their learning process while also reflecting upon the level of language competency they would like to attain

43. Measuring Student Learning • STAMP • Archived student samples • Proficiency based • Accessible • Provided a control group • Measures speaking, writing and reading

44. STAMP • Teacher independent proficiency assessment that archives language skills over time • Access to all reading, writing, speaking scores and individual student samples over time

45. STAMP • A summative assessment tool, measuring student proficiency levels in foreign languages in four skills • Test items are based on benchmarks linked to ACFTL guidelines • Securely and efficiently delivered through the internet • Computer adaptive online assessment

46. STAMP Report can be displayed in terms of a class, group or individual Is criterion-referenced (item evaluation is based on how they perform as students approach and then cross a mastery level)

47. STAMP • Assessment is made on basis of what students can actually perform in real-world context • Provides detailed and user-friendly report that aims to demonstrate that the best measure of educational success is progress in student performance. • The ultimate purpose is to improve language teaching and learning • Gives the teachers concrete feedback on students overall proficiency and breakdowns of performance according to topic and task