Interrelationship between assessment and instruction
Download
1 / 48

Interrelationship between Assessment and Instruction - PowerPoint PPT Presentation


  • 115 Views
  • Uploaded on

Interrelationship between Assessment and Instruction. Dr. Liying Cheng Faculty of Education, Queen’s University [email protected] Overview. How we define the interrelationship between assessment and instruction How we define learning and learning targets

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Interrelationship between Assessment and Instruction' - holden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Interrelationship between assessment and instruction

Interrelationship between Assessment and Instruction

Dr. Liying Cheng

Faculty of Education, Queen’s University

[email protected]


Overview
Overview

  • How we define the interrelationship between assessment and instruction

  • How we define learning and learning targets

  • Key components of classroom assessment

  • Project 1 – teacher classroom assessment

  • Project 2 – large-scale testing and second language students


Aeg faculty of education queen s u
AEG – Faculty of Education, Queen’s U

  • Assessment and Evaluation Group (AEG) is an evaluation research and consulting group at the Faculty of Education, Queen’s University. AEG operates in many contexts, at many levels, with many methods of inquiry all directed at the improvement of learning.

  • I acknowledge the use of certain slides from my colleagues – Don Klinger and Lyn Shulha – in this seminar.


Rationale key relationship
Rationale: Key relationship

Teaching

Learning

Assessment


Interrelationship
Interrelationship

  • Impact of assessment (testing and public examinations) on classroom instruction (teaching and learning) – washback research (Latham, 1877; Li, 1990; Alderson & Wall, 1993; Cheng, Watanabe, with Curtis, 2004)


Interrelationship1
Interrelationship

  • Links between assessment and instruction in terms of authenticity and congruence of assessment practices in relation to a program of study (Douglas, 2000; Bachman & Cohen, 1998)

  • Assessment of student outcomes in term of curriculum and teaching (Johnston, 2000)


Interrelationship2
Interrelationship

  • Teacher assessment where the teacher is the agent of assessment - conducting both formative and summative assessment – linking to the purposes of assessment (Brindley, 2001; Brindley, 2007; Cheng et. al. 2004; Rea-Dickins, 2004).


What is learning
What is Learning?

A curricular definition

Learning - acquiring knowledge, skills, attitudes, and values and experiences.

A cognitive definition

Learning - a process of formulating new and more complex understandings of the world

Learning - revising and refining mental constructs, i.e., the understandings that guide how we think, speak and behave


Wilson s taxonomy for teachers
Wilson’s Taxonomy for Teachers

Novice

  • A journey from novice to expert

  • Helps you monitor how students are thinking & performing

  • Cues you about the kinds of questions or activities that can help students think more deeply about their learning?

I.C.E.

Ideas

  • Connections

    • Within content

    • To self

Extensions

Other Contexts

Expert


Learning targets mcmillan 2004
Learning targets (McMillan, 2004)

  • Indicate 1) what a student is to know and/or do as a result of instruction and 2) the criteria for evaluating the performance (criteria)

    • Knowledge and simple understanding

      • Declarative (know what)

      • Procedural (know how)

    • Deep understanding and reasoning

    • Skill

    • Product

    • Affect


Classroom assessment
Classroom assessment

  • The collection, evaluation, and the use of information to help teachers make better decisions that improve student learning.

  • Classroom assessment is more than testing and measurement.

  • The fours essential components to implementation classroom assessment are

    • Purpose

    • Measurement

    • Evaluation

    • Use


Classroom assessments
Classroom Assessments

  • Developing and Choosing Methods for Assessment

  • Collecting Assessment Information

  • Judging and Scoring Student Performance

  • Summarizing and Interpreting Results

  • Reporting Assessment Findings

    Principles for Fair Student Assessment Practices for Education in Canada (1993)


Teach and e learning grants
TEACH and e-Learning grants

  • Classroom Assessment: Theory and Practice

  • Computer-Based Assessment Development

  • Researching Teaching English as a Second Language



B ed instructors
B Ed Instructors

  • The instructors commented much of their time with the B Ed candidates will be spent exploring

    • practicum expectations,

    • professional conduct,

    • classroom procedures and routines,

    • shaping instruction for individual, small group and whole class activities,

    • motivating students,

    • how to approach issues in classroom management, and

    • Emphasize issues in assessment and evaluation.


Associate teachers
Associate Teachers

  • All of the associate teachers suggested that candidates need some familiarity with

    • rubrics,

    • making judgments about student learning,

    • achievement chart categories, and

    • the four levels of achievement.

  • Many suggested that it would be an asset to be able to create rubrics. This requires some understanding of how to differentiate a Level 2, Level 3, and Level 4 response to a learning task.


School district administrators
School District Administrators

  • They expressed a strong interest in seeing B Ed candidates grapple with the concept of assessment for learning (formative assessment).

  • When they also motioned the need to construct grades (summative assessment), they suggested that candidates have a strong understanding of how to determine grades from a variety of instruments  and achievement charts and how to identify which of these represent most recent and consistent performance. 

  • Matching assessment tools to Ministry achievement charts and the ability to do performance assessment rounded out their suggestions to this year's class.  




Classroom assessment theory and practice
Classroom Assessment: Theory and Practice

  • The Fallibility of Assessment

  • What Does Learning Look Like?

  • Assessment of Learning

  • Assessment for Learning

  • Developing Rubrics

  • Developing Assessment Instruments

  • The Multiple Perspectives of Assessment


Differences in large scale testing

USA has

Standardized Testing

No set curriculum.

Separation of assessment and instruction.

No attempt to involve teachers in interpreting the responses and results.

Teachers administer the tests because they have to.

Canada has

Large-scale Assessment

Provincial curriculum.

Assessment are based on learning expectations and are integrated into instruction.

Teachers are involved with test design, development, scoring and interpreting.

Teachers administer the tests because they have to and because schools have growth plans and resources linked to assessments.

Differences in Large-scale testing


The myths of testing stiggins 2004
The Myths of Testing(Stiggins, 2004)

Testing motivates students to learn

  • If I threaten you will fail and then you will try harder

  • If a little intimidation does not work, try more intimidation

  • Maximize anxiety to maximize learning

    Testing helps teachers make important instructional decisions

  • Students are not assessment users

    Important assessment decisions can be made once a year

  • Investment of time, effort, and money into large-scale testing supports this belief

    Learning how to assess is not as important as learning how to teach

  • Teachers teach and testing professionals test

  • Assessment literacy


The challenges of the ontario secondary school literacy test osslt for esl eld students

The Challenges of the Ontario Secondary School Literacy Test (OSSLT) for ESL/ELD Students

Cheng, L., Klinger, D., & Zheng, Y. (2007). The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing, 24(2), 1-24.


Research context
Research Context (OSSLT) for ESL/ELD Students

  • The increasing number of English as a Second Language (ESL) and English Literacy Development (ELD) students in Ontario secondary schools. Immigration to Ontario increased 23% vs. ESL teachers and programs in Ontario schools declined 30%

  • Large-scale educational testing has increasingly been used to measure and ensure student competency or provide system accountability(Firestone, Mayrowetz, & Fairman, 1998; Ryan, 2002).


Research context1
Research Context (OSSLT) for ESL/ELD Students

  • These large-scale tests are constructed and normed for first language (L1) English speakers. Research suggests, however, that they may have lower reliability and validity for L2 students and should be interpreted differently (Abedi, Leon, & Mirocha, 2003).

  • The confluence of both increased numbers of L2 students and this expanding testing framework in schools has created a new and largely unanticipated educational problem – alarmingly high failure rates of these students (Watt & Roessingh, 2001).


Esl eld students performances in the osslt
ESL/ELD Students (OSSLT) for ESL/ELD Students’ Performances in the OSSLT

  • Of all the eligible ESL/ELD students, only 46% in Feb. 2002, 45% in Oct. 2002, and 54% in October 2003 participated in the test administration.

  • The pass rate of ESL/ELD students in the last three test administrations are 37%, 34% and 42% respectively. While the total pass rate of all students are 75%, 72% and 77% respectively

  • ESL/ELD students were failing the Ontario Secondary School Literacy Test (OSSLT) at a rate higher than any other group of students

    • Pass rates: 42% vs. 77% in 2003

    • Participation rates: 54% vs. 91% (EQAO, 2004)


The ontario secondary school literacy test osslt

The Ontario Secondary School Literacy Test (OSSLT) (OSSLT) for ESL/ELD Students

“The OSSLT is a useful quality assurance measure that shows the extent to which Ontario students are meeting a common, basic standard for literacy across the province” (EQAO[1], 2002, p. 1). The OSSLT is developed by the Education Quality and Accountability Office (EQAO) - an independent agency operating within a framework established by the provincial government of Ontario.

[1]We acknowledge the support from EQAO for releasing the February 2002 OSSLT data for this study.


The ontario secondary school literacy test osslt1
The Ontario Secondary School Literacy Test (OSSLT) (OSSLT) for ESL/ELD Students

  • All students in public and private schools working toward an Ontario Secondary School Diploma are required to write the OSSLT in Grade 10. Students who have been eligible to write the OSSLT at least twice and have been unsuccessful at least once are eligible to fulfill the requirement through the Ontario Secondary School Literacy Course (OSSLC). Successful completion of the OSSLT or OSSLC is a graduation requirement.


The ontario secondary school literacy test reading
The Ontario Secondary School Literacy Test (Reading) (OSSLT) for ESL/ELD Students

Reading component - a total of 12 short selections in three different text types:

  • information (e.g., explanation, opinion) 50%;

  • graphic (e.g., graph, schedule, instructions) 25%;

  • narrative (e.g., story, dialogue) 25%.

    in three test formats:

  • multiple-choice (40 questions);

  • constructed response (35 questions);

  • constructed responses requiring an explanation (25 questions).


The ontario secondary school literacy test reading1
The Ontario Secondary School Literacy Test (Reading) (OSSLT) for ESL/ELD Students

  • three reading skills:

    • understanding directly stated ideas and information;

    • understanding indirectly stated ideas and information;

    • making connections between personal experiences and ideas and information in a selection.

  • Four reading strategies:

    • vocabulary,

    • syntax,

    • organization,

    • graphic features.


Scoring criteria reading
Scoring Criteria (Reading) (OSSLT) for ESL/ELD Students

  • Multiple choice items are scored separately.

  • Constructed Response (CR) questions ask students to respond in a few words, and answers are marked correct (1 point) or incorrect (0 point).

  • Constructed Response with Explanation (CRE) questions ask the students to justify or explain the thinking behind their answers. And the answers are marked correct (1 point), partly correct (0.5 point) or incorrect (0 point)


The ontario secondary school literacy test writing
The Ontario Secondary School Literacy Test (Writing) (OSSLT) for ESL/ELD Students

Writing - four writing tasks:

  • a summary,

  • a series of paragraphs expressing an opinion,

  • a news report, and

  • an information paragraph.

    Students need to pass both components of the OSSLT.

    OSSLT is administered over a two-day period.


Scoring criteria writing
Scoring Criteria (writing) (OSSLT) for ESL/ELD Students

  • Two steps in the marking of writing--holistic scoring and analytical scoring.

  • Holistic scoring (0-4 points)

  • If the student performances do not meet the requirements to pass, then all four writing tasks will go on to step 2: analytic scoring, evaluated by four separate characteristics:

    • Main idea

    • supporting details,

    • organization,

    • Spelling, grammar and punctuation.


Research framework
Research framework (OSSLT) for ESL/ELD Students

  • Research studies have shown that the test items, the content, types and context of reading passages, and the relationships of these intervening factors can have a significant impact on students’ performance (Anderson et al., 1991; Freedle & Kostin, 1993; Kobayashi, 2002, Lee, 2002; Peretz & Shoham, 1990; Perkins, 1992).

  • Research into writing assessment has demonstrated that different types of writing tasks create different challenges for students (e.g., Connor-Linton, 1995a, 1995b, Kobayashi & Rinnert, 1996, Hamp-Lyons, 1996).


Research questions
Research questions (OSSLT) for ESL/ELD Students

  • Compared with Non-ESL/ELD Ontario students, what is the performance of ESL/ELD students on the OSSLT?

  • Is there a significant difference in test performance between the two groups of students in relation to the reading types, skills and strategies, and the four writing types?


Mean score for Non-ESL Ontario students#=138392 (OSSLT) for ESL/ELD Students

Mean score for ESL/ELD students #=2686

Reading Types:

-Information (e.g., explanation, opinion)

64/90 (71%)

46/90 (51%)

- Graphic (e.g., graph, schedule, instructions)

36/50 (72%)

26/50 (52%)

- Narrative (e.g., story, dialogue)

47/60 (78%)

34/60 (57%)

OSSLT Reading Results February 2002


Reading Skills: (OSSLT) for ESL/ELD Students

- Understands directly stated ideas and information

46/60 (77%)

36/60 (60%)

- Understands indirectly stated ideas and information

65/90 (72%)

46/90 (51%)

- Makes connections between experiences and the ideas, information in the reading

34/50 (68%)

24/50 (48%)

Reading Strategies:

- Vocabulary

22/30 (73%)

15/30 (50%)

- Syntax

19/30 (63%)

13/30 (43%)

- Organization

25/36 (69%)

19/36 (53%)

- Graphic features

16/24 (67%)

12/24 (50%)


Mean score for Non-ESL Ontario students#=4068 (OSSLT) for ESL/ELD Students

Mean score for ESL/ELD students #=2686

Reading Test Formats:

-Multiple choice (MC)

62/80 (78%)

47/80 (59%)

- Constructed response (CR)

53/70 (76%)

38/70 (54%)

- Constructed response with explanations (CRE)

31/50 (62%)

21/50 (42%)


OSSLT Writing Results February 2002 (OSSLT) for ESL/ELD Students

Percentage of Non-ESL Ontario students#=138392

Percentage of ESL/ELD students #=2686

Summary

Blank/Illegible & Irrelevant Content / Off-task (0 points)

18%

33%

1 point

3%

3%

2 points

26%

22%

3 points

36%

34%

4 points

17%

11%

Paragraphs expressing an opinion

(0 points)

7%

22%

1 point

<1%

<1%

2 points

18%

29%

3 points

50%

38%

4 points

25%

11%


Percentage of Non-ESL Ontario students#=138392 (OSSLT) for ESL/ELD Students

Percentage of ESL/ELD students #=2686

News report

(0 points)

7%

25%

1 point

<1

2%

2 points

12%

20%

3 points

53%

41%

4 points

27%

12%

Information paragraph

(0 points)

11%

27%

1 point

<1%

2%

2 points

15%

22%

3 points

50%

39%

4 points

23%

10%


Discriminant analysis
Discriminant Analysis (OSSLT) for ESL/ELD Students

  • Y grouping =AB1X1B2X2B3X3 ….E

    • ESL status: 1 = ESL/ELD students; 2 = non-ESL/ELD students

    • ESL N=2688; Non-ESL N=4068

    • For all the following tables, p < .000







Significance of the study
Significance of the Study OSSLT Feb 2002

  • ESL/ELD students make up an substantial and growing portion of Canadian high school population. It is therefore important to identify factors and barriers that are specifically associated with their literacy development.

  • This part of the results have informed us about potential difference in ESL/ELD students’ literacy development through OSSLT. Such an information can be used to support these students’ learning.


ad