Interrelationship between assessment and instruction
This presentation is the property of its rightful owner.
Sponsored Links
1 / 48

Interrelationship between Assessment and Instruction PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on
  • Presentation posted in: General

Interrelationship between Assessment and Instruction. Dr. Liying Cheng Faculty of Education, Queen’s University [email protected] Overview. How we define the interrelationship between assessment and instruction How we define learning and learning targets

Download Presentation

Interrelationship between Assessment and Instruction

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Interrelationship between assessment and instruction

Interrelationship between Assessment and Instruction

Dr. Liying Cheng

Faculty of Education, Queen’s University

[email protected]


Overview

Overview

  • How we define the interrelationship between assessment and instruction

  • How we define learning and learning targets

  • Key components of classroom assessment

  • Project 1 – teacher classroom assessment

  • Project 2 – large-scale testing and second language students


Aeg faculty of education queen s u

AEG – Faculty of Education, Queen’s U

  • Assessment and Evaluation Group (AEG) is an evaluation research and consulting group at the Faculty of Education, Queen’s University. AEG operates in many contexts, at many levels, with many methods of inquiry all directed at the improvement of learning.

  • I acknowledge the use of certain slides from my colleagues – Don Klinger and Lyn Shulha – in this seminar.


Rationale key relationship

Rationale: Key relationship

Teaching

Learning

Assessment


Interrelationship

Interrelationship

  • Impact of assessment (testing and public examinations) on classroom instruction (teaching and learning) – washback research (Latham, 1877; Li, 1990; Alderson & Wall, 1993; Cheng, Watanabe, with Curtis, 2004)


Interrelationship1

Interrelationship

  • Links between assessment and instruction in terms of authenticity and congruence of assessment practices in relation to a program of study (Douglas, 2000; Bachman & Cohen, 1998)

  • Assessment of student outcomes in term of curriculum and teaching (Johnston, 2000)


Interrelationship2

Interrelationship

  • Teacher assessment where the teacher is the agent of assessment - conducting both formative and summative assessment – linking to the purposes of assessment (Brindley, 2001; Brindley, 2007; Cheng et. al. 2004; Rea-Dickins, 2004).


What is learning

What is Learning?

A curricular definition

Learning - acquiring knowledge, skills, attitudes, and values and experiences.

A cognitive definition

Learning - a process of formulating new and more complex understandings of the world

Learning - revising and refining mental constructs, i.e., the understandings that guide how we think, speak and behave


Wilson s taxonomy for teachers

Wilson’s Taxonomy for Teachers

Novice

  • A journey from novice to expert

  • Helps you monitor how students are thinking & performing

  • Cues you about the kinds of questions or activities that can help students think more deeply about their learning?

I.C.E.

Ideas

  • Connections

    • Within content

    • To self

Extensions

Other Contexts

Expert


Learning targets mcmillan 2004

Learning targets (McMillan, 2004)

  • Indicate 1) what a student is to know and/or do as a result of instruction and 2) the criteria for evaluating the performance (criteria)

    • Knowledge and simple understanding

      • Declarative (know what)

      • Procedural (know how)

    • Deep understanding and reasoning

    • Skill

    • Product

    • Affect


Classroom assessment

Classroom assessment

  • The collection, evaluation, and the use of information to help teachers make better decisions that improve student learning.

  • Classroom assessment is more than testing and measurement.

  • The fours essential components to implementation classroom assessment are

    • Purpose

    • Measurement

    • Evaluation

    • Use


Classroom assessments

Classroom Assessments

  • Developing and Choosing Methods for Assessment

  • Collecting Assessment Information

  • Judging and Scoring Student Performance

  • Summarizing and Interpreting Results

  • Reporting Assessment Findings

    Principles for Fair Student Assessment Practices for Education in Canada (1993)


Teach and e learning grants

TEACH and e-Learning grants

  • Classroom Assessment: Theory and Practice

  • Computer-Based Assessment Development

  • Researching Teaching English as a Second Language


Classroom assessment1

Classroom Assessment


B ed instructors

B Ed Instructors

  • The instructors commented much of their time with the B Ed candidates will be spent exploring

    • practicum expectations,

    • professional conduct,

    • classroom procedures and routines,

    • shaping instruction for individual, small group and whole class activities,

    • motivating students,

    • how to approach issues in classroom management, and

    • Emphasize issues in assessment and evaluation.


Associate teachers

Associate Teachers

  • All of the associate teachers suggested that candidates need some familiarity with

    • rubrics,

    • making judgments about student learning,

    • achievement chart categories, and

    • the four levels of achievement.

  • Many suggested that it would be an asset to be able to create rubrics. This requires some understanding of how to differentiate a Level 2, Level 3, and Level 4 response to a learning task.


School district administrators

School District Administrators

  • They expressed a strong interest in seeing B Ed candidates grapple with the concept of assessment for learning (formative assessment).

  • When they also motioned the need to construct grades (summative assessment), they suggested that candidates have a strong understanding of how to determine grades from a variety of instruments  and achievement charts and how to identify which of these represent most recent and consistent performance. 

  • Matching assessment tools to Ministry achievement charts and the ability to do performance assessment rounded out their suggestions to this year's class.  


Interrelationship between assessment and instruction

Rubric to Monitor Learning and Achievement

Levels of Performance


Growth in bike riding

Growth in Bike Riding


Classroom assessment theory and practice

Classroom Assessment: Theory and Practice

  • The Fallibility of Assessment

  • What Does Learning Look Like?

  • Assessment of Learning

  • Assessment for Learning

  • Developing Rubrics

  • Developing Assessment Instruments

  • The Multiple Perspectives of Assessment


Differences in large scale testing

USA has

Standardized Testing

No set curriculum.

Separation of assessment and instruction.

No attempt to involve teachers in interpreting the responses and results.

Teachers administer the tests because they have to.

Canada has

Large-scale Assessment

Provincial curriculum.

Assessment are based on learning expectations and are integrated into instruction.

Teachers are involved with test design, development, scoring and interpreting.

Teachers administer the tests because they have to and because schools have growth plans and resources linked to assessments.

Differences in Large-scale testing


The myths of testing stiggins 2004

The Myths of Testing(Stiggins, 2004)

Testing motivates students to learn

  • If I threaten you will fail and then you will try harder

  • If a little intimidation does not work, try more intimidation

  • Maximize anxiety to maximize learning

    Testing helps teachers make important instructional decisions

  • Students are not assessment users

    Important assessment decisions can be made once a year

  • Investment of time, effort, and money into large-scale testing supports this belief

    Learning how to assess is not as important as learning how to teach

  • Teachers teach and testing professionals test

  • Assessment literacy


The challenges of the ontario secondary school literacy test osslt for esl eld students

The Challenges of the Ontario Secondary School Literacy Test (OSSLT) for ESL/ELD Students

Cheng, L., Klinger, D., & Zheng, Y. (2007). The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing, 24(2), 1-24.


Research context

Research Context

  • The increasing number of English as a Second Language (ESL) and English Literacy Development (ELD) students in Ontario secondary schools. Immigration to Ontario increased 23% vs. ESL teachers and programs in Ontario schools declined 30%

  • Large-scale educational testing has increasingly been used to measure and ensure student competency or provide system accountability(Firestone, Mayrowetz, & Fairman, 1998; Ryan, 2002).


Research context1

Research Context

  • These large-scale tests are constructed and normed for first language (L1) English speakers. Research suggests, however, that they may have lower reliability and validity for L2 students and should be interpreted differently (Abedi, Leon, & Mirocha, 2003).

  • The confluence of both increased numbers of L2 students and this expanding testing framework in schools has created a new and largely unanticipated educational problem – alarmingly high failure rates of these students (Watt & Roessingh, 2001).


Esl eld students performances in the osslt

ESL/ELD Students’ Performances in the OSSLT

  • Of all the eligible ESL/ELD students, only 46% in Feb. 2002, 45% in Oct. 2002, and 54% in October 2003 participated in the test administration.

  • The pass rate of ESL/ELD students in the last three test administrations are 37%, 34% and 42% respectively. While the total pass rate of all students are 75%, 72% and 77% respectively

  • ESL/ELD students were failing the Ontario Secondary School Literacy Test (OSSLT) at a rate higher than any other group of students

    • Pass rates: 42% vs. 77% in 2003

    • Participation rates: 54% vs. 91% (EQAO, 2004)


The ontario secondary school literacy test osslt

The Ontario Secondary School Literacy Test (OSSLT)

“The OSSLT is a useful quality assurance measure that shows the extent to which Ontario students are meeting a common, basic standard for literacy across the province” (EQAO[1], 2002, p. 1). The OSSLT is developed by the Education Quality and Accountability Office (EQAO) - an independent agency operating within a framework established by the provincial government of Ontario.

[1]We acknowledge the support from EQAO for releasing the February 2002 OSSLT data for this study.


The ontario secondary school literacy test osslt1

The Ontario Secondary School Literacy Test (OSSLT)

  • All students in public and private schools working toward an Ontario Secondary School Diploma are required to write the OSSLT in Grade 10. Students who have been eligible to write the OSSLT at least twice and have been unsuccessful at least once are eligible to fulfill the requirement through the Ontario Secondary School Literacy Course (OSSLC). Successful completion of the OSSLT or OSSLC is a graduation requirement.


The ontario secondary school literacy test reading

The Ontario Secondary School Literacy Test (Reading)

Reading component - a total of 12 short selections in three different text types:

  • information (e.g., explanation, opinion) 50%;

  • graphic (e.g., graph, schedule, instructions) 25%;

  • narrative (e.g., story, dialogue) 25%.

    in three test formats:

  • multiple-choice (40 questions);

  • constructed response (35 questions);

  • constructed responses requiring an explanation (25 questions).


The ontario secondary school literacy test reading1

The Ontario Secondary School Literacy Test (Reading)

  • three reading skills:

    • understanding directly stated ideas and information;

    • understanding indirectly stated ideas and information;

    • making connections between personal experiences and ideas and information in a selection.

  • Four reading strategies:

    • vocabulary,

    • syntax,

    • organization,

    • graphic features.


Scoring criteria reading

Scoring Criteria (Reading)

  • Multiple choice items are scored separately.

  • Constructed Response (CR) questions ask students to respond in a few words, and answers are marked correct (1 point) or incorrect (0 point).

  • Constructed Response with Explanation (CRE) questions ask the students to justify or explain the thinking behind their answers. And the answers are marked correct (1 point), partly correct (0.5 point) or incorrect (0 point)


The ontario secondary school literacy test writing

The Ontario Secondary School Literacy Test (Writing)

Writing - four writing tasks:

  • a summary,

  • a series of paragraphs expressing an opinion,

  • a news report, and

  • an information paragraph.

    Students need to pass both components of the OSSLT.

    OSSLT is administered over a two-day period.


Scoring criteria writing

Scoring Criteria (writing)

  • Two steps in the marking of writing--holistic scoring and analytical scoring.

  • Holistic scoring (0-4 points)

  • If the student performances do not meet the requirements to pass, then all four writing tasks will go on to step 2: analytic scoring, evaluated by four separate characteristics:

    • Main idea

    • supporting details,

    • organization,

    • Spelling, grammar and punctuation.


Research framework

Research framework

  • Research studies have shown that the test items, the content, types and context of reading passages, and the relationships of these intervening factors can have a significant impact on students’ performance (Anderson et al., 1991; Freedle & Kostin, 1993; Kobayashi, 2002, Lee, 2002; Peretz & Shoham, 1990; Perkins, 1992).

  • Research into writing assessment has demonstrated that different types of writing tasks create different challenges for students (e.g., Connor-Linton, 1995a, 1995b, Kobayashi & Rinnert, 1996, Hamp-Lyons, 1996).


Research questions

Research questions

  • Compared with Non-ESL/ELD Ontario students, what is the performance of ESL/ELD students on the OSSLT?

  • Is there a significant difference in test performance between the two groups of students in relation to the reading types, skills and strategies, and the four writing types?


Interrelationship between assessment and instruction

Mean score for Non-ESL Ontario students#=138392

Mean score for ESL/ELD students #=2686

Reading Types:

-Information (e.g., explanation, opinion)

64/90 (71%)

46/90 (51%)

- Graphic (e.g., graph, schedule, instructions)

36/50 (72%)

26/50 (52%)

- Narrative (e.g., story, dialogue)

47/60 (78%)

34/60 (57%)

OSSLT Reading Results February 2002


Interrelationship between assessment and instruction

Reading Skills:

- Understands directly stated ideas and information

46/60 (77%)

36/60 (60%)

- Understands indirectly stated ideas and information

65/90 (72%)

46/90 (51%)

- Makes connections between experiences and the ideas, information in the reading

34/50 (68%)

24/50 (48%)

Reading Strategies:

- Vocabulary

22/30 (73%)

15/30 (50%)

- Syntax

19/30 (63%)

13/30 (43%)

- Organization

25/36 (69%)

19/36 (53%)

- Graphic features

16/24 (67%)

12/24 (50%)


Interrelationship between assessment and instruction

Mean score for Non-ESL Ontario students#=4068

Mean score for ESL/ELD students #=2686

Reading Test Formats:

-Multiple choice (MC)

62/80 (78%)

47/80 (59%)

- Constructed response (CR)

53/70 (76%)

38/70 (54%)

- Constructed response with explanations (CRE)

31/50 (62%)

21/50 (42%)


Interrelationship between assessment and instruction

OSSLT Writing Results February 2002

Percentage of Non-ESL Ontario students#=138392

Percentage of ESL/ELD students #=2686

Summary

Blank/Illegible & Irrelevant Content / Off-task (0 points)

18%

33%

1 point

3%

3%

2 points

26%

22%

3 points

36%

34%

4 points

17%

11%

Paragraphs expressing an opinion

(0 points)

7%

22%

1 point

<1%

<1%

2 points

18%

29%

3 points

50%

38%

4 points

25%

11%


Interrelationship between assessment and instruction

Percentage of Non-ESL Ontario students#=138392

Percentage of ESL/ELD students #=2686

News report

(0 points)

7%

25%

1 point

<1

2%

2 points

12%

20%

3 points

53%

41%

4 points

27%

12%

Information paragraph

(0 points)

11%

27%

1 point

<1%

2%

2 points

15%

22%

3 points

50%

39%

4 points

23%

10%


Discriminant analysis

Discriminant Analysis

  • Y grouping =AB1X1B2X2B3X3 ….E

    • ESL status: 1 = ESL/ELD students; 2 = non-ESL/ELD students

    • ESL N=2688; Non-ESL N=4068

    • For all the following tables, p < .000


Discriminant analysis of reading writing in osslt feb 2002

Discriminant Analysis of Reading & Writing in OSSLT Feb 2002


Discriminant analysis of reading types in osslt

Discriminant Analysis of Reading Types in OSSLT


Discriminant analysis of reading skills strategies in osslt feb 2002

Discriminant Analysis of Reading Skills & Strategies in OSSLT Feb 2002


Discriminant analysis of reading formats in osslt feb 2002

Discriminant Analysis of Reading Formats in OSSLT Feb 2002


Discriminant analysis of writing types in osslt feb 2002

Discriminant Analysis of Writing Types in OSSLT Feb 2002


Significance of the study

Significance of the Study

  • ESL/ELD students make up an substantial and growing portion of Canadian high school population. It is therefore important to identify factors and barriers that are specifically associated with their literacy development.

  • This part of the results have informed us about potential difference in ESL/ELD students’ literacy development through OSSLT. Such an information can be used to support these students’ learning.


  • Login