Naplan workshop
This presentation is the property of its rightful owner.
Sponsored Links
1 / 46

NAPLAN Workshop PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on
  • Presentation posted in: General

NAPLAN Workshop. Assessment for Better Learning using NAPLAN Data. Presented by Peter Congdon, Principal Consultant – Kmetrics On behalf of the VCAA. Workshop structure. Main themes

Download Presentation

NAPLAN Workshop

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Naplan workshop

NAPLAN Workshop

Assessment for BetterLearningusing NAPLAN Data

Presented by

Peter Congdon, Principal Consultant – Kmetrics

On behalf of the VCAA


Workshop structure

Workshop structure

  • Main themes

  • How curriculum leaders and classroom teachers can use their school-level data to analyse the impact of their school’s learning programs.

  • How classroom teachers can use the responses to questions on the NAPLAN 2014 tests as a diagnostic tool to inform future teaching.


Workshop content

Workshop Content

  • NAPLAN Data Service reports and functions

  • Methods of using the data and results for monitoring and improvement purposes

  • Working with data, interpreting data, describing data and developing an informed response to the data.


Context professional practice

Context – Professional practice

  • Using assessment data effectively has become embedded in teaching expectations and school improvement processes.

  • Use of data:

  • National Professional Standards for Teachers – Australian Institute for Teaching and School Leadership

    • Standard 5 - Assess, provide feedback and report on student learning


Context to use student data to improve teaching practice

Context - To use student data to improve teaching practice.

Teachers need to be able to do the following:

• Find the relevant pieces of data in the data system or display available to them (data location)

• Understand what the data signify (data comprehension)

• Figure out what the data mean (data interpretation)

  • Substantive and contextual

    • Select an instructional approach that addresses the situation identified through the data (instructional decision making)

    • Frame instructionally relevant questions that can be addressed by the data in the system (question posing)

    Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports. U.S. Department of Education Office of Planning, Evaluation and Policy Development http://www2.ed.gov/rschstat/eval/data-to-inform-instruction/report.pdf


Https naplands vcaa vic edu au helpdesk phone 1800 648 637

https://naplands.vcaa.vic.edu.auHelpdesk phone 1800 648 637

AIM results (2007 and earlier) are no longer available


Reference documents

Reference documents

  • Assessment materials

  • Test performance & content summary guides

  • Reporting guides

  • Descriptive exemplars of marking guides

  • Analysis strategies

  • Online tutorial assistance for reports can be accessed at http://usingassessmentdata.vcaa.vic.edu.au/naplan/index.aspx

    All available within the NAPLAN Data Service to support use of the results


Box and whisker charts

Box and Whisker Charts

National

State

School

Reference groups

Focus group


Normal distribution

Normal Distribution

Number of students if total = 100

10

15

25

25

15

10

90th percentile

75th percentile

50th percentile

25th percentile

10th percentile


Skewed distribution

Skewed Distribution

Number of students per 100

10

15

25

25

15

10

90th percentile

75th percentile

50th percentile

25th percentile

10th percentile


Naplan reporting bands

NAPLAN Reporting Bands


Example naplan summary year 7

Example NAPLAN Summary Year 7

  • What are the main features of these results?

  • Strongest in Writing and Spelling

  • Lower students not as low as State’s low students

  • Higher students not as high as State’s higher students in Reading, G&P and Numeracy

  • Is this a reflection of the school’s teaching program, and/or a feature of this cohort?

  • How much of these differences are due to imprecision?


Naplan year 7 summary

NAPLAN Year 7 Summary

2014

2013


Year 7 results

Year 7 results

  • Usually, Year 7’s have only been at your school for a few months prior to testing.

  • Results can reflect feeder school programs.

  • Consider grouping students by main feeder schools and sharing results – network.


School summary exercise 5 mins

School Summary Exercise - 5 mins

  • Review your School summary report(s)

  • Address the following

    • Strongest in;

    • Lower students compared to State’s low students;

    • Higher students compared to State’s higher students;

    • Major influence on results include;

    • Strategies to consider;


Trend data

Trend Data

  • Find evidence of the impact of change over five years

    • Shows the range of student achievement levels, Box and Whiskers

    • Plots the mean student achievement level;


Naplan workshop

  • Main sources of variability:

  • Different students

    • Work ethic

    • Behaviour

    • Home support

  • Measurement imprecision

    • Test properties

    • Equating

    • Group size

  • School

    • Leadership

    • Resources

  • Teacher

    • Effectiveness

  • Program

    • Content

    • Alignment


Five year trend exercise 5 mins

Five Year Trend Exercise – 5 mins

Review your Five Year Trend report(s)

Address the following

  • Compare performance relative to Stategroup

    • High = Top 25% v State Top 25%,

      • Medium = Middle 50% v State Middle 50%

      • Low = Bottom 25% v State Bottom 25%

    • Reading

    • Writing

    • Spelling

    • Grammar & Punctuation

    • Numeracy

  • Identify Influencers on results

    • Cohort

    • School

    • Teacher

    • Programs

    • Main sources of variability:

    • Different students

      • Work ethic

      • Behaviour

      • Home support

    • Measurement imprecision

      • Test properties

      • Equating

      • Group size

    • School

      • Leadership

      • Resources

    • Teacher

      • Effectiveness

    • Program

      • Content

      • Alignment

    Choose

    one


    Group summary report

    Group Summary Report

    How do our groups stack up against the State groups?

    What does this tell us about;

    Cohort,

    School,

    Teachers,

    Programs.


    Group summary exercise 5 minutes

    Group Summary Exercise – 5 minutes

    Review your Group summary report(s)

    Address the following

    • Compare the performance of each group relative to State group in one domain

      • High = Top 25% v State Top 25%,

        • Medium = Middle 50% v State Middle 50%

        • Low = Bottom 25% v State Bottom 25%

      • Girls V State Girls

      • Boys V State Boys

      • LBOTE V State LBOTE

      • ATSI V State ATSI

  • Influencers on results

    • Cohort

    • School

    • Teacher

    • Programs

    • Main sources of variability:

    • Different students

      • Work ethic

      • Behaviour

      • Home support

    • Measurement imprecision

      • Test properties

      • Equating

      • Group size

    • School

      • Leadership

      • Resources

    • Teacher

      • Effectiveness

    • Program

      • Content

      • Alignment


    Assessment area report

    Assessment Area Report

    Raw score average, State (36 items @ 60% correct) = 36*.60 => 21.6

    Raw score average, School (36 items @ 52% correct) =36*.52 => 18.7

    Number of items

    Percentage of items answered correctly in short answer questions


    Assessment area exercise 2 minutes

    Assessment Area Exercise – 2 minutes

    • Identify if any dimensions have been flagged as significantly different from the State

    • Calculate or estimate the Raw score difference between your students and the State on one or more dimensions


    Writing criteria report

    Writing Criteria Report


    Writing criteria exercise 2 mins

    Writing Criteria Exercise – 2 mins.

    Compare modal scores.

    Modal score = most common score

    Which criterion are you relatively strongest on?

    School

    State

    3-4

    4

    2

    3


    Naplan workshop

    Item Analysis Report

    • How classroom teachers can use the responses to questions on the NAPLAN 2014 tests as a diagnostic tool to inform future teaching.

    Finding skills of relative strength or weakness – Graphical format

    Understanding student weaknesses – Table format


    Naplan workshop

    Item Analysis Report

    Test and group details

    Link to some test details

    Link to test items


    Naplan workshop

    Item Analysis Report – Graph

    Finding skills of relative strength or weakness

    Harder than State =

    Easier than State =


    Naplan workshop

    Item Analysis

    Understanding student weaknesses


    Item analysis understanding student weaknesses

    Item AnalysisUnderstanding student weaknesses

    Half of the group could do it – B

    Remainder had problems. Were they related to

    Format, language, concept, process, knowledge, skill,...?

    44+50 = 94

    50 is half of 100 , 22 is half of 44

    A

    D


    Naplan workshop

    One quarter of the group could do it – option A

    Remainder had problems. Were they related to

    Format, language, concept, process, knowledge, skill, opportunity to learn,...?


    Student responses individuals

    Student Responses - Individuals


    Item level diagnostics

    Item level Diagnostics

    • By comparing your students’ success at the item level, to that of all other students in the state, you can..

    • Look for relative differences as the test progresses

      • Did our students answer all the items?

      • Were they consistently above, below or similar to the rest of the state?

        • What do any difference represent? Put it into context

        • Is language or vocabulary an issue?

        • Is test taking an issue – format, motivation, terminology?

      • Are there items that represent areas of the curriculum not yet introduced?

        • How does this compare to the rest of the state?

      • Are there areas taught, but not as well as expected?

      • Consider what is happening at other Year levels – curriculum mapping.

      • Where are the relative strengths – how can you learn from them?

    Peter Congdon - Kmetrics


    Item analysis exercise 2 minutes

    Item Analysis Exercise – 2 minutes

    • Identify items indicating relative strengths and weaknesses

    • Follow up (for class room teachers)

      Understand and capitalise on strengths

      Investigate relative weaknesses and develop a plan in response

      Discuss and share with colleagues


    Zone of proximal development

    Zone of Proximal Development

    Level of potential after other steps have been made.

    Zone of Proximal Development:

    What students are capable of learning with the guidance and support from teachers and peers.

    What students can already do independently.

    Vygotsky and other educational professionals believed education's role was to give children experiences that were within their zones of proximal development, thereby encouraging and advancing their individual learning


    Substantive descriptions of achievement levels

    Substantive descriptions of achievement levels

    Using the NAPLAN items to identify skills, knowledge, procedures...

    National bands – parent reports skill descriptors, www.nap.edu.au


    Naplan workshop

    Describing Zone of Proximal Development – for most raw scores of 5-6

    Easier

    drafting

    blizzard

    tertels

    Harder

    Seriusly

    orkwardly


    Relative growth

    Relative Growth

    • How is relative growth defined?

    • Each student’s level of relative growth is determined by comparing their current year NAPLAN result to the results of the group of all ‘similar’ Victorian students. ‘Similar’ students are defined as those that had the same NAPLAN score two years ago.

    • Compared to these similar students, if a student’s current NAPLAN score is in the :

      • highest 25%, their growth level is categorised as ‘High’. (Green)

      • middle 50%, their growth level is categorised as ‘Medium’ (Yellow), and

      • lowest 25%, their growth level is categorised as ‘Low’ (Red).

    • Note that the percentages within each category will vary from school to school.


    Relative growth1

    Relative Growth

    +138

    +18

    -30

    +15


    Relative growth exercise 2 minutes

    Relative Growth Exercise – 2 minutes

    • Identify domains with low relative growth greater than 25%

    • And or

    • Identify domains with high relative growth greater than 25%

    • Was the relative growth even across the starting Bands

    • Follow up (Co-ordinators and class room teachers) implementation and effectiveness of differentiation


    Working with naplan data

    Working with NAPLAN Data

    • Principal

      • Analyse Summary and Trend results

      • Conduct program evaluations

      • Facilitate staff access to work with results

  • Co-ordinator

    • Map the relatively high and low performances against the delivery of the curriculum

    • Look for class/group differences

    • Work with colleagues .

  • Class room teacher

    • Diagnose misconceptions, weaknesses & strengths.

    • Develop teaching plans in response.


  • Reporting back

    Reporting Back

    • Summative position

    • Overall location of the students and subgroups

      • Shape of the distributions

        • Are students being left behind?

        • How spread out are they?

        • Is there too much focus on the low achieving students - top students held back?

      • How would describe the location of your students against the state?

      • How spread out are your students in the different dimensions?

    • Trends

      • How have your results changed over time?

        • Consider both the location and the shape of the distribution.

      • Are there identifiable factors that may be contributing?

        • Student level effects: motivation, engagement, home

        • Teacher level effects: method, style, experience, support, workload

        • School level effects: leadership, resources, programs

    • Growth,

      • Are students maintaining their position relative to the state?

        • Is the top growing as fast as the bottom relative to the state?

        • Is growth even across the dimensions?

    • Curriculum Mapping

      • Item level Diagnostics

        • Can you find relative strengths and weaknesses?

        • What can you and the school adjust based on these findings?

        • Link to curriculum scope and sequence

        • Link to programs and pedagogy

    Peter Congdon - Kmetrics


    Thank you for attending

    Thank you for attending

    Please use the rest of this time to go over your results, and clarify your interpretations.

    PowerPoint presentation available at www.kmetrics.com.au

    Further help is available by contacting me directly

    Peter Congdon

    Principal Consultant

    Mobile: 0434 000 561

    email: [email protected]

    web: www.kmetrics.com.au


  • Login