working with smart2 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Working with SMART2 PowerPoint Presentation
Download Presentation
Working with SMART2

Loading in 2 Seconds...

play fullscreen
1 / 83

Working with SMART2 - PowerPoint PPT Presentation

  • Uploaded on

Working with SMART2. Workshop for leaders in the use of SMART2. Workshop presentation 20/09/2010 Andrew Fraser and Carmel Kriz. Objectives. Gain greater understanding of using data from an inquiry frame of reference Develop greater familiarity with the SMART2 web-based application

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Working with SMART2' - jersey

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
working with smart2

Working with SMART2

Workshop for leaders in the use of SMART2

Workshop presentation 20/09/2010

Andrew Fraser and Carmel Kriz


Gain greater understanding of using data from an inquiry frame of reference

Develop greater familiarity with the SMART2 web-based application

Undertake an initial analysis of school SMART2 data

Identify strengths and issues to explore further

Develop strategies for effective leadership in the analysis and use of information

data matters

Data Matters

Leading learning with an inquiry habit of mind

setting the context

Setting the context

A climate of accountability

using data to learn
Using data to learn

What comes to mind when you hear the word:


(60 seconds)

using data to learn1
Using data to learn

What comes to mind when you hear the word:


(60 seconds)

A commitment to action (p. 16)

Schools need reliable, rich data on the performance of their students because they have the primary accountability for improving student outcomes.

Good quality data supports each school to improve outcomes for all of the students.

what is accountability1
What is Accountability?

Earl and LeMahieu, 1997

As cited in “Leading learning in a data-rich world” Earl and Katz (2006: 10)

Accountability is

the conversation

about what the information means


how it fits with everything else

that we know


about how to use it

to make positive changes


Internal motivation: Teachers improve through learning their craft, which happens best in PLCs which have access to valid data about achievement.

Professional Learning Communities

Inquiry Habit of Mind


Requires imagination

What happens?

Data: the divide…

The underlying assumptions


External motivation: Teachers are motivated to improve by the idea that someone is watching over them, judging and visiting consequences on them if the targets aren’t met.

Thanks to John DeCourcey

data worth looking at
Data worth looking at:

“The value of the data emerges only when analysis provides insights that direct decisions for students.”

Stephen White,

Beyond the Numbers, 2005

from data to professional knowledge
From Data to Professional Knowledge
  • Data are
  • Making useful information
  • Developing informed professional knowledge
data information knowledge
Data Information Knowledge
  • Types of Data
  • Outcome
  • Demographic
  • Process
  • Perceptual


Actionable knowledge

  • Types of Decisions
  • To identify or clarify a problem
  • Set and monitor progress towards goals
  • Address individual and group needs
  • Monitor and evaluate practices
  • Validate proposals for change
  • Assess whether student needs are being met
  • Strategically allocate resources
  • Adapt a new practice to fit the situation

“Data analysis is a team sport.”

Doug Reeves

  • Develops team thinking
  • Promotes insights that numbers alone can’t produce
  • Provides a forum for legitimizing practice
  • A characteristic of “Schools that Learn”


What does this indicate?


Top mark in the class, student has been studying well, highly motivated…

Wonderful learning?

Lowest he’s ever achieved, no sign of any preparation, smart student, easy test

He scored 42/60 on the midyear test


Whole class is showing improvement, class average is better than ever gained before…

Superb teaching?

Almost any evidence can be an indicator of many different constructs – finding the most productive questions is the art of data analysis.

Stole the answer sheet from the teacher’s desk the day before…

Dreadful cheat?

Thanks to John DeCourcey


Use it to ask the right questions

Test the ‘Strength’ of the evidence: is it a sound link to the construct?

‘Triangulate’ the construct – what other way do I have of looking at it?

Using data well

Decide on a direction – go top-down, or bottom-up

Slice the data different ways: mean, top, bottom, time, individual map

Be suspicious of yourself: ask the next question

Thanks to John DeCourcey

three principles of data analysis
Three Principles of Data Analysis
  • Exploring and determining the antecedents for success
  • Collaborating with colleagues
  • Embracing Accountability - Learning from our data
some questions for consideration
Some questions for consideration
  • “What does this ask us about how teaching and learning are going in our school?”
  • “What do we need to do about it?”
  • “What do we need to learn?”
achievement scale
Achievement Scale

Each Year level Student Report shows 6 of the bands.

Year 9 Reports show bands 5 to 10

Year 7 Reports show bands 4 to 9

Year 5 Reports show bands 3 to 8

Year 3 Reports show bands 1 to 6

Band 10

Band 9

Band 8

Year 9

Band 7

Year 7

Band 6

Year 5

Band 5

Band 4

Year 3

Band 3

Band 2

Band 1

achievement bands
Achievement Bands

Band 10

Band 9

Band 9

Band 8

Band 8

Band 8

Band 7

Band 7

Band 7

Band 6

Band 6

Band 6

Band 5

Band 5

Band 4

Band 3

Students working at proficiency would be in the top 2 achievement bands for the respective year level.

Students are proficient

Year 9

Year 7

Year 5

Students working at minimum standard would be in the second lowest band.

Year 3

Students that are in Band 1 are deemed at operating at below minimum standard.

Band 6

Band 5

Band 5

Band 4

Band 4

Band 3

Band 2

Students are at the national minimum standard

Band 1

Students are below the national minimum standard

naplan scales

There are five separate national scales, one each for:





and Numeracy.

Note: There are no more benchmarks. Students that are in the lowest band are deemed to be operating below minimum standard.










using means and standard deviations
Using Means and Standard Deviations

Good starting point

Gives overall school performance

Helps frame questions for further investigation

Gives difference in mean between the school and the state

Can be found in Means &Standard Deviations

Question : How significant is the difference from the state????

means and standard deviations
Means and Standard Deviations

When student numbers are small, summaries based on mean (average) scores can be misleading. In small groups, means can be affected by a couple of high performing or low performing students.

Means sometimes obscure real differences within a group, even when the numbers are large. For example, a school with concentrations of both educationally advantaged and disadvantaged students may find that the school mean actually describes very few of its students.

For schools with year cohorts less than 5 students no school means and deviations will be displayed.

means and standard deviations school reports
Means and Standard DeviationsSchool Reports

Select aspect – Reading or Numeracy

Compare the mean for the full cohort against the state mean

Compare full cohort with boys, girls, ATSI

Compare means in different strands of literacy (writing, reading etc) or numeracy (number, measurement )

significance of difference in means
Significance of difference in means

School mean – state mean, divided by the state SD.

This calculation is about substantive meaning of any difference. It is not a test of statistical difference.

significance of difference in means1
Significance of difference in means

Rule of thumb method :

> 0.5 Well above state

0.5 > 0.2 Above state

0.2 > -0.2 Within state

-0.2 > -0.5 Below state

< -0.5 Well below state

further investigations
Further investigations

Is the Significant difference of the mean:

Because of the results of a particular group? Eg.boys, girls

Because of the results of a particular class? Eg. class group that has had 4 different teachers in one year

The same for both year levels (3 & 5, 7 & 9)?

Has this been the pattern in previous years?

means and standard deviations1
Means and Standard Deviations

Significant difference of the mean allows for comparison between year levels

Means and Standard Deviations allow for creation of means table for special groups

Means and Standard Deviations allow for comparison with school groups

percentage in bands
Percentage in Bands

Choose Percentages in Bands from main menu.

View for full cohort, boys, girls, ATSI, or custom groups

Record bands where difference from the state is significant.

* less students in top band or top two bands

* more students in the bottom band (below National Minimum Standard)

* more students in the second bottom band (at the National Minimum Standard)

OR of course the reverse

further investigations1
Further Investigations

Check to see if there is there is a significant difference in performance between the strands e.g.

In Literacy - between writing and reading

In Numeracy - between Number/Patterns & Algebra and Data/ Measurement /Space and Geometry

More detailed investigation can be carried out through the Item Analysis.

Questions need to be asked re pedagogy in these areas – this could form part of the School Improvement Plan.

percentage in bands2
Percentage in Bands

Record names of students in the bottom two bands

Create a group of these students. (Create /Edit and Delete Groups)

Go to Item Analysis to look at which items all this group have correct /incorrect

percentage in bands3
Percentage in Bands

Check names of students in particular bands.

Are there any surprises?

Follow up responses of individual students whose band placement is of concern by going to Student Analysis

value added
Value added

Percentages in Bands may be of use for schools where they have tried to increase the movement of students into higher bands.

Care must be taken when using this information with schools who have less than 10 students. In schools with small numbers each individual student is worth a large percentage and this may impact greatly on the student numbers in a band.

student growth years 5 7 9
Student Growth (Years 5, 7, 9)

Select Aspect – Reading or Numeracy

Select Student Growth

student growth1
Student Growth

Arrows indicate growth for individual students (3 to 5, 5 to 7, 7 to 9).

Length of arrow indicates amount of growth.

Orange arrows indicates growth that is greater than or equal to expected growth for starting point.

Blue arrows indicate growth that is less than expected growth for starting point.

Downward blue arrow indicates student’s score less than their last score

Click on arrows for students’ names and scores.

Read from left to right – lowest performing students in the cohort’s last NAPLAN on the left – highest performing on the right.

student growth2
Student Growth

Check which students have made the most growth.

Is it the lowest third, middle third or upper third based on prior performance?

Is there any relationship between amount of growth and any teaching emphasis or intervention for a particular group?

Is there a group of students who have not made expected growth and who are clustered together?

Make a group of these students and examine their responses in Item Analysis and/or Student Analysis looking for common errors

student growth4
Student Growth

Check the tables on lower half of screen:

Average scaled score growth

Expected growth

student growth5
Student Growth

The tram lines on the main graph indicate the growth made by 75 %, 50 % and 25% of students in the state.

The middle table on the right hand side indicates the % of students in the school in the bottom 25th percentile, and the top 25th percentile and between the 25th and 75th percentile.

Does the school’s growth fit the anticipated 25 50 25 pattern ?

item analysis
Item Analysis

Use this table together with Item Analysis Table in the Main Menu.

Print: Report - Analysis by Questions (for required aspect – reading or numeracy and required year level)

Report – Analysis by Questions provides information on % of students choosing correct / incorrect options.

When investigating questions that a large numbers of students answered incorrectly, it is often helpful to ascertain whether a number of them choose the same incorrect answer.

item analysis1
Item Analysis

Go to Item Analysis

Select Year Level

Select Aspect

Sorting of items in different ways.

item analysis3
Item Analysis

Sort by School % Correct

Investigate items where school percentage correct is below state by 10% or more.

Using Report – Analysis by Questions to check for common response errors.

item analysis4
Item Analysis

Sort by Syllabus Outcomes

Check: Which Items have/have not been answered correctly by the majority of students in the Year.

Check for common incorrect responses.

item analysis5
Item Analysis

Select Filter by Substrands (drop-down menu)

Investigate performance on items assessing particular skill

Using Report check for common response errors

Check out the actual item using SCAN.

item analysis6
Item Analysis

can be viewed graphically - choose CHART

item analysis7
Item Analysis

Select : Filter by Group

Investigate aspects of Item Analysis for various subgroups – boys, girls, custom groups.

item analysis8
Item Analysis

Compare performance of Year groups in a particular skill area.

Are there any links items in this set of items?

What are the skills year groups are are finding difficult ?

leading inquiring transforming
Leading, Inquiring, Transforming

Developing an inquiry habit of mind

Being data literate

Creating a culture of Inquiry

school leadership teams
School leadership teams

Familiar with, and competent in, analysing and understanding NAPLAN data

Takes an active role in the analysing and use of the data with staff to bring about measurable improvements in student learning.

towards 2011
Towards 2011


Term 3

Term 4


Analyse NAPLAN data

Identify issues to follow up

Engage others in the analysis

Examine other data

Formulate a whole-school response – School Improvement Plan

Develop specific strategies to implement

Determine measureables, milestones, indicators of improvement

Implement plan

Explore changes – cause and effect

Work with SMART 2 Teaching Strategies

Monitor and evaluate – evidence of change

Critique impact





monitoring change
Monitoring change

How will you know change is or has occurred?

What evidence will you use to monitor the transformation of learners?

What evidence will you use to monitor the transformation of learning?


Dr Philip Pettit and the Canberra – Goulburn CEO

Rosemary Vellar and Sydney CEO

Dr John DeCourcy

NSW Educational Measurement and School Accountability Directorate

Australian Government National Partnerships Agreement

further assistance
Further assistance

Further support and advice can obtained by contacting the Catholic Schools Office, Broken Bay:

Andrew Fraser

Carmel Kriz

Or your school’s consultant.

reflecting on questions
Reflecting on questions
  • “What does this ask us about how teaching and learning are going in our school?”
  • “What do we need to do about it?”
  • “What do we need to learn?”
cause and effect data
Cause and Effect data

Effect Data – outcomes or results in student learning and achievement

Cause Data – professional practices (adult actions) that create specific effects or results

teacher inquiry and knowledge building cycle to promote valued student outcomes
Teacher inquiry and knowledge-building cycle to promote valued student outcomes

What are our students’

learning needs?

What are our own

learning needs?

  • What do they already know?
  • What sources of evidence
  • have we used?
  • What do they need to learn
  • and do?
  • How do we build on what
  • they know?
  • How have we contributed to
  • existing student outcomes?
  • What do we already know that we
  • can use to promote valued
  • outcomes?
  • What do we need to learn to do to
  • promote valued outcomes?
  • What sources of evidence/
  • knowledge can we utilise?

What has been the

impact of our

changed actions?

  • How effective has what we
  • have learned and done been
  • In promoting our students’
  • Learning and well-being?

Design of tasks and


Teaching actions

H. Timperley, A Wilson, H Barrar & I Fung (2007)

Teacher Professional Learning and Development: Best Evidence Synthesis Iteration

Wellington, New Zealand: Ministry of Education

leader inquiry and knowledge building cycle to promote valued student outcomes
Leader inquiry and knowledge-building cycle to promote valued student outcomes

What are our teachers’

learning needs?

What are our own

learning needs?

What knowledge

and skills

do our teachers and

students need?

What knowledge and skills

do we as leaders need?

What has been the

impact of our

changed actions?

Develop leadership




leadership skills.

What has been the impact

of our changed actions

on teachers and students?

Engage teachers

In new

learning experiences

Modified from H. Timperley (2010)

data teams
Data Teams
  • Guidelines for effective Data Teams
    • Have collaborative teams
    • Provide adequate time for collaboration
    • Engage in collective inquiry
    • Focus on the cause and effect data
    • Post graphs and charts so they are visible
    • Subscribe to action orientation and experimentation
    • React to our data with sound instructional and curricular decisions
    • Implement an effective communication
    • Are results driven
    • Are devoted to continuous improvement
data literacy a thinking process
Data literacy – a thinking process

Standing back and thinking about what you need to know and why

Collecting or locating the necessary data

Finding ways to link data sources

Ensuring that data are worth considering

Being aware of their limitations

Thinking about what the results mean

Systematically considering an issue from a range of perspectives so that you feel that you have evidence to explain, support, and also challenge your point of view.

data information knowledge1
Data Information Knowledge




Embedded in a context

That gives meaning and connectedness.

Adapted from: van Barneveld (2008) and Earl and Katz (2006)

which effect data to focus on
Which effect data to focus on?

“Will this piece of data help a classroom teacher change curriculum, assessment and instruction and thus improve student achievement?”

Douglas Reeves