slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
3rd ENSEC Conference 29th June - 3rd July 2011 University of Manchester, UK Phillip Slee PowerPoint Presentation
Download Presentation
3rd ENSEC Conference 29th June - 3rd July 2011 University of Manchester, UK Phillip Slee

Loading in 2 Seconds...

play fullscreen
1 / 53

3rd ENSEC Conference 29th June - 3rd July 2011 University of Manchester, UK Phillip Slee - PowerPoint PPT Presentation

  • Uploaded on

3rd ENSEC Conference 29th June - 3rd July 2011 University of Manchester, UK Phillip Slee Professor in Human Development Flinders University Adelaide, South Australia Director: Flinders Centre Promotion Well-Being: Violence Prevention (SWAPv).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '3rd ENSEC Conference 29th June - 3rd July 2011 University of Manchester, UK Phillip Slee' - leilani-briggs

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

3rd ENSEC Conference29th June - 3rd July 2011University of Manchester, UK

Phillip Slee

Professor in Human Development

Flinders University

Adelaide, South Australia

Director: Flinders Centre Promotion Well-Being: Violence Prevention (SWAPv)

kidsmatter the outcomes of an evaluation of a sec initiative in australian schools
“KidsMatter”- the outcomes of an evaluation of a SEC Initiative in Australian schools

Flinders University Evaluation Team :P. T. Slee, M.Lawson, A.Russell, H.Askell-Williams, K.Dix, L.Owens, G.Skryzpiec and B.Spears

outline of presentation

Outline of Presentation

Mental Health

‘KidsMatter Constructs’

KidsMatter Evaluation

KidsMatter Evaluation Findings

”KidsMatter & Disability

KidsMatter and Academic Outcomes’

The ‘Elephant in the Room’

Translating Research to Policy & Practice

Summary & Discussion

KidsMatter Suite of Initiatives
  • KidsMatter –Primary
  • KidsMatter –Early childhood
  • Kidsmatter Quality Assurance
  • Kidsmatter- Disabilities
schools as sites for interventions
Schools as Sites for Interventions
  • Attention now focussed on working through schools to improve the health & well-being of children

(Greenberg, et al,2003)

  • school–community partnerships positively influence outcomes for students, eg showing increases in attendance rates, resilience, behaviour and attitude

(Weare & Nind, 2011)

  • partnerships between school and community enable students to achieve the best life outcomes, eg academic and non-academic eg SEL

(Slee et al, 2009).

kidsmatter conceptual framework
KidsMatter conceptual framework

KidsMatter process assumptions:

protective factors/reduction in risk factors.

The factors reside within the school environment, the family context and the psychological world of the child, & include:

  • Positive school climate
  • Increases in students’ social-emotional competencies, eg sense of mastery and control
  • Increases staff knowledge and competence (e.g., in teaching SEL)
  • effective parenting
  • effective strategies for identifying and working with at risk students
two dimensions of mental health
Two dimensions of mental health
  • Adaptive defences are essential to positive mental health

(Vaillant 2000)

  • increasing international interest in the concept of positive mental health and its contribution to all aspects of human life

(Tennant et al, 2007)

  • The World Health Organisation
    • Mental Health is not simply the absence of mental disorder or illness, but also includes a positive state of mental well-being (WHO, 2004).
    • Mental health can be conceptualised as a state of well-being in which the individual realizes his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community (WHO 2001)
  • Both dimensions of mental health were of interest for us in this evaluation
representation of the two dimensions in our analytical models
Representation of the two dimensions in our analytical models?
  • Representation 1
    • The mental health outcome includes both strengths and difficulties as outcomes at the same level in the conceptual model
  • Representation 2
    • The measure of the positive dimension (SEC) is seen as mediating the relationship between the intervention and the level of mental health problems



Quantitative Measures in all 100 schools

Qualitative Measures: Visits to 10 Schools


Longitudinal data collected up to 4 times during 2007 and 2008

  • School implementation of:
  • KidsMatter specifically
  • Mental health
  • initiatives generally
  • Risk and protective factors:
  • School mental
  • health approach
  • Child and family
  • competencies
  • Change over time in:
  • Students (MH)
  • Parents
  • Teachers
  • Schools

State Project Officers

School engagement & implementation of KM, Facilitators, Barriers


Parent Focus Groups

Parent: Knowledge, Skill, Confidence, etc



Student MH: Parent-rated SDQ & SCS

KidsMatter Schools

100 schools across Australia


50 Round 1 started in 2007

50 Round 2 started in 2008



Student Voice

Background: gender, age, ATSI, ESL, Fee, ’at risk’


Student MH: Teacher-rated SDQ & SCS



Teacher Interviews

Teacher: knowledge, confidence, SEL, etc

Leadership Interviews

Principals & KM Leaders

KM Executive Summary, ‘pill & dose’


kidsmatter evaluation measures
KidsMatter evaluation measures

Student Mental Health Outcomes

SDQ (Goodman, 2005)

the four kidsmatter components
The Four KidsMatter Components

how welcomed members feel and their sense of belonging.

provision of social and emotional learning, professional development

parenting education and support.

supporting students who are experiencing, emotional - social - behaviour difficulties.


The analysis of change undertaken in the evaluation used:

Latent class analysis

Structural equation modelling

Multilevel modelling

Thematic analysis of qualitative data

Effect Size

We used the part-correlation coefficient r for reporting all effect sizes. We use correlations of 0.10, 0.24, and 0.37 as indicative of small, medium and large effects, respectively (Hattie,2009)

impact of kidsmatter overview
Impact of KidsMatter Overview

“[KidsMatter] has changed school culture, I think. It’s changed the way the school views mental health. It’s given a greater awareness, but it’s also changed the way, people relate to one another, particularly the students, and the way the classrooms operate”. (School Principal)

impact of kidsmatter on schools and teachers
Impact of KidsMatter on Schools and Teachers
  • 22% increase in teachers who ‘strongly agreed’ that schools were engaged with KidsMatter
  • 31% increase in schools using the 7-Step Implementation process
  • Progress on the four Components:
    • most progress on 2: Social and Emotional Learning for Students
    • least progress on 4: Early Intervention for Students

Why was least progress made on Component 4?

impact of kidsmatter on schools and teachers teachers knowledge competence and confidence
On developing social and emotional competencies, teachers perceptions were that, as a result of KidsMatter:

more knew how to help students

more ‘strongly agreed’ that the staff acted to help students

more ‘strongly agreed’ that their teaching programs helped students

more ‘strongly agreed’ that they had good self-efficacy

for teaching SEL

Impact of KidsMatter on Schools and Teachers:Teachers’ knowledge, competence and confidence

Impact of KidsMatter on Families

Parents perceptions were that, as a result of KidsMatter:

more became involved with the school

more had increased their capacity to help their children with social and emotional issues

more felt their school better catered for their child’s needs

Parenting knowledge and approach showed little change from the initial (self-reported) high levels

impact of kidsmatter on student mental health
Impact of KidsMatter on Student Mental Health

Strengths go up

Difficulties go down

KidsMatter achieved its major aims to improve student mental health and well-being and reduce mental health difficulties

impact of kidsmatter on student mental health1
Impact of KidsMatter on Student Mental Health

KidsMatter achieved a shift for a number of students within the ‘borderline’ and ‘abnormal’ ranges, into the ‘normal’ range

The numbers vary depending on how you assess mental health

“Mental Health is not simply the absence of mental disorder or illness, but also includes a positive state of mental well-being” (World Health Organisation, 2004)

methodological notes and limitations
Methodological Notes and Limitations

Strengths of the evaluation design

Multi-methods and tools

Multiple informants

Multiple time points

Multiple settings

Ecologically valid (based in authentic settings)

Stratified school selection process

The analysis of change

undertaken in this report uses

Latent class analysis

Structural equation modelling

Multilevel modelling

Thematic analysis

Limitations include

The Initiative is not, and was not designed to be, a strict randomly controlled trial. Because the sample is not a true random sample, caution should be taken if generalising findings to other students and other primary schoolsin Australia.



KidsMatter had a positive impact on schools, teachers, parents, and children

  • changed school culture
  • the ‘whole school’ approach enhanced academic and social competencies
  • facilitated Social and emotional learning for students, Parenting support & education, and Early intervention
  • strengthened protective factors within school, family & child
  • improved student mental health
“Look it really works. It can change school culture, which changes the way kids relate. It really does. By having that focus and by really thinking about how kids relate to one another; how the staff relate to the children and teaching them a set of relationship skills to help them cope. You can really make a profound difference in your school and in those children’s lives” (School Principal)


km disability
KM & Disability
  • The parents and teachers of 555 South Australian primary school students participated in the KidsMatter Evaluation.
  • Of this group, 494 South Australian students 61 students had an identified disability (11%) ,

co-morbidity is present and a student with a disability is more likely to also have mental health problems.

using the SDQ students without a disability had 1 in 8 chance of having mental health difficulties (i.e.being in the abnormal range),

students with one identified disability had a 1 in 3 chance

and students with multiple disabilities had a 1 in 2 chance


Results (Cont’)

The findings indicate, on average, a practically significant (parents=large & Teachers = medium) improvement in student wellbeing ie increase on the ‘Mental Health Strengths’ scale.

and a practically significant (Parents=small effect size & Teachers = medium effect size)decrease in mental health difficulties over time ie reduced SDQ (difficulties) scores

What else can be done for students with disabilities?

findings sec academic performance
Findings –SEC & Academic Performance
  • This research now extended to investigate the impact of KidsMatter on academic performance, based on the National Assessment Program - Literacy and Numeracy 2008 data (NAPLAN).
  • A two-level hierarchical linear model, using HLM-5

was employed to examine the relationships between school-level characteristics and student-level academic outcomes (Raudenbush &Willms, 1995).

implementation quality of whole school mental health promotion and students academic performance
Implementation quality of whole-school mental health promotion and students’ academic performance
  • a child’s social and emotional wellbeing impacts on their academic performance. (CASEL,2008)
  • BUT there is growing concern about how schools evaluate the implementation of such programs, and of the program’s effectiveness if the quality of implementation is not maintained.
  • few studies, internationally, report the relationship between quality of implementation of mental health promotion initiatives and student outcomes, eg academic performance
implementation index
Implementation Index
  • An Implementation Quality Index was developed (Greenberg, 2004; Domitrovich,2008; Askell-Williams, 2011).
  • revealed that the extent to which children’s social-emotional competencies improved over time was dependent, on how well the school had implemented KidsMatter (small effect size=.22).
findings cont
Findings (cont.)
  • Schools that implemented KidsMatter well also had improved learning outcomes for students of up to six months more schooling, over and above any influence of socio-economic background (Dix et al, 2011).
  • Over the 2 years of KM, 14% more teachers strongly agreed (scored 6 or 7) that KidsMatter had led to improvements in students’ schoolwork, equivalent to a medium (b = 0.30, p < .05) effect size.

Quality Assurance: “If we keep on doing what we have been doing, we are going to keep on getting what we have been getting”. (Wandersman et al, 2008)

  • The initiative is now approved for broader dissemination to 1700 schools by 2014
  • Upscaling of KM affects how KM is being disseminated post pilot
  • All KM stakeholders have an interest in ensuring that KM continues to be implemented in the manner that optimises the mental health and wellbeing of students, as well as achieving positive outcomes for parents and teachers.



Quality Assurance

has been seen as one way of maintaining and assessing quality outcomes

“concerns remain about the

transferability of intervention models beyond well-controlled settings” (Humphrey et al,2010,p.215)

defining quality assurance
Defining Quality Assurance
  • The development of evaluation standards is one part of a move toward ‘evidence-based’ practice.
  • “the determination of standards, appropriate methods and quality requirements by an expert body, accompanied by a process of inspection or evaluation that examines the extent to which practice meets the standards”.

Murgatroyd and Morgan (1993, p.45)


Quality Assurance








How to translate our research to outside settings?

(Slee, Murray-Harvey, Dix & van Deur,2011)


Promotion - How well information about the existence and value of the programme is promoted to new schools and the broader service community.

Readiness - This outlook refers to the extent to which the school recognises there is a problem to be addressed, is willing to address it, and has the capacity to do so.

Adoption –With the support of the staff, parents and carers, the principal commits to implementation on behalf of the school community. This stage may also be informed by local community agencies.

Implementation - The program must meet the local needs of the community in which the intervention is taking place and must be responsive to local conditions.

Sustainability - Whether the fidelity, dosage and quality are maintained over time

Monitoring - An accurate monitoring and feedback system. This aspect is central to quality assurance.

Incentive – infrastructure that provides incentives or recognition to schools and individuals within schools for achieving implementation milestones.

Slee, P.T., Murray-Harvey, R., Dix, K; van Deur, P (2011).


Researchers need to innovate & take risks in developing partnerships & exchanging knowledge to influence policy & practice

research to policy practice
Research to Policy & Practice
  • “The vitality and sustainability of a society depend on the extent to which it equalizes opportunities early in life for all children to achieve their full potential and engage in responsible and productive citizenship”(Shonkoff & Bales,2011p.117).
research to policy practice cont
Research to Policy & Practice (Cont’)

“Science has an important role to play on advising policy makers on crafting effective responses to social problems that affect the development of children” (Shonkoff & Bales, 2011,p.17)


“This is not an initiative for poor schools with disadvantaged families, it’s an initiative for all children in primary schools and all types of schools” (School Principal)


The Flinders University’s Evaluation overall main recommendation was “that the broad framework, processes and resources of KidsMatter be maintained as the basis for a national rollout”

Federal Minister for Mental Health & Aging

announced a rollout to 1700 schools with an additional $18.4m.

the change process ensec
The Change Process & ENSEC

Gladwell (2002) identifies 3 principals for creating social change:

  • The law of the few
  • ‘stickiness’ –the memorable key elements of the message
  • The power of context

“Do not go where the path may lead, go instead where there is no path, & leave a trail”.

Ralph W. Emerson

Thank You

the kidsmatter initiative
The KidsMatter Initiative
  • KidsMatter (KM) - Australian national primary school mental health promotion, prevention and early intervention initiative.
  • KM - developed in collaboration with the Australian Government Department of Health and Ageing, beyondblue: the national depression initiative, the Australian Psychological Society, and Principals Australia
  • KidsMatter uses a whole-school approach. It provides schools with

# a framework,

# an implementation process, and

# key resources to develop and implement evidence-based mental health promotion, prevention & early intervention strategies.

comparisons of effect sizes
Comparisons of effect sizes

Effect size: Part correlation r

‘social and emotional education across the world: celebrating diversity in theory, research and practice’.