Dr andrea baldwin creative industries faculty qut
This presentation is the property of its rightful owner.
Sponsored Links
1 / 68

Dr Andrea Baldwin Creative Industries Faculty QUT PowerPoint PPT Presentation


  • 44 Views
  • Uploaded on
  • Presentation posted in: General

Evaluating. Dr Andrea Baldwin Creative Industries Faculty QUT. a Train the Trainer program in Papua New Guinea. WHAT’S THE PROJECT?. Sexual health – focus on STIs and HIV Behaviour change communication using applied theatre

Download Presentation

Dr Andrea Baldwin Creative Industries Faculty QUT

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Dr andrea baldwin creative industries faculty qut

Evaluating

Dr Andrea Baldwin

Creative Industries Faculty

QUT

a Train the Trainer program

in Papua New Guinea


Dr andrea baldwin creative industries faculty qut

WHAT’S THE PROJECT?

  • Sexual health – focus on STIs and HIV

  • Behaviour change communication

    using applied theatre

  • Experiential learning, not one-way dissemination of health information

  • Train the trainer model

  • Ultimate beneficiaries are young people (15-25)

  • Developing hybrid forms of applied theatre for education/ health promotion


Dr andrea baldwin creative industries faculty qut

WHO ARE THE TEAM?

  • Professor Brad Haseman

    Creative Industries, QUT

  • Associate Professor

    Anne Hickling Hudson

    Education, QUT

  • Dr Andrea Baldwin

    Creative Industries, QUT

  • Ms Hayley Linthwaite

    Creative Industries, QUT

  • Ms Jane Awi

  • Creative Industries, QUT

  • Mr Martin Tonny

  • Research and Administrative

  • Assistant

  • Ms Jackie Kauli

  • Creative Industries, QUT

  • Research Advisory Groups

  • and Participants in Field Sites


Dr andrea baldwin creative industries faculty qut

WHO ARE THE PARTNERS?

  • Australian Research Council – Linkage Grant

  • Queensland University of Technology

  • Griffith University

  • National AIDS Council Secretariat PNG

  • University of Goroka

  • University of Papua New Guinea


Evaluation framework

Evaluation Framework

Haseman, 2006

Stufflebeam, 2003


Dr andrea baldwin creative industries faculty qut

THE MAP IS NOT THE TERRITORY


Dr andrea baldwin creative industries faculty qut

  • Spiral Out:

  • who learnt what from this project?

  • what impacts did the project have – positive and negative, expected and unexpected?

  • what does this mean for the future?

  • descriptive, expansive answers

  • qualitative, performative

  • Spiral In:

  • what’s the bottom line?

  • did implementation go to plan?

  • did the project create the desired change?

  • what are the take-home learnings?

  • yes/no, definitive answers

  • quantitative


Research evaluation methodologies must

RESEARCH/EVALUATION METHODOLOGIES MUST:

  • Be culturally appropriate, meaningful and relevant to the participants and other stakeholders

  • Be sensitive to issues of advantage/ disadvantage for individuals and groups of stakeholders

  • Complement and advance the aims and approach of the overall project


A conceptual metaphor

A CONCEPTUAL METAPHOR

PROJECT

What have we learnt about propagation?

PROGRAM

Is the seed viable?


Evaluation research

EVALUATION RESEARCH

  • Involves determining the worth, merit, or quality of an evaluation object, such as an educational program

  • Formative evaluation – helps in the design, implementation and improvement of the program

  • Summative evaluation – helps make decisions about whether a program should be supported into the future

    Johnson & Christensen, 2008


Magic pills

MAGIC PILLS?

  • Drug trial model

  • Laboratory

  • Quantitative, use of inferential statistics

  • Clear distinctions between the agent (drug) and conditions of administration (dosage, interactions, etc.)


Participatory action research

PARTICIPATORY ACTION RESEARCH

  • Much more organic – can’t make such clear distinctions between the agent (program, action, education materials) and the context (community or organisation)

  • Requires more holistic approach to collecting and analysing data, interpreting findings, making recommendations for future


Important to

IMPORTANT TO:

  • Capture negative/undesirable impacts of the project as well as positive/desirable ones

  • Capture and assess unexpected impacts of the project as well as intended/expected ones


New phase new approach

NEW PHASE, NEW APPROACH

The Life Drama Program

The Train the Trainer Program

Life Drama program content established

Train the Trainer program content and structure established – Handbook and Bilum

Collect qualitative and performative data from two sites

  • Development highly participatory – Tari, Madang, Karkar Island

  • Quantitative, qualitative and performative data collected

  • Triangulating and synthesising vast quantity of data from participants and other stakeholders


What more have we learnt about the seed

WHAT MORE HAVE WE LEARNT ABOUT THE SEED?

  • Have the educators themselves acquired new knowledge and attitudes in relation to sexual health and HIV?

  • Interview/focus group data

  • Performative data

    2. Do the educators believe their students will acquire/have acquired new knowledge and attitudes through the training?

  • Post and follow-up interview data


What more have we learnt about propagation

WHAT MORE HAVE WE LEARNT ABOUT PROPAGATION?

  • What factors enhance or impede

  • the delivery and effectiveness of training Life Drama trainers?

  • the formation and functioning of a network of trainers?


Dr andrea baldwin creative industries faculty qut

2. What are the similarities and differences between training community educators and teacher educators, which are likely to impact differently on the effectiveness of these two groups as trainers?

3. What are the similarities and differences that are likely to impact on their ability to support one another as a functioning network?


Generalisability

GENERALISABILITY?

  • These questions being investigated in PNG – two specific sites/groups – capture realistic picture of this specific scenario

  • Interpretation of data will include the attempt to draw out general principles that might apply in other settings


Dr andrea baldwin creative industries faculty qut

“BASIC”

EVALUATION


Dr andrea baldwin creative industries faculty qut

  • PROGRESS REPORTS – is the project being implemented as designed? Is the research team doing what it said it would do? Are budget milestones and timeframes being met? If not, why not? (emphasis on Context, Inputs and Processes)

  • FINAL REPORT– did the project achieve intended outcomes hoped for? What factors enhanced or hindered the achievement of these outcomes? (Emphasis on Context and Products)


Objective 1 trainee group

OBJECTIVE 1: TRAINEE GROUP

  • To train between 20 and 40 Life Drama trainers, with an equal balance of male and female trainers, through two training hubs:

  • University of Papua New Guinea, Port Moresby, National Capital District

  • University of Goroka, Goroka, Eastern Highlands


Evaluation

EVALUATION

Quantitative data – how many males and how many females completed training?

Performative data – how many trainees passed requirements for certification?


Requirements for certification

REQUIREMENTS FOR CERTIFICATION

By completion of the training, trainees will demonstrate -

  • A practical knowledge of contextual, communicative practices and their place in society;

  • A deeper awareness of community issues, in particular sexually transmitted illnesses and HIV;

  • Specific skills in using drama techniques confidently to help community groups explore community issues, in particular sexually transmitted illnesses and HIV;

  • Leadership skills which help community groups find their own solutions to those problems and barriers which are preventing widespread understanding of, and positive response to, the challenges of sexually transmitted illnesses and HIV;

  • Critical, analytical and creative approaches about connections between purposes, audiences and diverse ways of communication;

  • Individual and group interaction leading to brain-storming, understanding and action;

  • Attitudes appropriate to inclusive communication practices for cultural, linguistic, gender differences and collaborative contexts.


Objective 2 trainer network

OBJECTIVE 2: TRAINER NETWORK

2. To support the trainee group to function as a “trainer network” – sharing practice, exchanging ideas, assisting one another’s professional development, and expanding knowledge of the Life Drama program beyond the original group of trainees


Evaluation1

EVALUATION

Qualitative data – post-training and follow-up interview and focus group data on factors that may enhance or hinder/have enhanced or hindered the functioning of the trainer network


Objective 3 trainer effectiveness

OBJECTIVE 3: TRAINER EFFECTIVENESS

3. To assist participants to integrate Life Drama techniques with their existing educational skills and strategies, to enhance their effectiveness as trainers of HIV-related material


Evaluation2

EVALUATION

Qualitative data – post-training and follow-up interview and focus group data on how trainees intend to use/ are using their Life Drama skills for HIV education purposes

Performative data – review in-training video footage of trainers’ practice, and follow-up practice


Objective 4 research capacity

OBJECTIVE 4: RESEARCH CAPACITY

To build research capacity in Papua New Guinea:

  • Participatory Action Research project involving senior staff in arts health areas of University of Papua New Guinea and University of Goroka

  • Develop research skills of PNG national members of the research team

  • Encourage research participants to gather data as “co-researchers”


Evaluation3

EVALUATION

Quantitative data – how many PNG national staff from the two universities are actively involved in the project?

Qualitative data – what is the level of engagement/mutual benefit for the PNG national staff and the QUT research team?

Qualitative data – Quality of reflective journalling and reporting by participants as co-researchers


Group 1

GROUP 1

UNIVERSITY OF

PAPUA NEW GUINEA

PORT MORESBY


Dr andrea baldwin creative industries faculty qut

  • Port Moresby organisations:

  • University of Papua New Guinea

  • Lecturing staff

  • HIV peer educators

  • AnglicareStopAIDS

  • National Capital District Commission

  • Centre for Democratic Institutions PNG

  • Family Health International PNG

  • Motukoita Community Youth Development

  • Department of Education

COMMUNITY EDUCATORS

Hela Province Organisations:

Porgera Joint Venture

PNG Police Service

Hela Rural Women’s Foundation

OilSearch


Group 2

GROUP 2

UNIVERSITY OF GOROKA

GOROKA, EASTERN HIGHLANDS


Teacher educators

TEACHER EDUCATORS

  • University of Goroka:

  • U100 (mandatory HIV course)

  • Language and Literature Department

  • Social Sciences Department

  • Expressive Arts Department

  • Science Faculty

  • Education Faculty

  • Early Childhood Department

  • Student Services

  • Madang Teachers College

  • Oxfam PNG

  • Independent Community Theatre Practitioners


Objective 1 trainee group1

OBJECTIVE 1: TRAINEE GROUP

Port Moresby

  • 6 females

  • 14 males

  • 20 TOTAL

  • Goroka

  • 15 females

  • 11 males

  • 26 TOTAL

TOTAL = 46 TRAINEES, 25 MALE AND 21 FEMALE


Objective 2 trainer network1

OBJECTIVE 2: TRAINER NETWORK

  • Only post-training focus group data available so far (follow-up focus group data to be collected in November 2011)

  • Only in-training video performative data available so far (follow-up video of trainees’ practice to be collected in November 2011)


Factors enhancing

Factors Enhancing

  • Membership of same organisation (eg. university staff)

  • Ability to access communication technology (internet, email, phone)

  • Organisational support (esp. U of G)

  • Ongoing relationship with QUT

  • Ongoing resourcing


Factors impeding

Factors Impeding

  • Membership of different organisations

  • Working with different client groups – different needs

  • Communication barriers (few trainees have internet access; internet access often unreliable in PNG; phone access ditto; phone communication expensive)

  • Lack of resourcing


Objective 3 trainer effectiveness1

OBJECTIVE 3: TRAINER EFFECTIVENESS

Qualitative Data:

  • Only post-training focus group data available to date. Follow-up focus group/ interview data will be collected in November 2011


Dr andrea baldwin creative industries faculty qut

Performative Data:

  • Only in-training video available for review to date. Follow-up video will be collected in November 2011

  • Video to be analysed by members of team expert in:

    • Drama-in-Education/ Theatre for Development (Creative Industries Faculty)

    • Pedagogy (Education Faculty)


Objective 4 research capacity1

OBJECTIVE 4: RESEARCH CAPACITY

  • # PNG senior university staff actively engaged in project, not as trainees: 7 (5 in Goroka, 2 in Moresby)

  • “Active engagement” evidenced by: participating in training sessions, observing training sessions, providing additional supports and assistance to the program, contributing to focus groups


Dr andrea baldwin creative industries faculty qut

  • Level of mutual benefit for PNG national research staff:

  • 2 PNG national research staff undertaking PhDs with QUT as a result of Life Drama project (assisting with data collection and analysis for Train the Trainer program)

  • 1 PNG national research staff member now undertaking Masters with another university, partly as a result of involvement in Life Drama

  • QUT and University of Goroka now exploring future partnership opportunities


Dr andrea baldwin creative industries faculty qut

  • Quality of reflective journalling and report provision by participants – yet to be assessed (November 2011)


Dr andrea baldwin creative industries faculty qut

TWO GROUPS: COMPARE AND CONTRAST


Context

CONTEXT

Port Moresby

Goroka

Primarily teacher educators

Primarily local (living and working in Goroka)

Most trainees affiliated with University of Goroka at some level

  • Primarily community educators

  • Significant percentage from outside Moresby or away from the university (eg. settlement areas)

  • Large range of organisations represented, sometimes by 1 or 2 people


Inputs

INPUTS

Port Moresby

Goroka

Two indoor venues in good repair but not always available; using outdoor spaces without disturbing exams

More organisational commitment – more resourcing – more condusive learning environment

Issues over resourcing for non-local attendees

  • Less organisational commitment – less resourcing – more difficult learning environment

  • Indoor venue in poor repair

  • Several trainees self-funding attendance – resentments over resourcing


Processes

PROCESSES

Port Moresby

Goroka

Handbook available throughout training, used for daily review of lessons

Trainees all comfortable with English

  • Handbook not available until last day

  • Trainees generally less comfortable with English


Products

PRODUCTS

Port Moresby

Goroka

Easier to follow up and evaluate

We expect more effectiveness of trainer network because most participants have access to communication mechanisms and infrastructure, and/or are geographically co-located

  • Harder to follow up and evaluate

  • We expect less effectiveness of trainer network because fewer participants have good access to communication mechanisms and infrastructure


What more have we learnt about the seed1

WHAT MORE HAVE WE LEARNT ABOUT THE SEED?

  • Have the educators themselves acquired new knowledge and attitudes in relation to sexual health and HIV?

  • Interview/focus group data – yes, eg. difference between HIV and AIDS, how to live healthy life with HIV

  • Performative data – yes, eg. condom demonstrations


Qualitative data

QUALITATIVE DATA

“I thought AIDS was an instant disease. But I realise now there’s a harmony in the body that tries to fight against this disease. If I’m thinking that, what about the students? We need to teach them all the things you can do to stay healthy”

Female participant, early 50s, lecturer in U100 HIV program at Goroka University. Post-training focus group.


Performative data

PERFORMATIVE DATA


Dr andrea baldwin creative industries faculty qut

2. Do the educators believe their students will acquire/have acquired new knowledge and attitudes through the training?

  • Post and follow-up interview data – yes, particularly Dancing Diseases


What more have we learnt about propagation1

WHAT MORE HAVE WE LEARNT ABOUT PROPAGATION?

  • What factors enhance or impede

  • the delivery and effectiveness of training Life Drama trainers?

    Shared language, resources, communication, venues, equipment.

    Regular debriefing/consolidation periods.

    Handbooks.


Dr andrea baldwin creative industries faculty qut

b) the formation and functioning of a network of trainers?

Shared language, organisational commitment to trainees and their professional development/ongoing peer support, access to communication, resourcing.


Dr andrea baldwin creative industries faculty qut

2. What are the similarities and differences between training community educators and teacher educators, which are likely to impact differently on the effectiveness of these two groups as trainers?

  • Similarities include degree of commitment to education, identification with the educator role

  • Differences include foundational skills (in teaching, in drama), organisational support, resourcing, communication


Dr andrea baldwin creative industries faculty qut

3. What are the similarities and differences that are likely to impact on their ability to support one another as a functioning network?

  • Similarities include degree of commitment to education, identification with the educator role.

  • Differences include organisational support, resourcing, communication


Methods

METHODS

Quantitative

  • Individual interviews with rating scale responses

  • Observer ratings of skills displayed in workshops (live and on video)

  • Recordings of numbers eg. number of participants, sessions attended

  • Trainers rating their own participants’ understanding of HIV issues


Dr andrea baldwin creative industries faculty qut

Qualitative

  • Focus groups

  • Interviews

  • Stakeholder meetings

  • Case studies

  • Participant journals

  • Lead trainer reflective journals and field notes


Dr andrea baldwin creative industries faculty qut

Performative

  • Participants observed and videoed both as participants in training workshops, and conducting training of others

  • Participants performing “stories” illustrating “best” and “worst” moments delivering training (these stories videoed)


Unexpected outcomes

UNEXPECTED OUTCOMES?

  • Teacher educators excited by new approach to pedagogy – keen to apply it in areas apart from HIV (including early childhood teaching)

  • One director keen to re-engineer her “office job” and re-engage with teaching

  • Trainers identify range of applications for Life Drama in other life roles eg. Sunday School teacher, police officer, pastor


Challenges

CHALLENGES

PARTICIPANTS

  • Communication

  • Literacy

  • Numeracy

  • Languages

  • Cultural expectations

  • Culturally-appropriate data collection “tools” and methods


Dr andrea baldwin creative industries faculty qut

PARTNER ORGANISATIONS

  • Power relations between organisations

  • Organisational and national politics

  • Perceived benefits for individuals and groups

  • Infrastructure including venues, equipment, communications

  • Travel


Dr andrea baldwin creative industries faculty qut

STUDENTS OF TRAINEES

  • Religious and cultural resistance to discussing sex, body parts

  • Resistance to condom use

  • Gender issues

  • Sensitive topics such as sexual activity with children


Dr andrea baldwin creative industries faculty qut

RESEARCH OUTPUTS


Dr andrea baldwin creative industries faculty qut

  • Life Drama Train the Trainer booklets: Life Drama Handbook and Life Drama Bilum

  • Interim Report and Final Report for participating organisations and participants

  • Journal articles

  • Conference presentations

  • Further applications of Life Drama

  • Life Drama website – evaluate separately as a tool for communicating performative outcomes


Publications

PUBLICATIONS

Baldwin, A. (2010). Dancing Diseases: An Applied Theatre Response to the Challenge of Conveying Emotionally

Contradictory Messages in HIV Education. Applied Theatre Researcher, 11.

Baldwin, A. (2010). Life Drama Papua New Guinea: Contextualising Practice. Applied Theatre Researcher, 11.

Jennings, M. and Baldwin, A. (2010). “Filling out the Forms was a Nightmare”: Project evaluation and the reflective practitioner in community theatre in contemporary Northern Ireland. Music and Arts in Action, 2(2), pp. 72-89.


Dr andrea baldwin creative industries faculty qut

QUESTIONS?

www.lifedrama.net


  • Login