Evaluating the effectiveness of social media in promoting audience engagement

1. Evaluating the effectiveness of social media in promoting audience engagement The Boss of Me case study Jennifer Berktold Senior Study Director Westat National Conference on Health Communication, Marketing, and Media, Atlanta, Georgia, August 11, 2009

2. “I don’t want no drama in my life, even though we have a little bit, but no more letting people control you. That’s drama, because then you become something that you’re not.” -- Mary J. Blige

3. Reframing the Issue INTIMATE PARTNER VIOLENCE Teens don’t achieve true Intimacy Partners are constantly being redefined making the word less relevant Reducing violence is our ultimate goal but strategy should be prevention EARLY RELATIONSHIP CONFLICTS Formative years should be emphasized Relationship is a more open term reflective of shifting expectations Conflicts can be a frequent precursor to violence

4. The Strategic Approach Five key strategies from research for reaching teens (ages 14 – 18, both genders, all races and ethnicities) on this issue: 1) Take an empathic and non-judgmental stance 2) Target the “drama” with humor and authentic teen language 3) Promote self-empowerment 4) Offer tools and resources to “dial down the drama” 5) Leverage teens’ organic media culture

5. BOM or “Boss of Me” Being the boss of you means you’re in complete control of you at all times in your relationships. It means you refuse to submit to your own feelings of anger, fear, jealousy or insecurity. When you allow your emotions to negatively influence your actions, you lose control. And when you lose control, you’re not the “boss of” anything. BOM (“Boss of Me”) is dedicated to helping young people stay in charge of their own relationship drama. BOM provides tools to help teens maintain and manage their cool, instead of losing it, during a relationship conflict.

6. Organizational Partners Funder Managing the Campaign Brand Management Evaluation

7. Evaluating BOM

8. Research goals • Identify delivery mechanisms that engage young people with BOM • Understand how young people engage with BOM • Gender differences • Understand how the campaign impacts its target audiences

9. Intended uses of BOM evaluation results • Learning, best practices • Communicate with the teen dating violence prevention community • Effective delivery mechanisms to reach teens • Engaging messages / content • Appeal to new audiences?

10. Inputs Activities Outcomes Outputs Products from activities • Web traffic • Number of radio shows developed • Number of ads • Number of events • Reach Changes for participants • Increased knowledge • Attitudinal change • Behavior change What is done • Website • Radio show • Ads / PSAs • Community events Resources • Money • Staff and volunteer time • Facilities Basic model of an outcome evaluation

11. Outputs, Outtakes, Outcomes Outtakes Outcomes Knowledge change Attitudinal change Behavior change Outputs • Products • Activities • Reach • Awareness • Buy-in • Engagement • Participation

12. New Logic Model, Simplified Outtakes Inputs Outcomes Activities Outputs Changes for participants • Increased knowledge • Attitudinal change • Behavior change What is done • Tools development • Website development • Ads / PSAs • Community events • Contests Participation • Creative content • Conversation content • Tone of conversation • “Friending” • Texting opt-in Resources • Money • Staff and volunteer time • Facilities Products from activities • Web traffic • Number of videos developed • Number of ads/ad buy • Number of events • Reach

13. Approach

14. Our challenge… • Heard of BOM but hasn’t engaged • Radio ads / PSAs • Outreach at school • Engaged with BOM • BOM411 site visits • Social media friends • Text messaging • Community events • User-generated content Hasn’t heard of BOM

15. 5-Point Evaluation Plan • Monitoring • Content analysis • Web survey of teens • Pop-up survey at BOM411 • Visits to community events

16. Web Monitoring BOM Sites • BOM411 • Facebook • MySpace (2 sites) • Twitter • YouTube • Outside BOM • RSS (e.g., Google blogs, Google news) Data • Google analytics • Comment text • Comment analysis • Articles/posts with BOM mentions

21. Content analysis Subjects • Posted content on BOM411 • Posted content on social networking sites • User-generated content • Skateboard designs • Rap lyrics • Outside buzz Measures (some examples) • Type of conversation • Affect • Attributes • Type of information shared

22. Other Monitoring Measures (some examples) • GRPs • Number of subscribers • Number of texts • Number of phone messages Subjects • Radio • Cell phones • Voicemail line

23. Web survey of teens Sample • Teens 14-18 in SF Bay Area and Los Angeles • Recruited from panel that agreed to take surveys online • Parental consent obtained • Knowledge Networks Measures • Self-reported exposure to campaign • Awareness of relationship drama among peers • Attitudes toward / relationship with BOM • Self-reported attitude/behavior change

24. Pop-up survey at BOM411 Sample • Random selection of visitors to BOM411 Measures • How found out about website • Engagement with website content (e.g., rating different parts of website, would you recommend to others)

25. Visits to community events Sample • Random intercept interviews • Interviewer observation Measures • How found out about event • Why attending the event • Activities performed at the event • Description of event • Estimated attendance

26. Timeline

27. Questions

28. Thank you! Jennifer Berktold Senior Study Director Westat jenniferberktold@westat.com