Measuring Success Outcomes versus Outputs

1. Measuring Success Outcomes versus Outputs By: Bryant Christie Inc. Mike Rucier June 2012

2. Training Objectives • For QSSB staff to: • Make informed decisions regarding project approval and implementation • Strengthen linkages in project proposals between objectives, strategies, and intended results among target audiences • Develop a common methodology for defining success • Understand the difference between project outcomes versus outputs • Gain practical knowledge on ways to measure results

3. Presentation Outline • Target Identification • The Development Continuum • Defining the problem(s) / constraints • Establishing objectives • Measuring success • What to measure • How to measure • Project case studies: • Education Program • Promotion Program • Social Media

4. Market Development Process Goal Constraint(s) Projects Target(s)

5. Strategic Approach Targeted/Strategic Target Tactical/Opportunistic

6. Targeting • Identify potential target groups based on strategic initiatives • Is this the right target audience? • Is the description of the target audience too broad? Can you be more specific? • Are the needs of individuals in the proposed target audience similar/different? If different, then consider identifying them as a separate audience. • Can activities reach this target audience? • Would reaching this target audience make a difference in achieving overall program goals? • How does this target audience relate to broader organizational goals? • Where is the target on the development continuum? If they are at the top of the continuum, do you still need to address this audience?

7. Development Continuum Increasing product demand • Who is being targeted? • Where is the target on the development continuum? What is their current knowledge/purchase level? • What is the desired change in behavior necessary to move them to the next level?

8. Constraints • Describes what is preventing target from moving to next step in development continuum • Constraint statement should be specific and concise (conveyed in one or two sentences) • Constraint is a corollary of the objective • Purpose of projects is to address constraints • There should be a clear linkage from the constraint, objective, and strategy

9. Constraint Example • To develop a constraint statement, answer the following: • What is preventing the target from moving to the next step on the development continuum? • What is the next step in the development continuum and how does that relate back to buying more U.S. soy? • Example: The Chinese animal and feed production industry lacks an understanding of of the technical requirements (define) of using soy-based solutions in feed. This is preventing the target from using a greater percentage of soy in feed formulations.

10. Measuring Success What to Measure

11. Evaluating Progress • Are you making progress toward overcoming the constraint and reaching goals? • How can you tell that you are making progress? • If you are making progress, why is that? • What contributed to the progress • Can you link the success to specific projects? • Or, was the success the result of some other external factor? Some information provided courtesy of The Hale Group, Ltd.

12. Evaluating Progress (Con’t) • If you are not making progress, why not? • Were your projects ineffective? • Was it poor planning or poor implementation? • Or, did some external factor prevent the achievement of your objectives? • Are the basic assumptions of the project sound? • Are the identified constraints still relevant? Some information provided courtesy of The Hale Group, Ltd.

13. What is a KPI? • “Measure of a desired change in behavior of a target audience over a period of time” • Identifies where the target is on the development continuum and helps gauge progress along the continuum

14. Outcomes versus Outputs • Confusion between outcomes and outputs: • Outcomesare the desired change in behavior of your target audience • Outputs measure the work that you did as part of a project Some information provided courtesy of The Hale Group, Ltd.

15. Outcomes Measures the extent to which the target actually moved from one step to the next on the development continuum. Some information provided courtesy of The Hale Group, Ltd.

16. Outputs Measures the work you did to motivate the target to take the next step. Target Some information provided courtesy of The Hale Group, Ltd.

17. Output Examples • 235 sample products were given to consumers • Explained results of comparative tests to 6 buyers • Convinced 80 people to attend a buying seminar • Conducted survey of 450 consumers Some information provided courtesy of The Hale Group, Ltd.

18. Outcome Examples • 235 sample products were given to consumers • Result: 60% of surveyed consumers purchased the product after receiving a sample • Explained results of comparative tests to 6 buyers • Result: 5 of the 6 buyers said they understood the differences between our product and the competition. • Result: 4 of the 6 buyers decided to switch to our product from the competition after receiving information about the differences • Convinced 80 people to attend a buying seminar • Result: 75% of those attending the seminar sought further information about suppliers • Conducted survey of 450 consumers • Result: 35% of surveyed consumers choose SBO because of its health benefits

19. Measuring Success How to Measure

20. Considerations • How will success be measured? • How will information be collected? • Is the method to collect information cost effective? • Are there resources available to measure impact? • Is the measurement achievable? • Does the measurement accurately convey success? • Is there a better measurement of success?

21. Additional Considerations • Measurement methodology greatly depends on budget and type of project • Measurement methodology should be identified in the project description and the cost of collecting the data included in the budget • Ideally the measurement mythology is cost-effective yet will still accurately convey location and movement on the development continuum • Can the data collection be done as part of the project?

22. Spheres of Influence

23. Measurement Methodologies What is the change in behavior?

24. Measurement MethodologiesCase Study #1: Education Program • AG Adventure Program • Target: School Aged Children • Objective: learn more about soybeans and agriculture • Strategy/Tactics: mobile units which tour schools and to teach about science and animal agriculture • Evaluation: None noted

25. Measurement MethodologiesCase Study #2: Promotion Program • Regional Biodiesel Promotions • Target: Regulated Fleets, consumers, farmers • Objective: to encourage the use of biodiesel • Strategy/Tactics: answer requests for info; promote biodiesel use; promote biodiesel retail and wholesale facilities; identify best way to use check-off funds • Evaluation: None noted

26. Measurement MethodologiesCase Study #3: Social Media • Producer Communication • Target: Producers • Objective: (1) inform how check-off funds are invested to afford profit opportunities; (2) instill confidence in administrative quality; (3) create interest in becoming involved in program • Strategy/Tactics: Several communication activities • Evaluation: National survey of growers

27. Summary • Understand the market • Identify target and understand position on development continuum • Articulate the challenge in moving to the next step on development continuum (define the constraint) • Develop objective and strategy for moving target to the next step • Establish KPIs to understand movement on the continuum • Execute activities with focus on outcomes rather than outputs • Monitor progress based on results against KPIs

28. Project Development Template

29. Social Media Monitoring • Facebook • apps.facebook.com/ facebookgrader/ • edgerankchecker.com • Twitter • www.tweetdeck.com • cotweet.com • www.twilert.com • twitaholic.com • wefollow.com • thetwitcleaner.com • followerwonk.com • tweetmeme.com • General • www.hubspot.com • hootsuite.com • klout.com/understand/score • seesmic.com • www.socialmention.com • www.feedly.com • summify.com • www.radian6.com • www.lithium.com • www.viralheat.com • www.dandyid.org • simplymeasured.com