The days ahead

1. The days ahead • Monday-Wednesday • Training workshop on how to measure the actual reduction in HIV incidence that is caused by implementation of MC programs • Thursday & Friday • Share experiences about the problems countries are encountering in initial scale-up of MC services – and the solutions they have found. Clarify what the unanswered questions are about how best to implement MC. • Saturday • Explore how measurement of the impact on incidence could also answer unresolved questions about how to best implement MC (e.g. optimal task sharing…) • Monday-Wednesday next week • Third meeting on the use of the DMPPT tool to estimate the cost and forecast the possible impact of MC programs to guide initial program design (and help decision-makers decide whether they want to implement MC)

2. How can we estimate effectiveness? • E.g., What is the effect of: Male circumcision on HIV incidence • In other words:“How much does an increase in male circumcision cause HIV incidence to decrease?”

3. How can we estimate effectiveness? • If we implement MC in a community and incidence of HIV does not change does that mean that MC is not effective at reducing incidence of HIV? • If we implement MC and incidence goes up, does that mean that MC increases HIV transmission? • If we implement MC and incidence falls, do we know that the MC caused the fall?

4. How can we estimate effectiveness? • We are not asking whether incidence changes at the same time that MC is implemented – we are asking whether incidence changes because MC is implemented. • Thus, we need to know how much would incidence have changed if MC was not implemented – and how different is that from how much it changes with MC

5. How can we estimate effectiveness? • Unfortunately, we can’t rewind the clock and observe the same community over the same period of time with and without MC • We need an estimate what would have happened to HIV incidence in the absence of an MC program • We call this the counterfactual • i.e. the “factual” is what really happens where MC is implemented and the “counterfactual” is what would have happened if MC had not been implemented

6. Creating a Good Counterfactual • Sometimes it is very simple: • If I dye my hair tonight and come in tomorrow with black hair you will easily believe that the change in hair color was caused by the hair dye • In other words, if we strongly believe that something is not changing on its own, then we can assume that the counterfactual = baseline • Good example of that for HIV is treatment

7. Creating a Good Counterfactual • With HIV prevention it is never simple

8. Creating a Good Counterfactual Baseline MC Implemented

9. Creating a Good Counterfactual IMPACT Baseline MC Implemented

10. Creating a Good Counterfactual IMPACT ? MC doubles HIV incidence? MC Implemented Baseline

11. Creating a Good Counterfactual Real IMPACTMC Halves HIV incidence MC Implemented Baseline

12. Creating a Good Counterfactual IMPACT?MC reducesHIV incidence by 1/2 Baseline MC Implemented

13. Creating a Good Counterfactual Real IMPACTMC reducesHIV incidenceby 1/3 Baseline MC Implemented

14. Creating a Good Counterfactual • Where in the World Are We?

15. Creating a Good Counterfactual • Where in the World Are We? • If the purpose of the experiment was to test the effectiveness of the hints, should we have given the hints to the “experts” or to the rest of us?

16. Creating a Good Counterfactual EFFECT Ideally you want the communities that receive MC to be indistinguishable fromthose that don’t Low INCIDENCE High

17. Creating a Good Counterfactual NEGATIVE EFFECT? If they are different, it can bias the estimation of the effect Low INCIDENCE High

18. Creating a Good Counterfactual HUGE EFFECT? If they are different, it can bias the estimation of the effect Low INCIDENCE High

19. Creating a Good Counterfactual • The beneficiaries of the intervention and the counterfactual or control groups: • have identical characteristics, except for benefiting from the intervention • With a good counterfactual, the only reason for different outcomes between treatments and controls is the intervention (I)

20. Creating a Good Counterfactual • The beneficiaries of the intervention and the counterfactual or control groups: • have identical characteristics, except for benefiting from the intervention • With a good counterfactual, the only reason for different outcomes between treatments and controls is the intervention (I)

21. Creating a Good Counterfactual for MC • What if we compared the incidence among the men in a community who decide to get circumcised with the incidence among the men who decide not to get circumcised? • What if we compared the incidence in communities who decide to initiate circumcision services in their health center with the incidence in the ones that don’t? • What if we compare the incidence in towns without a clinic with the incidence in towns with a circumcision clinic?

22. Creating a Good Counterfactual for MC • What if we opened up a circumcision clinic in Soweto and another in Khayelitsha and another in KwaMashu – but they could only serve a small number of the men who want to be circumcised. If we give lottery tickets to see who can go in what month, could we use the ones who will wait a year as a comparison (counterfactual) group? • What if we focused circumcision services on the 15-24 year olds. Could we use the 25-30 year olds as a comparison group?