ASSESSING THE QUALITY OF POPULATION SIZE ESTIMATES OF PEOPLE wHO INJECT dRUGS (PWID)

1. ASSESSING THE QUALITY OFPOPULATION SIZE ESTIMATES OF PEOPLE wHOINJECT dRUGS (PWID) Waimar Tun, Population Council 20th International AIDS Conference Melbourne, AustraliaJuly 20 – 25, 2014

2. Background • Current PWID population size estimations (PSE) in many countries are not based on strong data • UNODC and World Bank requested a review of existing PWID population size estimates in 10 countries • Belarus, China, India, Libya, Myanmar, Philippines, Kazakhstan, Kyrgyzstan, Tajikistan, & Uzbekistan

3. Learning objectives • Be able to critically review existing estimates and their methodology • Be able to identify opportunities to improve the estimates (if required) • Understand the strengths and weaknesses of methods of PSE data collection

4. Where do I get my data? • Published and grey literature (including HIV- and drug- related country reports) • Discussions with stakeholders in-country (NAP/NAC, UNAIDS, WHO, UNODC, CDC, USAID, PWID representatives, civil society organizations) • Stakeholder meeting with representatives from MoH, drug control agencies, civil society, and implementing partners

5. Comment on data sources • PSE methodologies not or poorly specified • Reports not translated or translations of technical terms were ambiguous • Stakeholders not always aware of PSE activities happening in their own country • Time-consuming

6. What should I consider when I review the quality of the estimates? • Are the underlying assumptions of the method met? • What are the potential biases and how do they impact the estimate? • Are multiple methods used? • What is the quality of the data used?

7. Common methods to estimate PWID population size

8. Literature review/desk exercise • Review published and grey literature for similar context and geographic region • The benchmark from literature is applied to the adult male and female population • Strengths: • Low cost • Little time/resources required • Weaknesses: • Local contexts may be very different • Data sources may not be based on rigorous methods

9. Delphi • Systematically solicits and reviews selected experts’ estimates • Iterative process through a series of feedback and revisions • Strengths: • Utilizes local expert views and experience • Does not require raw data capture • May be only option for countries with limited data sources • Weaknesses: • Sometimes based only on qualitative or anecdotal information • Not good for identifying trends, comparing to other regions

10. Mapping/census and enumeration • Venues where PWID congregate are identified • Census counts all PWID at all hotspots • Enumeration counts PWID at a sample of sites • Strengths: • Is a real count, not an estimate • Can produce a credible lower limit • Weaknesses: • PWID not always accessible, may not be exposed to census data collectors • Assumes you can locate all PWID • PWID may not wish to reveal drug use due to stigma and/or legal concerns • Highly time consuming and expensive • Safety issues; dangerous hotspots

11. Capture-recapture • PWID are counted and ‘tagged’; a 2nd count is conducted • An estimate is obtained through a formula that includes captures and overlaps between the 2 rounds • Strengths: • Fairly easy to implement in short period of time • Usually cost-efficient to implement • Can be done with multiple service sources • Weaknesses: • Assumptions of method difficult to meet in reality (independent samples) • May be dangerous to implement (unsafe hotspots)

12. Service multipliers • Requires two data sources: • Benchmark (service data such as drug treatmentor HIV testing) • Population-based survey with PWID where you obtain info on the proportion who report that point of contact (‘benchmark) Estimate obtained from multiplying inverse of proportion to the benchmark • Strengths: • Uses existing and available data from service providers • Easily incorporated into IBBSS with minimal additional questions • Weaknesses: • High-quality service data may not be available • No duplicates • Each target population member must have chance of being included in service data

13. Wisdom of the crowd • Ask PWID survey participants to estimate the number of PWID in a given location • Assumes that the average response approximates the actual number • Strengths: • Very easy to implement. Only one question. • Easily incorporated into IBBSS or other size estimation methods • Weaknesses: • May be biased if large segment of population is not well-networked or “hidden” • Bias if not implemented in a representative survey

14. RDS† successive sampling size estimator‡ • Strengths: • Easily calculated with existing RDS survey data • Weaknesses: • Statistical validity currently under debate • Not recommended as a an “only” method of estimation. • Results may be biased depending on number of people surveyed and actual population size • In RDS-based survey, respondents indicate their network size • Modelling is based on the assumption that those with large networks are sampled first and that the population will be depleted at a certain point † Respondent-driven sampling; ‡ Handcock, Gile, Mar (2012)

15. Network scale-up method† (NSUM) • Uses general population survey; questions about: • Number of people they know of a known population • Number of PWID they know • Strengths: • Does not ask sensitive questions directly to respondent • National level estimate • Weaknesses: • Average personal network size difficult to estimate • Some PWID may not interact much with members of the general population • Respondent may not be aware that someone in their network engages in injection drug use †Bernard, Killworth, Johnsen, and Robinson (1991)

16. Triangulation of multiple methods • Data points from multiple methods are desirable when possible • Reduces bias from any single method • May provide plausible lower and upper bounds • Informs stakeholder debate • Facilitates consensus on estimate ranges

17. PWID population size estimation (2011 IBBSS, Nairobi) Upper plausible 10,937 (~0.5% adults) Lower plausible 5,031 Source: Population Council, UCSF, NASCOP/Kenya, CDC (2011)

18. RDS SS-Size Added Upper plausible 10,937 (~0.5% adults) Lower plausible 5,031 Median 6,107 Source: Population Council, UCSF, NASCOP/Kenya, CDC (2011)

19. Scientific rigor and costs of methods Census Population-based survey Network scale up Multiple sample recapture Capture-recapture Unique object multiplier Truncated Poisson Multipliers, multiple multipliers RDS – Sequential Size Scientific rigor Unique event multiplier Mapping, key informants, observation counting Nomination counting Place, RAP, ethnography Registries, police, SHC, drug treatment, unions, workplace Wisdom of the crowd Delphi Consensus Discrepancies Soft modeling Borrow from thy neighbor Conventional Wisdom Straw man Cost Source: University of California, San Francisco

20. No resources or opportunity for data collection Census Population-based survey Network scale up Multiple sample recapture Capture-recapture Unique object multiplier Truncated Poisson Multipliers, multiple multipliers RDS – Sequential Size Scientific rigor Unique event multiplier Mapping, key informants, observation counting Nomination counting Place, RAP, ethnography Registries, police, SHC, drug treatment, unions, workplace Wisdom of the crowds Delphi Consensus Discrepancies Soft modeling Borrow from thy neighbor Conventional Wisdom Straw man Cost Source: University of California, San Francisco

21. Data collected directly from PWID for size estimation purposes only Census Population-based survey Network scale up Multiple sample recapture Capture-recapture Unique object multiplier Truncated Poisson Multipliers, multiple multipliers RDS – Sequential Size Scientific rigor Unique event multiplier Mapping, key informants, observation counting Nomination counting Place, RAP, ethnography Registries, police, SHC, drug treatment, unions, workplace Wisdom of the crowds Delphi Consensus Discrepancies Soft modeling Borrow from thy neighbor Conventional Wisdom Straw man Cost Source: University of California, San Francisco

22. Data collected from general population (DHS, AIDS Indicator Survey) Census Population-based survey Network scale up Multiple sample recapture Capture-recapture Unique object multiplier Truncated Poisson Multipliers, multiple multipliers RDS – Sequential Size Scientific rigor Unique event multiplier Mapping, key informants, observation counting Nomination counting Place, RAP, ethnography Registries, police, SHC, drug treatment, unions, workplace Wisdom of the crowds Delphi Consensus Discrepancies Soft modeling Borrow from thy neighbor Conventional Wisdom Straw man Cost Source: University of California, San Francisco

23. Data from PWID collected for other purposes (IBBSS, registries, service data) Census Population-based survey Network scale up Multiple sample recapture Capture-recapture Unique object multiplier Truncated Poisson Multipliers, multiple multipliers RDS – Sequential Size Scientific rigor Unique event multiplier Mapping, key informants, observation counting Nomination counting Place, RAP, ethnography Registries, police, SHC, drug treatment, unions, workplace Wisdom of the crowd Delphi Consensus Discrepancies Soft modeling Borrow from thy neighbor Conventional Wisdom Straw man Cost Source: University of California, San Francisco

24. IBBSS integration • Size estimation methods increasingly being integrated worldwide • Leverages existing resources • Adds value to behavioral and seroprevalence data already being collected • RDS increasingly used for IBBSS recruitment • “Population-based” estimates • Forthcoming RDS software-based estimation (SS-Size) • Provides an estimate using existing IBBSS RDS network data • Has limitations and caveats, should not be used as a sole estimate source

25. Other considerations • Ethical reviews • Administrative approvals • Safety of research assistants/study team • Involvement local drug using community representatives

26. Conclusion • Important to review how researchers arrived at the estimate since many are not grounded in quality data • Multiple PSE methods should be used • PWID size estimation should be a part of routine surveillance • Stakeholder consensus on estimate ranges critical

27. Acknowledgement • Scott Geibel (Population Council) • Henry Fisher Raymond (UCSF) • Abu Abdul-Quader (CDC) • PanduHarimurti (The World Bank) • Riku Lehtovuori (UNODC)

28. Group Discussion

29. Country A • Epidemic concentrated in PWID (account for ~90% of HIV transmission) • HIV prevalence in PWID: 15-30% (up to 87% in one city) • Civil unrest has hindered PSE of PWID • PSE (2,000) based on government registration of drug users† (0.05% of population nationally) • RDS-based IBBSS was conducted in 2013 in one city; no PSE • No IBBSS planned for near future † Registries based on treatment registers to arrest counts.

30. Country B • Epidemic concentrated in PWID, FSWs and their clients; PWID HIV prevalence: 7% • PSE available for 25 out of the country’s 28 states • National PSE (177,000; 0.02% of population) obtained through: • District-level mapping/enumeration at hotspots • Data updated regularly (by NGOs that implement targeted intervention at hotspots (WOTC with PWID and gatekeepers at hotspots) • IBBSS conducted every 2-3 years; currently being conducted but no PSE

31. Country C • HIV epidemic concentrated in PWID (account for ~60% of HIV transmission) • Prevalence of PWID: 100,000-200,000 (~0.9% of population) • Has extensive epi-behavioral data, including PSE; regulated by government • Latest published PSE available from 2010/11 • Methods and quality varies across region • National PSE obtained from summing regional results • Current/Upcoming activities: • 2014 RDS-based IBBSS with service multiplier (6 sites); 7 multipliers being used • Some regions will include cap-recap with independent databases • NSUM (2012/13)

32. Country D • Epidemic concentrated (PWID, MSM, FSW/clients) • PWID HIV prevalence: 18% • Prevalence of PWID • Range: 60,000-195,000 • Stakeholder consensus (2002): 75,000 • Estimate based on 0.5% of the population being PWID; this may be based on registration of drug users • IBBSS completed in May 2014; includes PSE using service and unique object multiplier • Conducted in 16 sites (14 are in high opium-growing states and two are major urban centers) • Injection drug use is believed to be occurring outside of these sites as well

33. Questions for Group Work • What are the potential problems/biases with the current estimate? • What kind of opportunity can you identify for improving the estimate? What are possible next steps?