1 / 47

Assessing Applicability of Medical Test Studies in Systematic Reviews

Assessing Applicability of Medical Test Studies in Systematic Reviews. Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Medical Test Reviews Methods Guide www.ahrq.gov. Overview of a Medical Test Review. Analyze and Synthesize Studies

jarrod-bennett
Download Presentation

Assessing Applicability of Medical Test Studies in Systematic Reviews

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Applicability of Medical Test Studies in Systematic Reviews Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Medical Test Reviews Methods Guide www.ahrq.gov

  2. Overview of a Medical Test Review • Analyze and Synthesize Studies • Assess Risk of Bias as a Domain of Quality • Assess Applicability • Grade the Body of Evidence • Meta-analysis of Test Performance Evidence With a “Gold Standard” — or — • Meta-analysis of Test Performance Evidence With an Imperfect Reference Standard • Decision Modeling Extract Data From Studies • Prepare Topic • Develop the Topic and Structure the Review • Choose the Important Outcomes • Search for and Select Studies for Inclusion • Search for Studies Research Sources Report Medical Test Review Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  3. Learning Objectives • Explain how applicability extends to both the individual study and bodies of evidence • Describe three main challenges related to assessing applicability and how they are often related to context • Understand the four principles for assessing and reporting applicability • Apply the PICOTS (population, intervention, comparator, outcomes, timing, and setting) typology to assess applicability Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  4. Background • Decisionmakers need to understand whether a medical test is meaningful in a specific context in order to make an informed decision. For example: • Whether the test has been studied in the patient/disease/care settings of interest • Whether the test has been part of a management strategy of interest • Evidence-based Practice Centers use four principles to assess and report applicability: • Determine the most important factors that affect applicability • Systematically abstract key characteristics that may affect applicability • Make judgments about major limitationsto the applicability of individual studies • Consider the applicability of the body of evidence Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  5. Challenges Specific to Assessing Applicability(1 of 4) • Key questions lack clarity. • Studies are not specific to the key questions. • Tests are rapidly evolving. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  6. Challenges Specific to Assessing Applicability(2 of 4) • Key questions lack clarity • How the test is to be used should be stated clearly in the key questions to avoid confusion among decisionmakers about the management strategy being evaluated. • Example: Studies comparing the accuracy of one test versus another (i.e., as a replacement), but the new test is actually meant to be a triage/add-on test • Hinders the review process (slows inclusion/exclusion decisions) • Leads to inappropriate “lumping” and “splitting” of groups • Lumping subgroups that are fundamentally different can cause a false finding of a difference (type 1 error). • Splitting subjects into subgroups without evidence of different effects may reduce power to detect a difference (type II error). Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  7. Challenges Specific to Assessing Applicability(3 of 4) • Studies are not specific to the key questions • Studies may not reflect the subgroups or comparisons identified in the key questions. • Reviewers need to decide whether the deviation is minor or crucial. • Crucial deviations are those deemed likely to meaningfully affect: • Test performance • Clinical decisionmaking • Health outcomes Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  8. Challenges Specific to Assessing Applicability(4 of 4) • Tests are rapidly evolving and change often. • The literature often references tests that are not yet widely available or no longer common. • Secular trends in use patterns and market forces may shape applicability in unanticipated ways. • Example: An old test that gives dichotomous, qualitative results (present vs. absent) is replaced by a test that reports a continuous, quantitative measure. • Review the old test as a point of reference for determining the merit of a new test? • Only use the more limited, newer data from the new test? Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  9. Principles of Assessing Applicability • Identify important contextual factors • Be prepared to deal with additional factors affecting applicability • Justify decisions to restrict the scope of a review • Maintain a transparent process Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  10. Principle 1: Identify Important Contextual Factors (1 of 10) • It is not practical to consider all possible factors related to the impact of test on health outcomes • Consider factors that affect steps in a causal chain of direct relevance to the key question. • Example: Slice thickness is a relevant factor for assessing the applicability/accuracy of cardiac magnetic resonance imaging for detecting atherosclerosis. • Also consider factors that affect later links in the causal chain. • Example: What factors impact effectiveness of treatment for lesions identified by cardiac MRI versus angiography? • Consider contextual issues relevant to medical tests, for example, patient populations, management strategies, time effects, and secular trends in disease prevalence and severity. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  11. Principle 1: Identify Important Contextual Factors (2 of 10) • Spectrum effect • The severity or type of disease may affect test accuracy. • Example: Cardiac magnetic resonance imaging is generally accurate in identifying cardiac anatomy/functionality, but arrhythmias, lesion location, and obesity can affect performance of the test. • Reviewers must identify these disease-related factors ahead of time and justify when to split key questions or to conduct subgroup analyses. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  12. Principle 1: Identify Important Contextual Factors (3 of 10) • The medical test as part of a management strategy • It is important to clarify conditions in which a test is studied and used. • Example: The test performance of cardiac magnetic resonance imaging (MRI) differs when used as an add-on test after initial screening versus when used for triage or for patients without symptoms. • When cardiac MRI is used as add-on, the comparator for diagnostic accuracy is x-ray coronary angiography (the gold standard). • When cardiac MRI is used for triage, the comparator is stress electrocardiography or other initial screening test. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  13. Principle 1: Identify Important Contextual Factors (4 of 10) • Methods of the medical test over time • Diagnostics evolve rapidly. • Awareness of these changes should guide selection of the date range and eligible test types for included literature, • Example 1: The slice thickness of magnetic resonance imaging scans has decreased steadily over time, allowing resolution of smaller lesions. • Excluding studies with older technologies and presenting results of included studies by slice thickness may be appropriate. • Example 2: Antenatal tests are applied earlier and earlier in gestation. • Studies of test performance need to be examined by varied cutoffs for stage of gestation. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  14. Principle 1: Identify Important Contextual Factors (5 of 10) • Secular trends in population risk and disease prevalence • Direct/indirect changes in secular trends (or cultural differences) can influence test performance and applicability. • Example: Obesity and the average age of pregnancy have risen dramatically in the past 30 years, both of which are linked to type 2 diabetes. • The underlying prevalence of undiagnosed type 2 diabetes in pregnancy is increased. • As a result, the predictive values, the cost/benefit ratios, and the sensitivity and specificity of screening tests for gestational diabetes change over time. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  15. Principle 1: Identify Important Contextual Factors (6 of 10) • Secular trends in population risk and disease prevalence • Secular trends in population characteristics can influence the ability of clinicians to conduct a medical test. • Example: Obesity diminishes image quality in tests, such as ultrasound for gallbladder disease or magnetic resonance imaging for spinal conditions. • Studies of these tests often restrict enrollment to people with normal body weight. • These tests may not perform the same in practice because of the rising trend of obesity in the actual patient population. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  16. Principle 1: Identify Important Contextual Factors (7 of 10) • Secular trends in population risk and disease prevalence • Prevalence is tied to disease definitions that can change. Some examples are: • Criteria to diagnose AIDS • Cystometrically defined detrusor instability/overactivity versus the symptom complex “overactive bladder” • Continuous refinement of classifications of mental health conditions in the Diagnostic and Statistical Manual of Mental Disorders • The literature may not capture changes in diagnostic criteria. • Having expert knowledge with a historical vantage point is helpful. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  17. Principle 1: Identify Important Contextual Factors (8 of 10) • Routine preventive care over time • Routine use of a medical test as a screening test may indirectly alter population prevalence. • Preventive lipid testing in individuals with cardiovascular disease has changed the proportion able to be diagnosed and have the course of their disease altered. • The vaccine for the human papilloma virus (HPV) is postulated to change the distribution of viral subtypes in the population. • Preventive practices change the utility of medical tests. • Example: The utility of the test to detect HPV infection is decreasing as increasing numbers of people are being vaccinated for the virus. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  18. Principle 1: Identify Important Contextual Factors (9 of 10) • Treatment trends • Literature about the impact of diagnostic tools on outcomes needs additional interpretation in light of new therapeutics. • Example: The implications of testing for carotid arterial stenosis are changing as hypertension treatment and the use of lipid-lowering drugs have improved. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  19. Principle 1: Identify Important Contextual Factors (10 of 10) • Two steps to ensure that population/subgroup data are uniformly collected and useful: • Use the PICOTS (population, intervention, comparator, outcomes, timing, and setting) typology to identify the range of possible factors that might affect applicability and consider the six contextual factors. • Review the list of factors affecting applicability with stakeholders to: • Ensure common vantage points • Identify hidden factors that may influence applicability • Capture crucial applicability features in the evidence review Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  20. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (1 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  21. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (2 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  22. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (3 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  23. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (4 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  24. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (5 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  25. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (6 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  26. Using the PICOTS Framework To Assess and Describe Applicability of Medical Tests (7 of 7) Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  27. Principle 2: Be Prepared To Deal With Additional Factors Affecting Applicability • Some contextual factors are only found after a large volume of literature has been reviewed. • Example: A meta-analysis may find that a test is inaccurate in older patients, but age was not considered in the key questions. • May be a false association • May be an oversight • When in doubt between a false association or oversight, it is appropriate to comment on an apparent association and clearly state that it is only a hypothesis. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  28. Principle 3: Justify Decisions To Restrict the Scope of a Review • May be necessary to restrict the scope of the review to: • Specific versions of the test • Selected study methods or types • Specific populations (e.g., women, patients who are obese) • Specific settings (e.g., primary care practice) • Restriction is appropriate when: • All agree that a top priority is applicability to a particular target group or setting • Test performance is different in a specific subgroup, setting, or from another version of the test • Restriction is not appropriate when a fully comprehensive review is desired Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  29. Principle 4: Maintain a Transparent Process • Address applicability when defining review methods and documenting decisions in a protocol. • Example: Factors that change with time should prompt adding timeframes to inclusion criteria or description/ analysis of possible impacts on applicability. • Transparency is essential when a review decision may be controversial. • Example: Exclusion criteria leave a reviewer “empty handed;” the reviewer may decide some excluded evidence is relevant by extension or analogy. • In this event, clearly state how evidence may not be directly applicable to key questions but represents the limited state of the science. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  30. Illustrative Example: Cervical Cancer Screening (1 of 3) • 2002 Cervical Cancer Screening Summary of the Evidence for the U.S. Preventive Services Task Force • Challenges: • Changes in secular trends and availability/format of tests • Liquid-based cervical cytology (new technique) rapidly being used in practice • Resource shortages for reviewing conventional Pap smears • New computer systems to screen cytology slides to enhance accuracy • Literature starting to include adjunct usage of human papilloma virus vaccine • No randomized controlled trials available Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm. Hartmann KE, Hall SA, Nanda K, et al. Systematic Evidence Review No. 25. Available at www.ncbi.nlm.nih.gov/books/NBK42831.

  31. Illustrative Example: Cervical Cancer Screening (2 of 3) • Since no data compared new screening tools/strategies to outcomes, the 2002 Cervical Cancer Screening Summary of the Evidence for the U.S. Preventive Services Task Force report focused on medical test characteristics (sensitivity, specificity, predictive value, and likelihood ratios). • Three computer technologies • Two liquid cytology approaches • All methods of human papilloma virus testing • Included all technologies • Restricting technologies only to those available in the United States (most applicable setting) would have reduced the scope substantially. • The purpose was to determine if comparable or superior methods were being overlooked or no longer offered. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm. Hartmann KE, Hall SA, Nanda K, et al. Systematic Evidence Review No. 25. Available at www.ncbi.nlm.nih.gov/books/NBK42831.

  32. Illustrative Example: Cervical Cancer Screening (3 of 3) • There are two specific areas of applicability interest — age and hysterectomy status. • The underlying incidence and prevalence data included: • Provided context • Informed modeling of testing impact • Reviewers could not estimate the usefulness of human papilloma virus testing among older women from trends in younger women. • Extrapolating results from one context to another rarely enhances applicability. • The review found that older women are disproportionately more likely to have false-positive findings under current screening recommendations and supported the discontinuation of Pap testing after hysterectomy. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm. Hartmann KE, Hall SA, Nanda K, et al. Systematic Evidence Review No. 25. Available at www.ncbi.nlm.nih.gov/books/NBK42831.

  33. Summary (1 of 3) • To adequately assess applicability of a medical test, reviewers need to familiarize themselves with: • Availability of the test • Technology of the test • Contemporary clinical use of the test • Current treatment modalities for the related disease condition • Potential interplay of disease severity and the performance characteristics of the test • Implications of particular study designs and sampling strategies for bias in the findings about applicability Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  34. Summary (2 of 3) • Restricting inclusion of only marginally related or outdated studies can aid applicability. • Extrapolating results from one context to another does not help. • Guidance from clinical, medical testing, and statistical experts is helpful when considering restriction of scope or exclusion of publications with specific types of flaws. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  35. Summary (3 of 3) • The review team must group findings in a meaningful way, whether the review is exhaustive or narrow. • For medical tests, this means gathering and synthesizing data in ways that help to readily understand applicability. • Grouping summaries of findings using familiar structures like the PICOTS (population, intervention, comparator, outcomes, timing, and setting) enhances clarity in framing applicability. • Studies may be presented more than once within a review, in different “applicability axes.” Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  36. Key Messages • Identify contextual factors that may affect test performance early in the review process. • Carefully consider and document justification for addressing these factors by restricting key questions or creating a broader review. • Protocols should document which factors will be excluded and how subgroups will be assessed. • Reviewers should document how they plan to handle studies that only partly fit the key questions or inclusion/exclusion criteria. • Final reviews should include a description of test use in usual practice and how the included studies fit with this use. Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Methods guide for medical test reviews. Available at www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  37. Practice Question 1 (1 of 2) • When conducting most systematic reviews, it is important to limit the contextual factors considered in applicability assessment rather than considering all possible factors. • True • False

  38. Practice Question 1 (2 of 2) Explanation for Question 1: The statement is true. It is not practical to consider all possible factors that may impact applicability in all cases. This approach would be overwhelming and, in some cases, leave the reviewer with no applicable studies. A reasonable list of factors should be considered and agreed upon by the stakeholders in the systematic review before initiating the literature review.

  39. Practice Question 2 (1 of 2) • The routine use of the human papilloma virus vaccine to prevent cervical cancer may change the utility of the test to detect the virus. • True • False

  40. Practice Question 2 (2 of 2) Explanation for Question 2: The statement is true. This is an example of how the changes in routine preventive care over time can affect the prevalence of disease. Knowledge of these trends is important when looking at the applicability of a body of literature.

  41. Practice Question 3 (1 of 2) • One of the general principles in assessing the applicability is to keep the process as confidential as possible without involving the stakeholders and future decisionmakers. • True • False

  42. Practice Question 3 (2 of 2) Explanation for Question 3: The statement is false. Transparency is essential while addressing applicability. Review methods should be documented clearly in a protocol.

  43. Practice Question 4 (1 of 2) • The PICOTS (population, intervention, comparator, outcomes, timing, and setting) framework is a tool that can help to systematically abstract and report key characteristics that may affect applicability. • True • False

  44. Practice Question 4 (2 of 2) Explanation for Question 4: The statement is true. The PICOTS (population, intervention, comparator, outcomes, timing, and setting) framework is a tool that can be used in a systematic review to meticulously think through all of the factors that may affect applicability.

  45. Authors • This presentation was prepared by Brooke Heidenfelder, Rachael Posey, Lorraine Sease, Remy Coeytaux, Gillian Sanders, and Alex Vaz, members of the Duke University Evidence-based Practice Center. • The module is based on chapter 6,Assessing applicability of medical test studies in systematic reviews. In: Chang SM and Matchar DB, eds. Methods guide for medical test reviews. Rockville, MD: Agency for Healthcare Research and Quality; June 2012. p. 6.1-13. AHRQ Publication No. 12-EHC017. www.effectivehealthcare.ahrq.gov/medtestsguide.cfm

  46. References (1 of 2) • Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Review. Rockville, MD: Agency for Healthcare Research and Quality; April 2012. AHRQ Publication No. 10(12)-EHC063-EF. Chapters available at www.effectivehealthcare.ahrq.gov/methodsguide.cfm. • Hartmann KE, Hall SA, Nanda K, et al. Screening for Cervical Cancer. Systematic Evidence Review No. 25 (Prepared by the Research Triangle Institute/University of North Carolina under Contract No. 290-97-0011, Task Order No. 3). Rockville, MD: Agency for Health Care Research and Quality; January 2002. www.ncbi.nlm.nih.gov/books/NBK42831. • Hartman KE, Matchar DB, Chang SM. Assessing applicability of medical test studies in systematic reviews. In: Chang SM and Matchar DB, eds. Methods guide for medical test reviews. Rockville, MD: Agency for Healthcare Research and Quality; June 2012. p. 6.1-13. AHRQ Publication No. 12-EHC017. www.effectivehealthcare.ahrq.gov/medtestsguide.cfm.

  47. References (2 of 2) • Hartmann KE, Nanda K, Hall S, et al. Technologic advances for evaluation of cervical cytology: is newer better? Obstet Gynecol Surv. 2001 Dec;56(12):765-74. PMID: 11753179. • Hutchinson ML, Zahniser DJ, Sherman ME, et al. Utility of liquid-based cytology for cervical carcinoma screening: results of a population-based study conducted in a region of Costa Rica with a high incidence of cervical carcinoma. Cancer. 1999 Apr;87(2):48-55. PMID: 10227593. • McCrory DC, Matchar DB, Bastian L, et al. Evaluation of Cervical Cytology. Evidence Report/Technology Assessment No. 5 (Prepared by the Duke University Evidence-based Practice Center under Contract No. 290-07-0014. Rockville, MD: Agency for Health Care Policy and Research; January 1999. AHCPR Publication No. 99-E010. www.ncbi.nlm.nih.govbooks/NBK11840.

More Related