Grey matters finding grey literature
1 / 46

Grey Matters Finding Grey Literature - PowerPoint PPT Presentation

  • Uploaded on

Grey Matters! Finding Grey Literature. Amanda Hodgson, MLIS Sarah Normandin, BSc, MLIS CE Course May 26, 2008 CHLA 2008. What is Grey Literature?. Grey

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Grey Matters Finding Grey Literature' - keona

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Grey matters finding grey literature

Grey Matters! Finding Grey Literature

Amanda Hodgson, MLIS

Sarah Normandin, BSc, MLIS

CE Course May 26, 2008

CHLA 2008

What is grey literature
What is Grey Literature?

  • Grey

  • Gray

  • In-house

  • Unpublished

  • Fugitive

  • Deep Web

What is grey literature1
What is Grey Literature?

International Conference on Grey Literature. "Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body." (Luxembourg, 1997 - Expanded in New York, 2004)

Document types in grey literature:

  • Source: GreyNet: grey literature network service. Amsterdam: GreyNet; 2008.

What is grey literature2
What is Grey Literature?

  • How are grey literature definitions changing…or going grey?

  • What is grey literature searching?

  • Remember, the search is greater than the definition!

Benefits of grey literature
Benefits of Grey Literature

  • Current

  • Often free

  • Relevant and unique

  • Information on non-mainstream topics or aspects

  • Aids in identifying terms for search strategies

Publication bias
Publication Bias

  • Necessary for certain types of research, like systematic reviews ---

  • “…. published trials tend to be larger and show an overall greater treatment effect than grey trials. This has important implications for reviewers who need to ensure they identify grey trials, in order to minimise the risk of introducing bias into their review.”

  • Source: Hopewell S, McDonald S, Clarke MJ, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev 2007;(2).

Benefits of searching for grey literature
Benefits of Searching for Grey literature

  • “Unpublished trials contribute about 20% of the weight in individual meta-analysis.”

  • Source: Krishnan RR. Evidence-based practice: how to read what you read. Psychopharmacology Bulletin 2004;37(4):88-91. [our emphasis]

Challenges of grey literature
Challenges of Grey Literature

  • Hard to find

  • Much content isn’t web-published

  • Rarely indexed, or minimally indexed

  • Basic bibliographic information may be unavailable

  • Older reports may not be archived

  • Volume of material can be overwhelming and time-consuming

Challenges of grey literature1
Challenges of Grey Literature

  • Evidence provided by grey literature may be criticized.

Starting your grey literature search
Starting your Grey Literature Search

  • Who is the recipient of the search results?

  • What is the resulting information product?

  • What are your search parameters, and in what ways must they be either flexible or more rigorously applied?

Cadth search approach
CADTH Search Approach

  • Scope definition (PICO) and search terms

  • 2. Comprehensiveliterature search

  • Database search for commercially published literature (e.g. Ovid databases, PubMed)

  • (b) “Grey Literature” component

    • Often started at Step 1 to unearth search terms

    • CADTH Grey Literature Checklists

B grey literature checklists
(b) Grey Literature Checklists

  • CADTH-standard lists

    • customized for each report type

  • PLUS included in the “grey literature” package:

    • May include both commercially published literature as well as grey literature

Cadth grey literature checklists
CADTH Grey Literature Checklists

  • New! Single compiled list available online:

  • Grey Matters: a practical tool for evidence-based searching


  • About CADTH > CADTH Products (scroll down)

Tips for saving time
Tips for Saving Time

  • Is general information or a comprehensive search required?

  • Scope out major systematic reviews (e.g. Cochrane reviews) and HTAs first

    • Search strategy and keyword ideas

  • Have a well-developed strategy in advance

    • Reduces backtracking

    • Use a checklist for tracking sources

  • Corollary: don’t spend too much time tracking!

    • Record minimal details to fully cite later

  • Exercises grey literature sources
    Exercises: Grey Literature Sources

    • Health Technology Assessment (HTA) agencies

    • Health Economics

    • Clinical Practice Guidelines

    • Drug/Devices Approval

    • Advisories/Warnings

    • Drug Class Reviews

    • Clinical Trials

    • Databases

    • Statistics

    • Open Access Journals

    • Additional Grey Literature Types and Sources

    Class exercises case studies
    Class Exercises: Case Studies

    • PICO Model: defining the research question

      Population (Patient, Population, Problem)




    Pico example
    PICO Example

    • Question: Conduct a grey literature search on combination long-acting beta-2 agonists (LABA) plus corticosteroids inhalers for asthma, compared to separate medication treatments.

    • Asthma patients

    • Long-acting beta-2 agonists (LABA) plus inhaled corticosteroids (ICS) administered by inhaler

    • LABA alone

    • ICS alone

    • other asthma medications/inhalers

    • Better control of asthma symptoms; reduced airflow obstruction





    Health technology assessments htas
    Health Technology Assessments (HTAs)

    • Highest quality evidence comes from:

    • Systematic reviews (SR)

    • Meta-analyses (MA)

    • Health technology assessments (HTA)

    Systematic review
    Systematic Review

    • Review of studies in which evidence has been systematically searched for, studied, assessed, and summarised according to predetermined criteria.

    Definitions based on NLM, Cochrane Glossary, INAHTA, CADTH definitions.

    Meta analysis

    • Use of statistical techniques for combining results from different studies to obtain a quantitative estimate of the overall effect of a particular intervention, potentially producing a stronger conclusion than any individual study

    Health technology assessment
    Health Technology Assessment

    • Multidisciplinary and systematic policy analysis, that studies the clinical, social, ethical, and economic implications of a drug, medical system, or health technology.

    • Primary purpose is to inform health policy and clinical practice decision makers.

    Hta highlights canada
    HTA Highlights - Canada

    • CADTH

    • Canadian Agency for Drugs and Technologies in Health


    • AETMIS

    • Agence d’évaluation des technologies et des modes d’intervention en santé


    Hta highlights international
    HTA Highlights - International

    • HTAi Vortal

    • Health Technology Assessment International


    • NCCHTA

    • National Coordinating Centre for Health Technology Assessment


    Health economics
    Health Economics

    • Focus is on choices, value judgements with regard to health care decision making

    • Direct/indirect costs

    • Statistics, incidence/prevalence

    • Socioeconomic

    • Quality of life

    Health economics1
    Health Economics

    • Economic evaluations

    • Cost-minimization analyses

    • Cost-effectiveness analyses

    • Cost-utility analyses

    • Cost-benefit analyses

    • At CADTH, we may be conducting searches for economic evaluations.

    Health economics2
    Health Economics

    • Highlights

    • Centre for Health Economics and Policy Analysis (CHEPA)


    • EURONHEED(European Network of Health Economic Evaluation Databases)


    Clinical practice guidelines
    Clinical Practice Guidelines

    • A search for clinical practice guidelines may extend to association websites or organizations dedicated to a particular disease or patient group, depending on the requirements of the search.

    • Guidelines

    • Protocols

    • Positions statements

    • Patient information sheets and resources

    Clinical practice guidelines1
    Clinical Practice Guidelines

    • Highlights

    • Guidelines & Protocols Advisory Committee, British Columbia Ministry of Health


    • National Guideline Clearinghouse


    Drug and devices approvals
    Drug and Devices Approvals

    • Regulatory approvals must be granted by government bodies for drugs and health devices. Information on the expansion of patient groups for previously approved drugs or withdrawals of medicines/devices from the market are often available.

    Drug and devices approvals1
    Drug and Devices Approvals

    • Highlights

    • Medical Devices Active License Listing (Canada)


    • Center for Drug Evaluation and Research (CDER)


    Advisories warnings

    • Medical advisories and warnings are reported by governments internationally.

    • Adverse effect/events

    • Safety concerns (devices and medical procedures)

    • Withdrawals from the market

    Advisories warnings1

    • Highlights

    • Health Canada

    • MedEffect: Canadian Adverse Reaction Newsletter, Health Canada


    • European Agency for the Evaluation of Medicinal Products (EMEA)

    • Product safety announcements


    Drug class reviews
    Drug Class Reviews

    • Highlights

    • DERP: The purpose of DERP reports is to make available information regarding the comparative effectiveness and safety profile of different drugs within pharmaceutical classes.

    • Drug Effectiveness Review Project (DERP), Oregon Health & Science University


    Clinical trials
    Clinical Trials

    • Valuable information on the safety and efficacy of new drugs and therapies as well as new uses for known treatments

    • Information on studies that may not be published yet

    • Tip: Check the product manufacturer’s website for trial lists

    • (U.S. National Institutes of Health)


    • Current Controlled Trials



    • Retrieve grey literature in addition to traditional commercially published literature

    • NHS CRD

    • Centre for Reviews & Dissemination, University of York, UK


    • Cochrane Library (free full text in NB, NS, NU, SK, YK, NWT)

    • TRIP Database


    • Statistics on incidence and prevalence of specific conditions, population information

    • Consult association websites for statistics on specific diseases not covered on major statistical websites

    • Statistics Canada

    • Public Health Agency of Canada (PHAC)


    Open access journals
    Open Access Journals

    • Journals where all content is freely available, no embargo period

    • Often peer reviewed

    • PubMed Central

    • Directory of Open Access Journals

    Additional grey literature sources
    Additional Grey Literature Sources

    • Online catalogues

    • Amicus, NLM, British Library (BL)

    • Company Directories and Patents

    • Industry Canada databases:

      Patents database:

    • Blogs/Discussion lists

    • CanMedLib

    • Medical writers

    • Patient advocates

    Conferences congresses symposia

    • Manufacturer/Association websites

    • Input from experts/researchers

    • Online conference planners, conference reporting




    • Database records

    • ex. Biosis

    • Internet search

    • (disease OR drug) AND (meeting* OR abstract* OR poster* OR conference* OR proceeding* OR poster*)

    Managing and presenting grey literature findings
    Managing and Presenting Grey Literature Findings

    • Presenting results

    • Difficult citations

    • Citing Medicine: the NLM style guide for authors, editors, and publishers. 2nd edition. Bethesda (MD): U.S. National Library of Medicine; 2007. Available:

    • Citation management

    • ProCite, Reference Manager, EndNote, etc.

    • Keeping grey literature results separate vs. integrating with database search results

    Alerts and awareness tools
    Alerts and Awareness Tools

    • Limit searches by date range

    • Google Advanced – last 24h, week, month(s), year

    • E-mail alerts & RSS feeds

    • Set up alerts in databases such as PubMed (MyNCBI), Cochrane, CRD

    • RSS feeds: PubMed, MedScape,, journals

    Alerts awareness tools
    Alerts & Awareness Tools

    • Webpage change detection services



    Closing remarks
    Closing Remarks

    • Tips for your grey literature search!

      Have a well-developed strategy before you start

      Re-negotiate PICO & parameters as you search

      Keep it simple: paste title and link into a “good copy” document as you proceed

      Grey literature searching is key for finding certain types of information and reducing publication bias

    References of interest canadian
    References of Interest - Canadian

    • Grey matters: a practical tool for evidence-based searching. Ottawa: Canadian Agency for Drugs and Technologies in Health;2008.

    • Giustini D, Thompson D. “Finding the Hard to Finds”: Annotated bibliography (Appendix E). Vancouver: University of British Columbia; 2006.

    • IHE report: health technology assessment on the net: a guide to Internet sources of information. Edmonton: Institute of Health Economics; 2007.

    References of interest international
    References of Interest - International

    • GreyNet (Grey Literature Network Service).

    • New York Academy of Medicine. Grey Literature Report.

    • Helmer D. Chapter 10: Grey literature. In: Auston I, Topfer L, eds. Etext on health technology assessment (HTA) information resources. Bethesda (MD): National Information Center of Health Services Research and Health Care Technology (NICHSR); 2006.