introduction to research methods n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Introduction to Research Methods PowerPoint Presentation
Download Presentation
Introduction to Research Methods

Loading in 2 Seconds...

play fullscreen
1 / 44

Introduction to Research Methods - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Introduction to Research Methods . Research Methods for Public Administrators Dr. Gail Johnson. Numbers and Facts. Any policy debate is awash in numbers that are presented as absolute fact. But are they? Or are we being tricked into “premature certainty” because numbers appear concrete?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Introduction to Research Methods' - tamyra


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
introduction to research methods

Introduction to Research Methods

Research Methods for Public Administrators

Dr. Gail Johnson

Dr. G. Johnson, www.researchdemystified.org

numbers and facts
Numbers and Facts

Any policy debate is awash in numbers that are presented as absolute fact.

But are they?

Or are we being tricked into “premature certainty” because numbers appear concrete?

Dr. G. Johnson, www.researchdemystified.org

in the news
In the News
  • According to a 2009 study done for the American Health Insurance Plans by Price Waterhouse and Coopers:
  • The average family health insurance coverage costs approximately $12,300 today
  • Key finding: The overall impact of the proposals for health care reform will be to increase the cost of private insurance coverage for individuals, families, and businesses above what these costs would be in the absence of reform.

Dr. G. Johnson, www.researchdemystified.org

their numbers
Their Numbers
  • Health Insurance premiums could be expected to increase to approximately:
    • $15,500 in 2013 under current law and to $17,200 if these provisions are implemented.
    • $18,400 in 2016 under current law and to $21,300 if these provisions are implemented.
    • $21,900 in 2019 under current law and to $25,900 if these provisions are implemented.

Dr. G. Johnson, www.researchdemystified.org

policy debate
Policy Debate
  • What does this data mean in terms of Congressional proposals to require everyone purchase health insurance?
  • What would you conclude based on this research result?
  • What do you want Congress to do based on these numbers?
  • What do you want Congress to do based on your beliefs?

Dr. G. Johnson, www.researchdemystified.org

paradox
Paradox
  • Research: the search for rationality

But:

“We must be aware that our knowledge and experiences are finite, and always imperfect.”

Hitoshi Kume

Dr. G. Johnson, www.researchdemystified.org

assessing credibility
Assessing Credibility

Requires

  • Basic knowledge of research methods
    • Maintaining a research perspective
    • Knowing how data was collected is necessary to help assess whether the results are believable
  • Critical thinking skills
    • A hint of detachment and skepticism
    • Being aware of our beliefs yet setting them aside to see things clearly

Dr. G. Johnson, www.researchdemystified.org

the tough questions
The Tough Questions
  • Is it true?
  • How do you know?
  • What is the logic of the argument?
  • What is the evidence?

Remember:

Much of what is presented as fact melts away under scrutiny.

Beware of premature certainty.

Dr. G. Johnson, www.researchdemystified.org

critical thinking
Critical Thinking
  • Ask Yourself:
    • “How do I know that the things I believe are true?”
    • “What would it take to convince me otherwise?”
  • The greatest challenge comes when looking at research that reflects what we already believe
  • Critical thinking challenges our most cherished beliefs
  • To see clearly, we must be aware of our beliefs while also remaining detached from them

Dr. G. Johnson, www.researchdemystified.org

the role of research in the public sector
The Role of Research in the Public Sector
  • Guide policy
  • Target programs
  • Find out what works and what doesn’t
  • Convince funders
  • Provide feedback from customers
  • Help plan interventions
  • Advocate for change

Dr. G. Johnson, www.researchdemystified.org

research in the public sector
Research in the Public Sector
  • The Intersection of Data, Values and Beliefs

and Politics

    • Does welfare help or hurt people?
    • Should oil companies be prevented from earning too much profit?
    • Is investment in primary prevention more effective in reducing crime than spending on law enforcement?

Dr. G. Johnson, www.researchdemystified.org

different research approaches
Different Research Approaches
  • Qualitative:
    • Stories
    • Observations
    • No numbers
  • Quantitative:
    • What you can count.

Dr. G. Johnson, www.researchdemystified.org

qualitative data stories
Qualitative Data: Stories
  • Anecdotes
      • Example: Michael Moore’s “Sicko” Example: Barbara Erhen “Nickled and Dimed”
  • Paints a picture, makes it feel real
  • Limited in scope:
    • Is this a small or large problem?

Dr. G. Johnson, www.researchdemystified.org

quantitative data numbers
Quantitative Data: Numbers
  • Percents, Counts, Correlations
    • Example: Study: U.S. Ranking for Preventable Deaths:
      • U.S. lowest of industrialized countries
      • But need to check how they defined and measured preventable deaths

Dr. G. Johnson, www.researchdemystified.org

quantitative data numbers1
Quantitative Data: Numbers
  • Be aware that some things are not really knowable—so how did they come up with the numbers?
    • Number of illegal immigrants
    • Amount of money spent on illegal drugs
    • 10-year projection of annual U.S. budget deficits (even 1-year is not precise)
    • The true cost of mandatory health insurance
    • Percent of carbon from raising animals

Dr. G. Johnson, www.researchdemystified.org

as an advocate
As An Advocate

Which would be more effective?

Voice your concerns about a problem?

Or

Show data documenting the problem?

Dr. G. Johnson, www.researchdemystified.org

as a citizen
As a Citizen

What do you base decisions on?

  • Data?
  • Emotional appeals?
  • Personal opinion?
  • Political demands?

Do some work better than others in different situations?

Dr. G. Johnson, www.researchdemystified.org

as a decision maker
As a Decision Maker

What do you base decisions on?

  • Data?
  • Emotional appeals?
  • Personal opinion?
  • Political demands?

Do some work better than others in different situations?

Dr. G. Johnson, www.researchdemystified.org

the tough question
The Tough Question
  • Generally, we prefer data, facts, hard evidence
  • But the question is:
    • How good is the research?
    • Meaning: does the research yield accurate, reliable, valid, and unbiased information?

Dr. G. Johnson, www.researchdemystified.org

what is research
What is Research?
  • A systematic search for answers to questions.
  • Search: to uncover, examine, find by exploration, to investigate, to inquire.
  • Research: "the systematic inquiry into a subject in order to discover or revise facts, theories

Dr. G. Johnson, www.researchdemystified.org

what is research1
What is Research?

Research: Root of work means “to know.”

Re---Search:

conveys the idea of searching again, from a different perspective, using different approaches, looking at different data.

Dr. G. Johnson, www.researchdemystified.org

what is research2
What is Research?
  • Empirical: derived from experience or experiment; observation and experience.
  • Science: "a branch of knowledge or study dealing with a body of facts or truths systematically arranged and showing the operation of general laws; systematic knowledge"

Dr. G. Johnson, www.researchdemystified.org

this thing called science
This Thing Called Science
  • Scientific Method
    • Curiosity
    • Systematic observation
    • Systematic experimentation
  • Theories and Hypotheses:
    • Used to Explain Relationships

Dr. G. Johnson, www.researchdemystified.org

different goals
Different Goals

Scientists:

    • More interested in creating knowledge
    • Not concerned about utility or application of the results
  • Practitioners:
    • More interested in application and problem solving
    • If the research isn’t useful, then so what?

Dr. G. Johnson, www.researchdemystified.org

research process simplified 3 step
Research Process: Simplified 3-step
  • Plan
  • Do
  • Report

Dr. G. Johnson, www.researchdemystified.org

research planning process
Research Planning Process
  • Determine the issues
  • Decide on the research question(s)
  • Select measures
  • Identify the ideal design given the type of research question
  • Develop data collection methods and instruments

Dr. G. Johnson, www.researchdemystified.org

research planning process1
Research Planning Process
  • Decide on sampling frame and strategy
  • Develop analysis plan
  • Articulate your entire research plan
  • Review research plan and test all data collection instruments
  • Prepare work plan with resource and time requirements

Dr. G. Johnson, www.researchdemystified.org

planning matters
Planning Matters
  • When people think about research, they tend to immediately focus on statistics
  • But no amount of statistical wizardry can save mistakes made in the planning process
  • Planning is harder and takes longer than most people expect

Dr. G. Johnson, www.researchdemystified.org

doing research
Doing Research
  • Gathering the data
  • Preparing data for analysis
  • Analyzing and interpreting the data

Dr. G. Johnson, www.researchdemystified.org

reporting
Reporting
  • Executive Summary
  • Reports
  • Charts and Tables
  • Oral Briefings

Dr. G. Johnson, www.researchdemystified.org

research planning process2
Research Planning Process
  • Presented as linear but in practice, it is not
  • The word is iterative: meaning researchers make some initial assumptions about how to proceed and then go back and make changes to the plan as new information shows that their initial plans will not work
  • There is a lot of back and forth before it all clicks into place (and researchers live with the limitations of the situation)

Dr. G. Johnson, www.researchdemystified.org

ethical research
Ethical Research
  • Do No Harm
    • Protect participants from being harmed by the research
    • Guard the confidentiality of participants
    • Do not coerce people to participants: they must freely consent
    • No one should be denied benefits to which they are entitled because they refuse to participate in research project
    • Do not quickly conclude a program does not work just because there does not appear to be an impact

Dr. G. Johnson, www.researchdemystified.org

objective strive to see clearly
Objective: Strive to See Clearly
  • Objectivity is a worthy goal even if not 100% attainable
    • Think in terms of being free of ideological or political blinders or desired outcomes
    • Be aware of biases
      • Researchers should state biases and build in quality controls to minimize bias
      • Quality Control: Have others with different perspectives review the research design, analysis and results.

Dr. G. Johnson, www.researchdemystified.org

principles of good research
Principles of Good Research
  • Honest
    • Do not lie
    • Do not distort or “spin” the data
    • Fully disclose methods, definitions, assumptions, biases
    • Fully disclose limitations of the research and implications for making any conclusions

Dr. G. Johnson, www.researchdemystified.org

principles of good research1
Principles of Good Research
  • Accurate: Strive to be error free
    • Build quality control procedures into data collection, analysis, interpretation, and written product
    • Verify data entry and analysis.
    • Have someone check your numbers.

Dr. G. Johnson, www.researchdemystified.org

principles of good research2
Principles of Good Research
  • Technically Correct
    • Use appropriate designs, data collection methods, analysis, statistics and charts
    • The right data collection instruments.
    • The right statistics.

Dr. G. Johnson, www.researchdemystified.org

principles of good research3
Principles of Good Research
  • Reliable and Valid Measures
    • Use reliable and valid measures
    • Measure what matters using systematic approaches with fixed measurement rules
    • Ask: do they measures actually measure what they claim to measure?
      • Are they measuring reported crime but claiming to be measuring all crime?

Dr. G. Johnson, www.researchdemystified.org

research flaws
Research Flaws
  • Perfection is not the standard to assess the credibility of research results.
    • All research is flawed, so the trick is to distinguish minor flaws from major ones.
  • Skill: to assess the relative strengths and limitations of research to determine the credibility of the results.

Dr. G. Johnson, www.researchdemystified.org

research flaws1
Research Flaws
  • It means working in the gray area.
  • If the research is the best given the circumstances, flawed though it may be, it should be considered.
  • However, decisions “to do something” based on that data should be made with caution, and implemented incrementally with built-in feedback.

Dr. G. Johnson, www.researchdemystified.org

seriously flawed research
Seriously Flawed Research
  • While you might not want to make decisions based on seriously flawed research, it still may actually be correct, provide useful insights, or some guidance about how to do the next study.
    • It might provoke important conversations.
  • Remember: perfection is not a useful standard.

Dr. G. Johnson, www.researchdemystified.org

takeaway lesson
Takeaway Lesson

Sophisticated Users of Research Question Numbers!

“Many a statistic is false on its face. It gets by only because the magic of numbers brings out a suspension of common sense."

---How to Lie With Statistics, Huff, p. 138.

Dr. G. Johnson, www.researchdemystified.org

ask tough questions
Ask Tough Questions
  • Does the researcher have axe to grind?
    • Particular political ideology? Desire to get more funding?
  • Who was included and excluded in the study?
  • Is the data really knowable?
    • Number of actual crimes is not the same as the number of reported crimes.

Dr. G. Johnson, www.researchdemystified.org

ask tough questions1
Ask Tough Questions
  • Is reported behavior based on self-reports?
    • How many people used illegal drugs in the past week?
  • Does the research make a giant leap to conclusion?
    • Did a study with all male participants but generalizes to everyone?

Dr. G. Johnson, www.researchdemystified.org

creative commons
Creative Commons
  • This powerpoint is meant to be used and shared with attribution
  • Please provide feedback
  • If you make changes, please share freely and send me a copy of changes:
    • Johnsong62@gmail.com
  • Visit www.creativecommons.org for more information

Dr. G. Johnson, www.researchdemystified.org