research methods design in psychology l.
Skip this Video
Loading SlideShow in 5 Seconds..
Research Methods & Design in Psychology PowerPoint Presentation
Download Presentation
Research Methods & Design in Psychology

Loading in 2 Seconds...

play fullscreen
1 / 61

Research Methods & Design in Psychology - PowerPoint PPT Presentation

  • Uploaded on

Research Methods & Design in Psychology. Lecture 2 Survey Design 2 Lecturer: James Neill. Overview. Survey construction - nuts & nolts Sampling Ethics Levels of measurement Measurement error. What is a Survey?. A standardised stimulus A measuring instrument

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Research Methods & Design in Psychology' - Olivia

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
research methods design in psychology

Research Methods & Design in Psychology

Lecture 2

Survey Design 2

Lecturer: James Neill

  • Survey construction - nuts & nolts
  • Sampling
  • Ethics
  • Levels of measurement
  • Measurement error
what is a survey
What is a Survey?
  • A standardised stimulus
  • A measuring instrument
  • A way of converting fuzzy psychological stuff into hard data for analysis
survey construction nuts bolts
Survey Construction – Nuts & Bolts
  • Constructing questions
  • Modes of response
  • Response formats <-> LOM
  • Measurement error
  • Survey formatting
constructing questions
Constructing questions
  • Define target constructs
  • Check related research & questionnaires
  • Draft items (aim to have multiple indicators)
  • Pre-test & revise
when drafting questions aim to
When drafting questions aim to:
  • Focus directly on topic/issue
  • Be clear
  • Be brief
  • Avoid big words
  • Use simple and correct grammar
bias in questions
Bias in questions
  • Inapplicable
  • Over-demanding
  • Ambiguous
  • Double negatives
  • Double-barrelled
  • Leading
  • Loaded
bias in responding
Bias in responding
  • Social desirability
  • Acquiescence or Yea- and Nay-saying
  • Self-serving bias
  • Order effects
modes of survey administration
Modes of Survey Administration


  • high demand characteristics
  • can elicit more information


  • lower demand characteristics
  • information may be less rich
objective vs subjective
Objective vs. subjective

Objective: How times during 2000 did you visit a G.P.?

Subjective: Think about the visits you made to a G.P. during 2000. How well did you understand the medical advice you received?perfectly very well reasonably poorly not at all

open ended vs close ended
Open-ended vs. close-ended


  • rich information can be gathered
  • useful for descriptive, exploratory work
  • difficult and subjective to analyse,
  • time consuming


  • important information may be lost forever
  • useful for hypothesis testing
  • easy and objective to analyse
  • time efficient
open ended questions examples
Open-ended questions - Examples

What are the main issues you are currently facing in your life?

How many hours did you spend studying this week? _________

close ended questions example 1
Close-ended questions – Example 1

What are the main issues you are currently facing in your life? (please all that apply)

 financial

 physical/health

 academic

 employment/unemployment

 intimate relations

 social relations

 other (please specify)


close ended questions example 2
Close-ended questions – Example 2

How many hours did you spend studying this week?

 less than 5 hours

 > 5 to 10 hours

 > 10 to 20 hours

 more than 20 hours

close ended rating scales
Close-ended rating scales
  • Likert scale
  • Graphic rating scale
  • Semantic differential scale
  • Non-verbal scale
  • Frequency scale
likert scale
Likert Scale

Pick a number from the scale to show how much you agree or disagree with each statement:

1 2 3 4 5

strongly disagree neutral agree strongly

disagree agree

1 2 3 4 5

strongly agree neutral disagree strongly

agree disagree

graphic rating scale
Graphic Rating Scale

How would you rate your enjoyment of the movie you just saw? Mark with a cross (X)

not enjoyable very enjoyable

semantic differential scale
Semantic Differential Scale

What is your view of smoking?

Tick to show your opinion.

Bad ___:___:___:___:___:___:___ Good

Strong ___:___:___:___:___:___:___ Weak

Masculine ___:___:___:___:___:___:___ Feminine

Unattractive ___:___:___:___:___:___:___ Attractive

Passive ___:___:___:___:___:___:___ Active

non verbal scale
Non-verbal Scale

Point to the face that shows how you feel about what happened to the toy.

verbal frequency scale
Verbal Frequency Scale

Over the past month, how often have you argued with your intimate partner?

1. All the time

2. Fairly often

3. Occasionally

4. Never

5. Doesn’t apply to me at the moment

sensitivity reliability
Sensitivity & Reliability
  • Scale should be sensitive yet reliable.
  • Watch out for too few or too many options

Scale of measurement guidelines

General aim: Maximise sensitivity (i.e. more options)

Maximise reliability (i.e. less options)

How many measurement options?

  • Minimum = 2
  • Average = 3 to 7
  • Maximum = 10?










watch out for too many or too few responses
Watch out for too many or too few responses

“Capital punishment should be reintroduced for serious crimes”

1 = Agree 2 = Disagree

1 = Very, Very Strongly Agree 7 = Slightly Disagree

2 = Very Strongly Agree 8 = Disagree

3 = Strongly Agree 9 = Strongly Disagree

4 = Agree 10 = V. Strongly Disagree

5 = Slightly Agree 11 = V, V Strongly Disagree

6 = Neutral

  • Sampling Terminology
  • What is Sampling?
  • Sampling Techniques
  • Example: Shere Hite’s Sex Survey
  • Summary of Sampling Strategy
sampling terminology
Sampling Terminology
  • Population
  • Sampling Frame
  • Sample
  • Representativeness
what is sampling
What is sampling?

“Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.”

- Trochim, 2002

sampling techniques
Sampling Techniques
  • Probability sampling
    • Random
    • Systematic
    • Cluster
      • Multi-Stage Cluster
  • Non-probability sampling
    • Quota
    • Convenience
    • Snowball
representativeness of sample depends on
Representativeness of sample depends on:
  • adequacy of sampling frame
  • selection strategy
  • adequacy of sample size
  • response rate – both the % & representativeness of people in sample who actually complete survey
  • Note: It is better to have a small, good sample than a large, poor sample.
male female relations
Male-Female Relations
  • Shere Hite ‘doyenne of sex polls’
  • Media furors & worldwide attention
  • 127-item questionnaire about marriage & relations between sexes
  • 4500 USA women, 14 to 85 years
  • Society and men need to change to improve lives of women
some of hite s findings
Some of Hite’s findings....
  • 70% married for 5 years having affairs...

(usually more for ‘emotional closeness’ than sex)

  • 76% did not feel guilty
  • 87% had a closer female friend than husband
  • 98% wanted “basic changes” to love relationships
  • only 13% married for 2+years were still in love
  • 84% were emotionally unsatisfied
  • 95% reported emotional & psychological harassment from their men
some of the critical comments
Some of the critical comments....
  • “She goes in with prejudice & comes out with a statistic.”
  • “The survey often seems merely to provide an occasion for the author’s own male-bashing diatribes.”
  • “Hite uses statistics to bolster her opinion that American women are justifiably fed up with American men.”
response rate selection bias 1
Response rate & Selection bias - 1

100,000 questionnaires

Sent to a variety of women’s groups - feminist organisations, church groups, garden clubs, etc.

4,500 replied(4.5% return rate)

response rate selection bias 2
Response rate & Selection bias - 2

“We get pretty nervous if respondents in our survey go under 70%. Respondents to surveys differ from nonrespondents in one important way: they go to the trouble of filling out what in this case was a very long, complicated, and personal questionnaire.”- Regina Herzog, University of Michigan Institute for Social Research

summary of sampling strategy
Summary of sampling strategy
  • Identify target population and sampling frame
  • Selection sampling method
  • Calculate power and required sample size
  • Maximise return rate
survey format checklist
Survey Format Checklist
  • Introduction/covering letter or verbal introducation
    • e.g. Who are you? Are you bona fide? Purpose of survey? Ethical approval? How results will be used? Confidentiality? Further info? Complaints?
  • Instructions
    • Sets the “mind frame”, but be aware few people will read it without good prompting and being easy-to-read
  • Group like questions together
  • Consider order effects, habituation, fatigue, switching between response formats
survey format
Survey Format
  • Font type / size, number of pages, margins, double vs. single-siding, colour, etc.
  • Demographics - single section, usually at beginning or end of questionnaire, only use relevant questions
  • Space for comments?
  • Ending the questionnaire – say thanks!
  • Pre-test the questionnaire & revise/refine
pre test revise
Pre-test & Revise
  • Pre-test items and ask for feedback
  • Revise:
    • items which don’t apply to everybody
    • redundancy
    • skewed response items
    • misinterpreted items
    • non-completed items
  • Reconsider ordering & layout
ethical issues how to treat respondents
Ethical issues: How to treat respondents
  • Minimise risk/harm to respondents
  • Informed consent
  • Confidentiality / anonymity
  • No coercion
  • Minimal deceit
  • Fully debrief
other ethical issues
Other ethical issues
  • Honour promises to provide respondents with research reports
  • Be aware of potential sources of bias/ conflicts of interest
  • Represent research literature fairly
  • Don’t search data for pleasing findings
  • Acknowledge all sources
  • Don’t fake (or unfairly manipulate) data
  • Honestly report research findings
levels of measurement type of data
Levels of Measurement=Type of Data

Levels of measurement = type of data

4 levels of measurement
4 levels of measurement
  • Nominal/Category
  • Ordinal
  • Interval
  • Ratio
levels of measurement discrete vs continuous
Levels of measurement – discrete vs. continuous
  • Categorical / Nominal (Discrete)
  • Ordinal / Rank (Discrete)
  • Interval (Discrete?)
  • Ratio (Continuous)
categorical nomimal
Categorical / Nomimal
  • Arbitrary assignment of #s to categories

e.g. male = 1, female = 2

  • No useful information, except as labels
ordinal ranked scales
Ordinal /Ranked Scales
  • #s convey order, but not distance

e.g. in a race, 1st, 2nd, 3rd, etc.

  • Often must be treated as categorical
interval scales
Interval Scales
  • #s convey order & distance, 0 is arbitrary

e.g. temperature (degrees C)

  • Usually treat as continuous for >5 intervals
ratio scales
Ratio Scales
  • #s convey order & distance, meaningful 0

e.g. height, age

  • ratios - e.g. 2 x old, 3 x high
why do levels of measurement matter
Why do levels of measurement matter?

different analytical proceduresare used for different levels of data

More powerful statistics can be applied to higher levels

measurement scales analysis
Measurement scales -> Analysis

categorical & nominal -> non-parametric

interval & ratio-> parametric

what are parametric stats
What are parametric stats?

= procedures which estimate PARAMETERS of a population, usually based on the normal distribution

  • any procedure which uses M, SD
    • e.g. t-tests, ANOVAs
  • any procedure which uses r
    • e.g. bivariate correlation, linear regression
what are non parametric stats
What are non-parametric stats?

(Distribution-free Tests)

= procedures which do not rely on estimates of population PARAMETERS

  • any procedure which uses frequency
    • e.g. sign test, chi-squared
  • any procedure which uses rank order
    • e.g. Mann-Whitney U test, Wilcoxon matched-pairs signed-ranks test
parametric vs non parametric stats
Parametric vs. non-parametric stats?

parametric statistics are more powerful

but are also more sensitive to violations of assumptions

measurement error
Measurement error
  • Observed score

= true score + measurement error

= true score + systematic error + random error

  • Measurement error is any deviation from the true value.
sources of error
Sources of Error





sources of measurement error
Sources of measurement error
  • Paradigm
  • Personal researcher bias
  • Sampling
  • Non-sampling
to minimise measurement error
To minimise measurement error
  • Use well designed measures
  • Reduce demand effects
  • Maximise response rate
  • Ensure administrative accuracy
  • Survey construction - nuts & bolts
  • Sampling
  • Ethics
  • Levels of measurement
  • Measurement error
respond to uc s student survey and win an ipod video valued at 380
Respond to UC’s student survey and win an iPod Video valued at $380

Features: 30GB, music videos, home movies and video podcasts, new iPod games, audiobooks, photo albums and, of course, an entire library of up to 20,000 songs.

  • Logon to OSIS before 14 March, fill in the student survey & you’ll enter the draw!