session 20 data collection in the field n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Session 20: Data Collection in the Field PowerPoint Presentation
Download Presentation
Session 20: Data Collection in the Field

Loading in 2 Seconds...

play fullscreen
1 / 28

Session 20: Data Collection in the Field - PowerPoint PPT Presentation


  • 148 Views
  • Uploaded on

Session 20: Data Collection in the Field. Ina F. Wallace RTI International. IES/NCER Summer Research Training Institute, June 2007. Data Collection in the Field Topics Covered. Methods of Data Collection Use of State-of-the-Art Data Collection Procedures Ensuring Data Quality

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Session 20: Data Collection in the Field' - emily


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
session 20 data collection in the field

Session 20: Data Collection in the Field

Ina F. Wallace

RTI International

IES/NCER Summer Research Training Institute, June 2007

data collection in the field topics covered
Data Collection in the Field Topics Covered
  • Methods of Data Collection
  • Use of State-of-the-Art Data Collection Procedures
  • Ensuring Data Quality
  • Conducting Small-Scale Validation Studies
  • Ethical Issues in Data Collection
methods of data collection
Methods of Data Collection
  • Interviewer Administered
    • In Person
      • Surveys
      • Assessments
    • Telephone
  • Self-Administered
    • Paper and Pencil
      • Mail-in
      • In-person
    • Computer Assisted Self Interview
    • Web procedures
data collection mode characteristics
Data Collection Mode Characteristics

Adapted From Biemer & Lyberg (2003). Introduction to Survey Quality

face to face interviewing
Advantages

Maximum degree of communication and interaction

Can utilize PAPI or CAI

Allows for long, complex interviews

Variety of tactics can be used to gain cooperation

Response rates usually high

Disadvantages

High cost due to travel

Can generate social desirability for sensitive questions

Interviewers can affect respondents

Can generate interviewer variance

Presence of others can affect responses

Falsification of interview

Face-to-Face Interviewing
telephone interviewing
Advantages

Cost lower than face to face

More easy to provide training, supervision, and feedback

Social desirability bias less than with face to face

Interviewer variance less

Disadvantages

Less flexible

No ability to use visual aids

Must be shorter

Questions must be less complicated

Less capacity for motivating the respondent

No coverage of non-telephone units

Response rates lower

Telephone Interviewing
mail surveys
Advantages

Lowest cost

Low social desirability

Can utilize visual aids

Allows respondent to take time and look up information (if necessary)

No interviewer variance

Disadvantages

Little control over who completes the survey

Item non-response high

Response rates lower

Require a long field period to obtain an acceptable response rate

Respondents must have good literacy skills

Limit to closed-format questions

Mail Surveys
computerized procedures
Computerized Procedures
  • Computer Assisted Personal Interview (CAPI)
  • Computer Assisted Telephone Interview (CATI)
  • Computer Assisted Self Interview (CASI)
  • Audio Assisted Self Interview (ACASI)
  • Telephone Audio Assisted Self Interview (T-ACASI)
  • Computer Assisted Data Entry (CADE)
  • Web Surveys
computer assisted procedures
Advantages

Complex skip patterns and branching are easier to follow

Previously acquired information can be programmed

Ranges, inconsistencies, and illegitimate values can be checked

Help screens can be included

Data are available for analysis more quickly

Disadvantages

Time consuming to develop and program computer

Must be adequately tested

Can’t change things easily once fielded

Risk of computer crashing

Costly

Computer Assisted Procedures
direct assessment
Direct Assessment
  • Used to obtain psychological and educational information about an individual
  • Individually administered
  • Research driven NOT for clinical or diagnostic purposes
  • Follows a standardized protocol
observational techniques
Observational Techniques
  • Involve a researcher making observations within a natural setting
  • Direct Observation - individuals know they are being observed
    • Continuous Monitoring recording as much behavior as possible
    • Time Allocation randomly visiting the sample to observe at different times
  • Unobtrusive Observation – individuals do not know they are being observed
observational techniques1
Advantages

No or limited participation needed

May be good for generating hypotheses

Sensitive social issues can be obtained more accurately

High in validity because of the depth of information collected

Disadvantages

Variance associated with observers is similar to that found with interviewers

Can be very costly as they are often labor intensive

Time consuming

Reliability can be low

Generalizability can be low

Observational Techniques
summary issues in choosing a strategy
Summary: Issues in Choosing a Strategy
  • Type of Population
  • Nature of Research Issue
  • Question Form
  • Question Content
  • Response Rates
  • Costs
  • Length of Data Collection
methods of administration
Methods of Administration
  • Standardized Interviewing Approach
    • A protocol in which interviewers interact with the respondent in a manner which is consistent across all interviewers
  • Conversational Interviewing
    • A protocol in which interviewers modify and adapt questions to the respondent’s situation
  • Hybrid Interviewing
    • Combination of thetwo approaches
factors that can explain interviewer variance
Factors that Can Explain Interviewer Variance
  • Respondent
  • Characteristics
  • Knowledge
  • Interest/motivation
  • Confidence
  • Strength of convictions
  • Expectations
  • Interviewer
  • Characteristics
  • Appearance
  • Motives
  • Beliefs/attitudes
  • Perceptions
  • Expectations
  • Skills/Knowledge

Interviewer Effects

  • Survey Conditions and Setting
  • Mode of interview
  • Standardization
  • Interviewer training
  • Interviewer supervision
  • Monitoring/observation
  • Questionnaire
  • Definition clarity
  • Terminology/jargon
  • Question form
  • Instructions
  • Question wording
  • Question topic

From Biemer & Lyberg (2003). Introduction to Survey Quality

factors that can explain interviewer variance1
Factors that Can Explain Interviewer Variance
  • Respondent
  • Characteristics
  • Knowledge
  • Interest/motivation
  • Confidence
  • Strength of convictions
  • Expectations
  • Interviewer
  • Characteristics
  • Appearance
  • Motives
  • Beliefs/attitudes
  • Perceptions
  • Expectations
  • Skills/Knowledge

Interviewer Effects

  • Survey Conditions and Setting
  • Mode of interview
  • Standardization
  • Interviewer training
  • Interviewer supervision
  • Monitoring/observation
  • Questionnaire
  • Definition clarity
  • Terminology/jargon
  • Question form
  • Instructions
  • Question wording
  • Question topic

From Biemer & Lyberg (2003). Introduction to Survey Quality

role of the interviewer assessor
Role of the Interviewer/Assessor
  • Gaining Cooperation
    • Converting reluctant participants
    • Recognizing and dealing with refusals
  • Training and Motivating Respondents
    • Interviewers implicitly and explicitly teach respondents how to behave
  • Obtaining Quality Data
  • Ensuring Confidentiality
assessing children
Assessing Children
  • Assessor Characteristics
    • Flexibility
    • Vigilance
    • Self-awareness
    • Eye contact
  • Establishing Rapport
    • Allaying apprehensions
    • Helping children feel at ease
    • Maintaining limits on permissible behavior
  • Arranging the Physical Environment
ensuring quality data
Ensuring Quality Data
  • Recruiting interviewers/assessors
    • Characteristics of all data collectors
    • Characteristics of interviewers
    • Characteristics of assessors
  • Training on standardized procedures
  • Certification as data collectors
  • Monitoring performance
training
Content of Training

Presenting the study

Contacting participants

Administering the questionnaire/ assessment

Probing

Recording responses

Confidentiality

Interpersonal relations

Administrative procedures

Procedures for Training

Written materials

Lecture and presentation

Computer-based tutorials

Planned exercises

Role Playing

Observation

Certification

Training
interviewer training agenda
Interviewer Training Agenda
  • Introduction to the Study
  • Role of the Interviewer
  • Confidentiality Procedures
  • Review of Questionnaire and Protocols
  • Standardized Interviewing Techniques
  • Round Robin Interview
  • Gaining Cooperation
  • Presentation of Scenarios
  • Round Robin Demonstrations
  • Paired Mocks
  • Certification
monitoring performance
What to Monitor

Detection and prevention of falsified information

Compliance with rules and guidelines about administering the questionnaire or assessment

Performance on non-interview/assessment tasks

How to Monitor

Telephone verification with a sample of respondents

Direct observation – in-person or by telephone

Audio-recording – CARI

Review of materials

Performance and production measures

Monitoring Performance
human subjects issues
Human Subjects Issues
  • Institutional Review Boards
  • Informed Consent
    • Who is carrying out research under whose sponsorship
    • Description of purposes and procedures
    • Duration of involvement
    • Risks and Benefits
    • Confidentiality
    • Voluntary nature of research including stopping early and skipping questions
    • Compensation
    • Contact information
  • Confidentiality Issues
human subjects issues1
Human Subjects Issues
  • Institutional Review Boards
  • Informed Consent
    • Who is carrying out research under whose sponsorship
    • Description of purposes and procedures
    • Duration of involvement
    • Risks and Benefits
    • Confidentiality
    • Voluntary nature of research including stopping early and skipping questions
    • Compensation
    • Contact information
  • Confidentiality Issues
data collection evaluation
Data Collection Evaluation

Adapted From Biemer & Lyberg (2003). Introduction to Survey Quality

post data collection evaluations
Post-Data Collection Evaluations
  • Experiments
  • Observational Studies
  • Internal Consistency Studies
  • External Validation Studies
  • Administrative Record Check Studies
  • Re-interview/Re-Test Studies
post data collection evaluations1
Post-Data Collection Evaluations
  • Experiments
  • Observational Studies
  • Internal Consistency Studies
  • External Validation Studies
  • Administrative Record Check Studies
  • Re-interview/Re-Test Studies