Data Collection and Sampling

1. Data Collection and Sampling

2. Primary Data • There are various methods for collecting primary (original) data • Eg questionnaire, survey, interview, observation • Control over investigation much greater • Can more easily avoid “data-driven” research • Cost can be prohibitive • Pilot studies can be very helpful

3. Choice of method • Shipman: choice often between sampling and case study • Intensive versus extensive research design • Qualitative versus quantitative data • Interpretivists favour the former; positivists favour the latter • All primary research involves selection • Most methods require sampling

4. Sampling: general principles • No a priori superiority of any method • Trade-offs: standardisation versus control, generalisability versus flexibility • Shipman: sampling method used dependent on nature of study undertaken • Basis for sample must be transparent • Cost of surveying entire population is prohibitive (e.g. census) • Constraint of feasibility

5. Sampling: definitions • Population: must be defined • Finite population: e.g. voters • Sampling unit: single potential member of sample • Sampling frame: list of sampling units (NB 1936 US Presidential election) • Sample: drawn from sampling frame

6. Probability Sampling • Probability of each sampling unit being chosen is known (often equal probability) • Simple random sampling: classic method, regarded as most reliable, least biased • List numbered sampling frame members and select via random number generator • Other probabilistic methods are available

7. Systematic sampling • List members of sampling frame • Choose first sample member randomly • Then choose every Kth unit, where K=N/n • More convenient than SRS for large popn • Can be a systematic pattern in sample list, leading to bias; e.g. corner shops

8. Stratified sampling • Divide population into groups of alike members • Strata sizes usually proportionate to popn • Draw randomly from groups • Cost effective • Ensure representativeness • Can lead to excessive number of sub-groups

9. Cluster Sampling • Select large groups • Select sampling units from clusters randomly • Example: take a city, divide into areas, number areas, select areas randomly, number units within areas, select units randomly • Very cost-effective • Very good if sampling frame poorly defined

10. Non-probability Sampling • Convenience sampling: select whoever is available • Quota sampling: collect data according to proportions of the population • Selection of subjects absolutely crucial • Requires great skill of interviewers • Snowball sampling: select next subject from previous subject

11. Non-Probability Sampling • Theoretical sampling: select those most likely to be affected by an issue • Can ignore things which do not fit • Can interpret observations according to the theory • Non-prob sampling cannot claim representativeness as easily but gives much more discretion and control

12. Response Rates • Another possible trade-off is on response rates • R = 1 - (n-r)/n • Even if initial sample size is appropriate (n’ = n/(1+(n/N)) where n = s2/SE2: see F-N and N: 194-9) response rates can be low • Postal questionnaires: typically 20-40% • Non-response bias

13. Response Rates • Non-respondents could affect findings • If reason for non-response is related to issue: e.g. reluctance to interview drunks hampers study on alcoholism • Response rate can be improved by cover letter, callbacks, skill of researcher, length of questionnaire, types of question

14. Conclusions • All types of primary data require selection • If sampling used: various methods possible • Sampling method relates to research tool • Different data collection techniques: questionnaires, interviews, etc. - all to be studied in Research Methods 2 - all have advantages and disadvantages

15. Secondary Data

16. Introduction • Primary quantitative data has several advantages, particularly control; qualitative data too • Do not equate primary and qualitative • Today: advantages of secondary data • Searching on electronic data sources including the Internet

17. Secondary data • Primary/secondary is not = qualitative/quantitative • Qualitative can include secondary data sources such as personal documents, auto/biographies, etc. • Secondary: collected by someone else, e.g. another academic researcher, business, government agency, etc.

18. Secondary data • Used extensively in social science • Durkheim: suicide • Marx: wages, incomes, prices • Weber: church records • Economists mainly use secondary data

19. Advantages of Secondary Data • Might be the only data available • Enables longitudinal /time series work • Cheaper (cost and time) and more convenient than primary data • Aids generalisation • Arises from natural settings (nonreactive/unobtrusive data) • Allows replication and checking - validity

20. Disadvantages of Secondary Data • May be not exactly the data required • Differences in underlying sampling, design, questions asked, method of ascertaining information, etc. • Differences lead to bias • Method of data generation crucial to econometric studies

21. Electronic Data Sources • Through the library system • Through the internet • Known versus unknown sources • Known sources via library catalogue • Problem of reliability/credibility is common to all electronic sources (more than non-electronic sources)

22. Electronic Data - Literature • You can search by author or subject across journals, via several static websites/portals: • www.econlit.org/ • www.sosig.ac.uk • www.mimas.ac.uk • www.economics.ltsn.ac.uk • www.esds.ac.uk

23. Electronic Data: Databases • There are many databases available online • Most have standardised, national data free to download in various formats • Common file format is .csv; but .html and even .xls files also common

24. OECD: • ONS: • UN: • Penn World Tables: • BEA (US): • Ameristat: • Eurostat: • World Bank: • CIA: • US Statistical Abstract: • See Dissertation homepage/hb

25. Conclusions • Secondary data has many advantages and disadvantages relative to primary • There is a wide range of secondary data available • Much data is available on the internet • Internet sources must be scrutinised more closely than other sources

26. Qualitative Data

27. Introduction • Principals of research design and sampling basically hold for quantitative and qualitative data • However, they apply most easily to quantitative analysis • Qualitative analysis has different foci • Qualitative analysis relatively (to quant; other soc sci) unused in economics

28. Qualitative techniques: types • Case study • Fieldwork (ethnography) • Observation • Unstructured interviews • Analytic induction/grounded theory • Discourse analysis • Theoretical sampling

29. Qualitative techniques: principals • Qual often = not quantitative • Can use quant for pattern detection, qual for causal analysis • Or use qual and quant as equals in inference (triangulation) • Quantification often inappropriate

30. Qualitative techniques: principals • Interpretivism, verstehen • Used to be associated only with using autobiography, letters, personal documents, diaries • Ethnography fairly recent: • Focus on cases rather than generality

31. Qualitative techniques: principals • Analysis not really a separate stage of research • Design, data collection and analysis all simultaneous and continuous • Open-ended approach: Theory and conclusions formed iteratively • Imagination is crucial • Recognise importance of exceptions • Context is crucial

32. Fieldwork • Study of people acting in their daily lives • Access a group but remain somewhat detached • Approach with key questions • Teams get range of perspectives • Danger of self-perception and bias

33. Participant Observation • Adopt perspectives of subject group in order to understand them • Learning language, customs, behaviours, work, leisure, etc. • Hanging around and learning the ropes • Being an outsider can changes subjects’ behaviour • Complete participation - researcher wholly concealed –  contamination and artificiality

34. Participant Observation • Researchers can go native (internalise group lifestyle) • Covert researchers can be in danger or create detrimental behaviour • Researchers can be “piggy in the middle” • Covert: recording observations can be difficult (e.g. need hidden cameras) • Serious ethical issues with covert observation

35. Employ analytic induction • Go in with prejudices and theories • Revise theory in light of evidence • Generate new theories until evidence seems to fit • Flexibility accorded but also required by the researcher • Need to be open to disconfirming cases

36. Grounded theory • Data collected • Develop categories (with inevitable theoretical priors and language) • Categories checked by data • Once categories seem secure and grounded in the evidence, formulate interconnection between categories

37. Evaluation • Broad range of qualitative techniques • Control over the investigation; less data driven; flexibility much greater than quantitative studies • Logistically difficult: Huge amounts of data produced and problems with manipulation (although Nvivo will help with this) • Must be careful to collect evidence widely to avoid bias • Can be ethical issues re: data collection and reporting