Measurement

1. Measurement Measurement is the process where we �map� phenomena using numbers or values. Questionnaires, �map� social phenomena using numbers that correspond with responses (e.g., the values and value labels in SPSS). What is your religious affiliation? 1. Protestant 2. Catholic 3. Jewish 4. Muslim 5. Other

3. Types of Measurement Ordinal Measurement: Measures social phenomenon that can be rank-ordered or sequenced in terms of more or less of the phenomenon being measured. 1 2 3 4 5 (Range) * * * * * (Phenomenon) How often do you attend religious services? 1. Never 2. A few times a year 3. Once a month 4. Once a week 5. More than once a week

4. Types of Measurement Interval Measurement: Measures social phenomenon on the basis of equal underlying intervals on the measurement scale. 0 1 2 3 4 5 (Range) * * * * * * (Phenomenon) Please indicate your overall level of approval for the government of Ontario. 0 -------- 1 -------- 2 -------- 3 -------- 4 -------- 5 Do not approve Total Approval �.where the meaning of 0 is arbitrary or created by the researcher.

5. Types of Measurement Ratio Measurement: Measures social phenomenon on the basis of equal underlying intervals on the measurement scale with a non-arbitrary zero point. 0 1 2 3 4 (Range) * * * * * (Phenomenon) How many siblings do you have? ____ Number of siblings

6. More on Measurement�. Ratio-level variables contain the most information . So can always measure down from ratio to interval, ordinal, or nominal through recoding variables in SPSS. Nominal & ordinal measures have less information so we cannot recode nominal or ordinal variables up to the interval or ratio level. However, by summing nominal or ordinal values from 2+ variables, you can create an index that boosts the level of measurement for the variable measured by the index.

7. Survey Research Surveys have been used by governments for a long time! (ancient Egypt and Rome; in the U.S. and Europe since the 19th century) Surveys remain a dominant method of inquiry in the social sciences, marketing, private polling firms and by government agencies (e.g., Statistics Canada).

8. When to use Survey Research? When studying social phenomenon not amenable to direct observation�.you observe the phenomenon indirectly (by asking others who have experienced it or formed an attitude on it). Surveys have individual people as the unit of analysis. Need data from a population too large to observe directly. Probability sampling optimizes external validity.

9. Why do people respond to surveys? Exchange theory posits that people behave in ways that maximize personal utility (the difference between expected rewards and expected costs of an action). For the researcher this implies that 3 goals must be achieved for a successful survey: (1) minimize costs; (2) maximize rewards; and (3) establish trust.

10. Ways to Maximize Rewards: Show positive regard to respondent. Explain they are �special� & vital for your research. They are being consulted. Show sincere appreciation. Use interesting, respondent-friendly questionnaires or interview schedules. Offer tangible rewards to respondents (e.g., lottery tickets and draws; direct $$$ payments for small surveys that use face-to-face interviews.)

11. Ways to Minimize Costs: Minimize time & effort. Keep questionnaires clear & simple. Use ONLY relevant questions. Minimize chances for emotionally upsetting respondents, or making them feel threatened. Provide assurances of anonymity & confidentiality. Minimize direct & indirect costs.

12. Ways to Maximize Trust: Identify research with legitimate, trustworthy organization or public figure. Open lines of communication (e.g., researcher�s phone number, e-mail, snail mail address). Promise executive summary.

13. Mailed (Self-administered) Surveys Traditionally popular because of low, fixed costs, the major downside is potential for a very low response rate (10% or lower). A properly designed & executed mailed survey should garner a response rate of 60% +�but they are NOT cheap!!! Require careful planning & execution, they �sell themselves� to the respondent. There is no interviewer to gloss over errors, eliminate worries, clarify questions!!!

14. Mailed Surveys Always pretest the questionnaire on colleagues. Good idea to pretest end-users of the research (e.g., clients, sponsors). Ideally, pretest on a small sample from the population�.helpful in identifying ambiguous and / or threatening questions. Small quota-samples (formed into focus groups) are great for pretesting.

15. How Mailed (Self-administered)Surveys should look�. First impressions & image are everything! The questionnaire must appear important, well-organized, and easy to complete! Use official letterhead on envelopes and paper.Cover should stimulate interest and convey legitimacy & importance of study. Print questionnaire in reduced form (85%-90%) or smaller font�except if studying an older population where enlarged form or large font is better. Print questionnaire on good quality, white or off-white paper.

16. How to implement Mailed Surveys: Timing is everything! If questionnaire is not completed & returned within 5 days of arriving at respondent�s home it is likely to end up in the garbage or recycling bin. Week 1: First contact (call or postcard) Week 2: First mailing of questionnaire + cover . Week 3: General reminder letter. Week 5: Second mailing of questionnaire + 2nd cover. Week 7: Third mailing of questionnaire or a 2nd reminder letter.

22. Final Guidelines for how Mailed Surveys should be implemented�. Never have a mail or self-administered survey �still in the field� in December because of Christmas. You should aim to have all the surveys back from respondents by the end of November. Avoid going into the field in mid-summer (July or August)�.due to summer holidays & vacations.

23. Telephone Surveys Telephone surveys depend entirely on verbal communication�so our concern is with how the questionnaire sounds when delivered over the phone! Most people are much better at reading than they are at listening. Respondents generally feel more stress with telephone surveys.

24. More on Telephone Surveys... Find ways to attract & maintain repondent interest & concentration. Begin with user-friendly questions�easy, interesting, non-trivial. Avoid long, complex questions�.break them into 2 or more shorter, simpler questions (follow the KISS rule), or use a key-word summary Embed responses in stem of question. Spread out the most interesting questions (might include a few interesting pseudo-questions).

25. Face-to face Surveys Most popular type is the structured interview where the questions, wording, and sequence are fixed and determined by the researcher. Interviewer and Respondent similarity (sex, age, ethnicity, dress)�.interviewer should convey a non-intimidating but professional appearance. Place of interview (plan place & time so as to maximize respondent utility). Familiarity with Questionnaire by interviewer. (goal is a team of interviewers who are like a troupe of actors). Interviewer training... recruit �sociable� persons. Use role playing in interviewer training.

26. Comparing the 3 Survey Types: Recruiting, training, management of interviewers makes face-to-face surveys the most costly in terms of $$$, time, energy. Researcher has no control over face-to-face interview . Lack of anonymity. Face-to-face interviews have the highest response rate (85%+). Interviewer can assist confused or uncertain respondents with clarifications & probes. Interviewer control of interview situation. Interviewer collection of ancillary information.

27. Comparing the 3 Survey Types: Telephone surveys (TS) provide timely information at minimal cost. Researcher can monitor & manage interviewer. If well designed & executed TS enjoy high response rate (70%+). TS higher anonymity than face-to-face. TS vulnerable to telephone biases. Complex questions poorly suited to TS. Respondent vulnerable to distractions & lapses of concentration.

28. Comparing the 3 Survey Types: Mailed surveys (MS) vulnerable to low response rates. Properly designed & implemented MS costs $$$, time, energy�although MS still less expensive than most face-to-face interviews. Little control over the questionnaire. More anonymity than face-to-face & telephone surveys. Better with complex questions than telephone surveys. Less influenced by �anti-telemarketing bias�

29. Online Surveys: E-mail surveys are sent via e-mail to a respondent; Web surveys direct respondents to a website to answer a questionnaire. E-mail surveys used for smaller, more homogenous groups; Web surveys for larger, heterogenous groups of on-line users.

30. E-mail surveys Embedded e-mail surveys include questions in body of e-mail. Respondents indicate replies with simple notations; Open-ended questions ask respondent to type in an answer. Attached questionnaires arrive as attachments to e-mail. Respondents have the option of returning attachment in a subsequent e-mail, or printing attachment out and returning by fax or snail-mail.

31. Web Surveys Web surveys invite or direct respondents to a web-site where questionnaire can be found. Web surveys use a wide variety of embellishments (e.g., radio buttons, check boxes, icons, pull-down menus, colour & background, boxed areas for open-ended responses, filter questions, multiple or single questions on screen, progress bars, real-time data collection & analysis).

32. Advantages/Disadvantages of Online Surveys: 1. Low Cost (+) 2. Fast response and data analysis (+) 3. Fewer unanswered questions (+) 4. Better responses to open-ended questions (+) 1. Low response rate (-) 2. Unknown representativeness of online samples (-) 3. Higher motivation in respondents (-) 4. Confidentiality/Anonymity (-) 5. Bias from multiple replies (-)

33. General Principles: Questionnaire Construction Questions must provide information to test hypotheses. Questions must motivate. (1) Kind of information (2) Question structure (3) Question wording.

34. Kind of Information Sought: Attitudes: Evaluations toward some attitude �object� (e.g., abortion). Beliefs: Assessments of what is true or false (e.g., global warming). Behaviors: What respondents have done, are currently doing, or intend to do (e.g., voting behaviors). Attributes: Demographic traits (e.g., education level). Attitude questions more sensitive to variations than Beliefs which are more sensitive than Behaviors which are more sensitive than Attributes.

35. Attitudes & beliefs are�.. complex & related to other attitudes/beliefs. held with degrees of conviction. fluctuate.

36. Methodological Implications 1. More sensitive a question is to wording variations, the more care you use in wording the question. 2. More sensitive a question is to wording variations, the more you should consider using multiple questions (indexes).

37. 1. (Open Ended) What should be done in order to improve the economy of North Bay? 2. (Closed with ordered choices) In the past year, about how often did you attend church services? 1. Once a week or more. 2. Once a month 3. Less than once a month 4. Never 3. (Closed with unordered choices) Which one of the following is most responsible for inflation in Canada? 1. Union demands for more pay 2. Excessive corporate profits 3. Spending by government 4. (Partially closed) Do you consider yourself to be 1. Employed full-time 2. Employed part-time 3. Unemployed 4. Other, specify____________________.

38. More on Question Structure... Open-ended Questions are employed when you cannot anticipate responses�or when listing all major responses is impractical. Stimulating thought, soliciting strong opinions, probing memories, clarifying complex attitudes. Valuable in exploratory research. Very demanding. Responses can be difficult to classify & code. Use sparingly in mailed or self-administered surveys

39. More on Question Structure... Close-ended questions with ordered choices restrict thinking�.use for topics that are well-defined. Well-suited to measuring attitudes / beliefs�.often combined into indexes. Less demanding.

40. More on Question Structure... Close-ended questions with unordered choices�used to measure priorities & preferences. Potentially very demanding. Are all relevant options are stated?

41. More on Question Structure... Partially close-ended question avoid overlooking important answer choices. Useful where respondents may resent being �pigeon-holed�. Choice depends on: information needed, research goals, and type of survey.

42. Vignette Questions: Uncover norms & values. Present scenarios & ask response. Complicated & controversial issues (e.g., assisted suicide, same-sex parenting, stem-cell research).

43. Brian and Cathy are a couple in their early 40s. Brian�s parents, who live several hundred miles away, had a serious car accident and need long-term daily help. Brian is their only son. He and his wife both could get job transfers to work near his parents. What should Brian and Cathy do? Move to live near Brian�s parents. Have Brian�s parents move to live with them. Give Brian�s parents money to help them pay for daily care. Let Brian�s parents make their own arrangements. Do something else, please specify:

44. Question Wording Simple & Straightforward Wording. 1st caveat: If subject is inherently complex, you may need to use complex words to convey meanings. 2nd caveat: If sampling certain groups, complex wording may be necessary to facilitate communication & credibility.

45. Question Wording Aim for Clarity�. or get vague answers. Too vague: How often do you read? Too precise: How many non-fiction books did you read in the past 3 years? Just right: About how many books did you read in the past year? Avoid slanting questions with loaded or biased words: (unequal comparisons, unbalanced answer categories, emotionally charged words).

46. Treat objectionable questions with care 1. Broad answer categories. 2. Use contextual material to presenting topic as a �debatable� or �open� issue. 3. Attitudes not behaviors (e.g., ask opinions rather than behaviors).

48. Avoid double-barrelled questions. Do you favour legalization of marijuana for use in private homes and public places? 1. NO 2. YES Mutually exclusive answer choices. How did you hear about this job? 1. INTERNET 2. AT WORK 3. FROM A FRIEND 4. ON TELEVISION 5. ON RADIO

49. Question Ordering: Order to motivate . Descending gradient of social importance (important before less important questions). Group similar content together. Within content areas, group similar response categories together (less effort; better validity). Group subject areas together�gives your survey a logical flow. Move from least to most objectionable. Place most demographic questions near end (e.g. age, marital status, education�.income is normally the last question asked).

50. Question Formatting: Use lower case for questions; upper case for answers. Arrange response categories and associated numbers (values) in a vertical line: Your present marital status is: 1. NEVER MARRIED 2. MARRIED 3. COHABITING 4. SEPARATED 5. DIVORCED 6. WIDOWED