Download
1 / 77
790 likes | 935 Views
On The Qualitative Frontline: Fieldwork Recruiters’ Views On Respondent Cooperation. Prepared by: QRCA Field Committee March 2010. Table of Contents. Study Overview 3 Background 4 Study Objectives 5 Methodology 6 Note of Thanks 8 Caveat 9 Summary and Implications 10
E N D
On The Qualitative Frontline:Fieldwork Recruiters’ Views On Respondent Cooperation Preparedby:QRCA Field CommitteeMarch 2010
Table of Contents Study Overview 3 Background 4 Study Objectives 5 Methodology 6 Note of Thanks 8 Caveat 9 Summary and Implications 10 Detailed Discussion 26 Trends in Respondent Cooperation 27 Factors that Affect Respondent Cooperation 30 Recruiter Suggestions for Decreasing Cheating/Repeating 56 Recruiter Suggestions for Increasing Respondent Cooperation 61 Appendix A: Moderator’s Guide 73 Page hyperlinks work directly in the Slide Presentation View, otherwise right click on page links above, then click ‘open hyperlink’ to jump directly to a page
Background • QRCA’s Field Committee formed a Task Force to gain insights into respondent cooperation in qualitative market research studies by interviewing fieldwork service recruiters. The Committee believes that recruiters are “on the frontline” and, therefore, can be a valuable source of information and insights. • The goal is to understand what qualitative research consultants (QRCs), fieldwork services (facilities), and clients can do to increase respondent cooperation, by: • Developing best practices for increasing respondent cooperation rates. • Improving communication between QRCs/clients and fieldwork services on recruiting practices. • Reducing the incidence of cheating and repeating (respondents who lie in order to get into studies and/or to participate too often).
Study Objectives • Specific study objectives are to: • Identify trends in respondent cooperation • Identify factors that increase or decrease respondent cooperation • Obtain recruiters’ suggestions for increasing respondent cooperation • Obtain recruiters’ suggestions for reducing the incidence of cheating/ repeating
Methodology • Five QRCA Field Committee members who are practicing qualitative research consultants, most of them also facility owners, volunteered for the Task Force. Each conducted 2 groups with recruiters. • Jan Lohs, Lohs Research Group, in association with Smith Research, Chicago, IL • Anndel Martin, Opinions Unlimited, Houston/Dallas, TX • Andrea Schrager, Meadowlands and New York Consumer Centers, New Jersey and New York • Merrill Shugoll, Shugoll Research, Bethesda, MD • Diane Trotta, Trotta Associates, Los Angeles and Irvine, CA
Methodology • A series of 10 focus groups was conducted, divided between: • Recruiters specializing in consumer recruiting • Recruiters specializing in executive/professional and medical recruiting • Focus groups were about 90 minutes, with 6-8 respondents each. • Recruiters interviewed were a mix in terms of recruiting experience (minimum 6 months), gender and age. • Recruitment criteria and the moderator’s topic guide, developed by the Task Force, are appended.
Note of Thanks • Special thanks to the following companies for providing transcription services: Transcription Services, Inc TSItranscripts.com • 909-484-8185 Mark Hampton Transcripts markhampton88@yahoo.com 212-595-4083 • MCC Field and Transcription • Nadya@consumercenters.com • 201-537-5256
Caveat • The qualitative research methodology seeks to develop direction rather than quantitatively precise measures. Because of the limited number of respondents involved in this type of research, the study should be regarded as exploratory in nature and the results should be used to generate hypotheses for further testing. • This study presents views of qualitative recruiters. Comments by the Field Committee are discussed separately.
Trends in Participation • Several trends are contributing to respondent participation • Awareness – more people know about market research generally and qualitative in particular, especially focus groups. • Positive past participation – people who have participated usually have a good experience, are more likely to come again, give referrals, spread word-of-mouth. • Recession – increases interest because of incentives.
Trends in Participation • But certain trends present obstacles to respondent cooperation: • Continuing lack of research awareness among some consumers and office gatekeepers. • Company policies against employee participation, affecting business-to-business studies. • The recession – more people working longer hours, don’t want to leave office for a study. • Busy lifestyles – less time for screening, participation. • Cellphone use – harder to reach respondents, safety issues while driving.
Factors in Cooperation: Overview • Recruiters/facilities look at cooperation both in terms of the individual study and keeping respondents in their database long-term. • The screener plays a major role in cooperation (discussed below).
Factors Helping Cooperation • Study topic – those respondents consider interesting, personally relevant. • Incentives – appropriate to the market, respondent segment, amount of pre-work. • Facility contact prior to screening – emails/letters/fax introducing the research company are a good first step in recruiting. • The actual research experience – respondents who find the interview worthwhile/ enjoyable are open to participating again; seeing results of the research makes them feel they have had an impact (on product development, etc.).
Factors Hurting Cooperation: Sampling/Requirements • Narrow study requirements, quotas – database respondents become frustrated to be called repeatedly but not qualify; some opt out. • Client list studies – inaccurate information is common; respondents are wary if not told the client name. • Random digit dialing – people unaware of research are less likely to participate.
Factors Hurting Cooperation: Sampling/Requirements • Homework assignments have become more demanding – resulting in lower acceptance and show rates. • Incentives – too low for homework required, no incentive for completed homework if respondents are cancelled; respondents refuse if less than prior studies. • Scheduling – times of day that are difficult for the respondent segment. • Last-minute changes – in interview scheduling, new/revised screener questions. • Confirmation of holds – if not done quickly, respondents make other plans.
Factors Hurting Cooperation: Screeners • Length – many screeners are longer today; with questions “for information” or as a mini-survey, rather than for the purpose of screening. • Questions that are complicated or poorly written are more common: • Unclear wording. • Run-on sentences. • Long response lists. • Respondents can’t remember what they did a while ago, especially everyday usage. • Redundant questions, repeated verbal scales annoy/offend respondents. • Terminology that is hard to pronounce/understand. • Poorly translated from other languages. • Organization/flow • Difficult skip patterns • Terminate questions at end – waste respondents’ and recruiters’ time.
Cheating/Repeating • Cheaters/repeaters have become more knowledgeable about “gaming the system” in order to get into studies. • The recession raises facility concerns about respondents doing this for the money, lying about employment. • Facility procedures can help reduce cheaters/repeaters. • Electronic monitoring recruiting, confirmation calls checking screener information using a different recruiter. • Keeping the database updated on participation.
Recruiters’ Suggestions: Screeners • Introduction – be engaging and honest about the study topic/purpose; ask the project manager how to present the subject in an interesting way. • Respondents desired – talk with the project manager about what you want. • List studies – tell respondents the client name, if possible. • Length – keep screeners short (under 10 minutes at most), ask only questions needed to recruit. • Flow – make skip patterns and instructions simple/clear. • Terminates – do this on the question after the one where respondents disqualify, especially on sensitive issues (e.g., age, ethnicity) • Algorithms – use them only if absolutely necessary; recognize they complicate recruiting and increase costs.
Recruiter Suggestions: Screener Questions • Ask key questions, including security, in an open-ended way, without reading pre-coded answers – more difficult for cheaters/repeaters to game the system. • Ask children questions in simple, age-appropriate language. • Use numeric scales instead of verbal ones on a list of statements. • Ask articulation questions related to study subject – avoid ones seen as “silly.” • Put terminate questions as close to beginning as possible. • Phonetically spell out difficult terminology. • Translate screeners originally in another language into colloquial English.
Recruiter Suggestions: Other Issues • Requirements/quotas – recognize that studies with very narrow specifications are more difficult, expensive, take longer to recruit; ethnicity is more complex today. • Incentives – consult with the project manager on what’s appropriate to the market, segment, amount of pre-work required; partial payment for cancelled respondents who complete homework; give a small gift in addition to cash at the end of the interview to create goodwill. • Confirm holds – do this as soon as possible to retain desired respondents. • Homework – keep it to a minimum, especially for businesspeople; make help available on weekends for Web assignments; consider cameras as incentives. • Scheduling – ask the project manager for the best interview start times in that market; consider weekends and holidays for longer interviews (e.g., mock juries) or to recruit certain segments (e.g., students). • Refreshments – serve meals at standard mealtimes.
Overview • This study with qualitative research recruiters provides a number of very useful observations and suggestions for QRCs and end clients in planning their fieldwork in today’s world. • QRCs should try to avoid practices that cause recruiting problems and/or price increases. • QRCs should make their clients aware of the consequences of practices that complicate recruiting and/or are more costly. • It is important that QRCs/clients work collaboratively with facility project managers, asking their advice on recruiting and their particular market. • We also strongly recommend that QRCs and facilities follow the guidelines for best practices in communication and recruiting practices, including email recruitment, on the QRCA website.
Better Communication • There are reasons why QRCs/clients do not or cannot follow practices suggested by recruiters. Explaining the reasons to the facility helps the project manager and recruiters understand the situation. Examples: • Study specifications may be narrow in order to find people relevant to the client. • Homework assignments sometimes need to have multiple parts in order to enrich the research. • Modifications in respondent qualifications may be necessary after interviews in one market, such as learning that certain types of people are not true prospects for the client. • Small changes in an answer (e.g., one rating point) may disqualify a respondent – while this is frustrating, it may be necessary. • Respondent callbacks to ask additional or revised questions may be needed based on new information/ideas or on the mix of recruited respondents. • A few terminate questions may need to be toward the end of the screener – e.g., respondents can be put off if household income is asked too early.
Respondent Cooperation Trends • The economic downturn is having a slight positive impact on respondent cooperation. More people are returning recruiters’ calls and agreeing to participate in qualitative research. “Maybe it has to do with the economy; everyone needs more money.” “People are more willing to listen to you and take your call. They will put dinner on hold to answer your questions – the kids could be screaming in the background but they keep on going.” • Negatively, business-to-business studies can be more difficult to recruit today. • Some professionals have heavier-than-usual workloads and are concerned about being out of the office during the recession. • More companies have policies against employees’ participation in market research.
Respondent Cooperation Trends • Many people remain unfamiliar with market research. They are more likely to disengage from a phone conversation with a recruiter, are skeptical when told that the research company wants their opinions and will compensate them for their time. “Executive and medical recruiting is more difficult if the gatekeeper is not familiar with qualitative research.” “They have to be familiar in order to trust you with whatever you are talking about. When you call them, they do not know who you are. You mention money – ‘we’re going to pay you’ – that is a scam in their mind.” “If you are not allowed to say the company that you are calling on behalf of and they don’t know anything about you, they don’t care; they don’t want to talk to you because they don’t know anything about a focus group. They don’t know about marketing research.” “’Are you serious! You pay me?’” “You can’t get past the administrator. They think you are trying to sell something; they are paid to protect the doctor, not let people get through to them.”
Respondents’ Past Experience • Many respondents have had positive experiences in qualitative studies, which makes them willing to participate again. • They enjoyed expressing their opinions and being heard. Most are anxious the first time and then find the experience pleasurable. • Seeing an actual outcome of the research (e.g., new product/advertising discussed come to market) makes respondents excited about having had an impact. • Respondents expect the same level of incentives in future studies, are less likely to participate when it is lower. “There are people who have told me [no] if it’s not $100.”
Study Topic • Interesting or personally relevant topics attract people to participate. The gratuity becomes secondary, although it is still important. “I think it depends on the topic. It could be something that’s very interesting to them as a person and that makes them much more interested in participating.” “They will make an effort to come, even if it’s inconvenient, because they enjoy it [the topic].” “Some doctors want to learn about new techniques. They’re not just doing it for the money. And they want to get feedback about medications from other doctors.” • Topics that may not sound inherently interesting can be positioned in a positive (and honest) way. • Example: a study on “what consumers do for personal grooming” vs. “a study of deodorants.”
Study Topic • If the actual interview topic differs from what respondents expected/were told or is about issues of little interest to them, they may feel misled or be reluctant to participate again. “Very often, we find out after the fact that the vague topic ended up being an ad campaign. ‘What color or what font should we be using?’ Some doctors can get angry to think they have attended something that was quite different than what they thought it would be.”
Screener Introduction • Introductions need to capture respondent interest, provide reassuring information. • Generic introductions that give no real information about the study topic (e.g., “some issues”) do not intrigue respondents. • Clients willing to reveal their sponsorship of the study, especially on a list job, sometimes leave it out of the introduction, which can result in terminates by people who would be interested in participating. • The recruiter’s tone at the outset also plays a role in encouraging participation. “I think a good intro is one where we can say within a couple of sentences who we are, what we’re trying to do, and if you qualify we’d like to invite you to participate.” “The more the qualitative research people can provide you with ‘this is what’s going to be happening in the group,’ the more we can really sell them on the idea of coming to the session.” “Sometimes it’s not what you say but how you say it. If the topic sounds interesting in your introduction I think you can sell the idea that it will be fun to participate.”
Screener Length • Screeners have been growing longer (up to 20 minutes), creating major problems with participation rates. • More screeners today include questions for general information purposes or as a mini-survey, which goes beyond screening. • Screeners longer than about 7 minutes can cause respondents to become frustrated, to lose interest and terminate midway. “Some questions are a whole paragraph, and that’s not necessary. By the time you get to the question, they’ve already forgotten.” “Respondents sometimes feel like the screener goes on forever and they get lost in the detail.” • Terminate questions near the end of a screener annoy respondents, waste both their and the recruiters’ time. “I try to get permission from the project manager to go directly to the termination question [when I begin a screener].”
Screener Questions • Simple and direct questions are most easy to answer correctly. • Long questions with run-on sentences confuse respondents – they don’t know what part of the sentence they should answer. • A difficult-to-administer screener makes respondents thinkthe study will also be difficult. “Respondents tune out to long lists. They just answer and are not listening to what you are really saying.” “Directions can be so long that [respondents] don’t remember what they are doing.”
Screener Questions • Numeric scales with anchor points are simpler, less confusing and more time-efficient than verbal scales that recruiters have to reread for each statement. • Numeric scale example: “for each statement, choose a number from 1 to 5, where 1 means disagree completely and 5 means agree completely.” Verbal scale example: “for each statement, tell me if you agree completely, agree somewhat, neither agree nor disagree, disagree somewhat, disagree completely.” “You have to make sure that they understand the scale. Sometimes when they are filling out a rescreener before the group, they answer differently and I think it is because they didn’t understand the scale on the phone because it wasn’t logical or explained well.” “When I ask how many times they were at a particular restaurant or drank a beverage in the last year and then drill down to the last month, the numbers change and the respondent gets confused. Start with the last month. That changes when you ask about purchasing a big item like a car or taking a cruise. Those things they can remember.”
Screener Questions • Redundant questions are seen as being used to verify answers – they can annoy or even offend some respondents, who feel they are being tricked. “They say, ‘Do not waste my time.’” • Skip patterns are often not clear – more screeners now are questionnaires with complicated, hard-to-follow instructions. “The screeners used to be simpler and they have become more complicated over time. It’s not about ‘keep it simple.’” • Some screeners originally in foreign languages are not translated well. “With doctors, they use the same screener in Europe, India, China, wherever, and sometimes it doesn’t fit into our language or how we talk.” • Terminology that is not commonplace lay language (e.g., financial, medical) can be difficult for recruiters to pronounce if they are not spelled out phonetically. “I had a couple of screeners where it seemed like the screeners were like research papers. Some of them are cumbersome; they’re poorly written. The terminology they use do not encourage a successful completion.”
Study Specifications • Increasingly, narrow specifications and quotas mean that fewer people qualify for study participation. • Respondents in facility databases become discouraged about repeatedly failing to qualify, and are negative about being screened for other studies. “I’ve had some get very angry on the phone. They say, ‘Why are you wasting my time? I never qualify.’”
Algorithms • Algorithms are a major concern – these multi-question formulas have to be calculated, recruiters cannot determine immediately if a respondent qualifies. • Respondents feel their time has been wasted if they make it through most of the screener, then don’t qualify because of the algorithm. They may be less willing to be screened for the future studies. “The problem with the algorithm is that the person doing the interview can’t determine if they qualify.” “Respondents do not want to waste a lot more time on the phone while I figure out if they qualify or not. They get annoyed and hang up if the calculation takes too long.” “It builds frustration. If it happens too often, they just say ‘I don’t want to do this any more.’” • Some recruiters explain the rationale to respondents (clients look for people based on their attitudes); calculate the algorithm offline, calling back those who qualify. “Once I understood what an algorithm was and why it is important, I could help the respondent feel more comfortable going through the process.” • Some facilities pre-screen respondents with a form online, screen those who pass.
Follow-Up Calls • Last-minute changes in interview scheduling, screener questions and respondent approval are more common today – these irritate and put off respondents. • Calling respondents back with new questions is seen as an imposition. A small change in an answer (e.g., one rating point) can result in disqualification. “Once we are halfway through with the recruit, they’ll [the client] call back with a new question.” “You feel badly when you have to go through lots of questions that they already answered.” “It is ridiculous sometimes. Once they give a 7 and then an 8 when re-asked and now they do not qualify.”
Confirming Holds • Respondents put “on hold” for participation because they do not totally fit screening qualifications can become annoyed or make other plans if the client “waits too long” in accepting them for the interview. “Sometimes, the client won’t get back to the project manager until a day or two before [the session]. We’ve lost holds because they’re not available anymore.”
Incentives • Respondent cooperation is largely affected by the amount and type of gratuity. • The right amount varies by market – the project manager has experience to know what works best in their market with different segments. • Beyond the basic incentive, it can be useful to pay additional gratuities, such as for completed homework, for parking, and drawings for early arrival at the facility. • While high-interest topics increase participation rates, this does not mean that lower incentives are a good idea – show rates can be hurt. “If it’s two hours, and it’s $75, they’re usually not interested. I would say if you have somewhere between $100 to $125, you’ve got them.” “Some people, that’s the first thing they’ll ask me, ‘Is the gratuity [amount]?’” “I don’t know why but respondents seem to like it better when we separate the incentive for babysitting or parking rather than telling them one lump sum amount.”
Sampling Method • Cold calling through random digit dialing is the most challenging method – many people contacted may not be familiar with research generally or the company. • Similarly, client list studies on which recruiters are not authorized to name the study sponsor are difficult – respondents are wary about where the research company got their name/information. “[They] are more suspicious, they don’t know who we are, what we do. You have to give them a level of comfort, or they’ll hang up on you.” “’I’m so-and-so from such-and-such’ – it suddenly sounds like a telemarketing call.” “We need to spend a lot of time explaining the nature of the call, the difference between telemarketing and research to establish their trust.” “A lot of the time they think I’m a bill collector until I tell them about [our company].”
Sampling Method • Client lists need to be up-to-date, with full contact information. Many lists have inaccuracies in addresses, telephone numbers and respondent information. • Recruiters sometimes are not warned that people listed may have negative reactions to a call and/or to the client. “There are a lot of wrong numbers. It seems like some lists are old.” "It is great when lists have more than one phone number. You can try them during the day or at night.” “We need to know the type of person on the list so we can calm them down. If they are people who have called to complain, someone with a lousy credit history, they can get angry.”
Contacts Before Calls • Emails, letters by mail or a fax introducing the research company and the study can be a helpful first step in recruiting. • With respondents who know the facility, these steps avoid calls bothering them about a study they are not qualified for or interested in. “It is best that way. They want to be included if they respond to an email.” • People on a client list have heard the research company’s name when they are called, making them more receptive. “If we can, emailing ahead of time really helps. They know the name [of the research company] and they will answer my call.”
B2B and Consumers • Recruiting B2B respondents is generally more challenging than consumers. • Businesspeople and professionals are often more difficult to reach, have less time to be screened or to participate in a study. “For business recruits, it’s more challenging because you’re interfering in their day, because it’s hard to get them on the phone to screen them, and then it’s hard to get them to come in. It’s easier with consumers because word-of-mouth and past participation makes them want to do it again.” “You need to get through, you have to work your way through the people, the receptionist.” • Consumers are easier to recruit than businesspeople, once they are contacted, because of experiences in previous studies or word-of-mouth recommendations. • Characteristics like age and gender do not seem to make a major difference in respondent cooperation but do play a role in show rates. “It’s not that younger/older is easier or harder to recruit, it depends on whether the topic relates to them or not.”
Type of Interview • Most respondents are willing to participate in a range of interview types. • Businesspeople /professionals have divided preferences for depth interviews and focus groups. Depth interviews tend to be shorter and have greater flexibility in scheduling, giving respondents choice of the time. Focus groups give them the opportunity to hear others’ opinions and learn from the interaction. “Sometimes theysay they only want to do groups – especially if the topic is interesting or they get to learn about new products from others.” • Respondent concerns about ethnography home visits were noted by some recruiters. • Even respondents who know/trust the field service can be hesitant about strangers visiting; researchers coming to see how they live seems foreign/“weird” to some. • Respondents want to know details, e.g., who will be visiting, the visit purpose. Securing cooperation is especially difficult when recruiters cannot tell them. Some women will not allow males into their home unless their husband or a friend is present. “They’re like, ‘You want to come in and watch my family eat dinner? I don’t feel comfortable with that.’” “It is the fear factor. They know my company but do not know the person coming. They sometimes ask if I can be there too.” “They are just uncomfortable and it is even worse if more than one person is going. They feel overpowered in their space.”
Type of Interview • Store shopalongs are easier to recruit than home visits but some respondents express concerns that they might be asked to make purchases. “I get asked these days if I have to purchase things. They say they just shopped and do not want to spend more money on things they do not need at the moment.“
Time of Day • Interviews scheduled at times when respondents are busy/unavailable make recruiting farmore difficult. “In the old days, it was what was convenient for the respondent and now it’s what’s convenient for the researcher or the clients.” • Best start times for early evening interviews vary by market. • Traffic patterns and distance to the facility from the highway or downtown area need to be considered. “If a focus group is scheduled any time around rush hour, it is a problem for some to come.” “For respondents who live or work far away from the facility, getting to a 6pm group can be very challenging during rush hour traffic.”
