Working at a firm focused primarily on quantitative data analytics, I often hear the argument that survey work has “poisoned the ecosystem of research” and we should focus our efforts on the power of qualitative approaches. Indeed, poor methodological practice has produced myriad examples to support such a position. But in the current climate of big, small, deep, and thick data, it’s important to consider how our current (and potential) clients might react to calls for methodological change. The pendulum is swinging toward qualitative approaches and far be it from me to discount their effectiveness. However, with this swing, I have also seen both clients and researchers impose arbitrary constraints on survey research in the name of superior practice. But, diluting the survey instrument in the name of speed or simplicity misinterprets what can be so “poisonous” about survey research. Instead, respondent engagement requires engagement. If we want consumers to take time out of their busy schedules to answer our questions, we need to keep it interesting.
Quick and simple is an oversimplification of what makes a survey successful. Survey instruments should be clear and concise, but to fixate on question or time limitations misses the point of sound survey research methodology. These limitations are still prone to poor questions design, vague instructions, or a reliance on claims that inevitably run the risk of respondent satisficing, straight-lining, or random selection. And these response tendencies surface whether a survey instrument requires 15 or 45 minutes. Design can account for some of this through branching, trapping, and duration monitoring. But rather than relying solely on data cleaning, take the time on the front end to develop a better survey experience to engage your target audience.
Respondents will lose interest if the survey experience, aesthetic, and language aren’t appealing. But achieving high levels of respondent engagement requires more. I suppose the saying “garbage in, garbage out” might apply here. Or is it, “slow down to speed up”? It’s probably a little of both. Most private sector clients want results yesterday. And in a consultancy role, we understandably get caught in this frenzy and fail to take the time to thoughtfully consider how to best engage respondents. Can we show respondents how they are helping? Can we make them feel involved in the process by giving them space to differentiate themselves? Can we engage them on a human level? A focus on speed and duration is a focus on the lowest common denominator. As a result, the quality of these data are often poor. Moreover, the resulting respondent voice rarely reflects the perspective of clients’ most attractive, potentially loyal consumers.
For some, finding new and interesting ways to present questions to respondents can often feel like more trouble than it’s worth. But making a survey interesting for respondents and maintaining a commitment to high quality data doesn’t have to be complicated. Much of my survey work is based on choice-based approaches. When properly executed, these approaches engage consumers in real-world scenarios where their preferences matter. More recently, my team has been following standard and choice-based batteries with open-ended questions. These responses not only give consumers a space to explain the choices they make but also provide exciting qualitative depth to our results through a range of computer-assisted qualitative data analysis techniques.
Choice-based approaches and open-ended responses are not new, but they are effective. And in addition to the superior results they can offer, respondents regularly tell us how much more interesting these types of exercises are. In many respects, the respondent feedback we receive mirrors gamification trends that consistently show a positive response to quantitative research that incorporates a goal for those that participate. By taking the time to think outside standard, claimed response sets, we collect better data that meets the needs of our clients. While these approaches require expertise to ensure precise execution, they do not require reinventing the wheel to engage respondents.
Image: Purchased from iStock
About the Author: Jerry Hendricks, PhD is a Senior Research Analyst at RSG: Chicago. A former academic focused on organizations, economy, and research methodology, he has brought his sociological expertise on consumer trends and product innovations to private-sector clients across a range of industries with rapidly changing markets. While focused on new ways to translate the complexities of data and strategic context into actionable insights, he continues to write on topics including market value, authenticity, market collaboration, and organizational identity. RSG has eight offices across the United States and is a national and international leader in designing, implementing, and applying sophisticated data-driven models with in-depth analytics to predict consumer behavior and explain the choices they make.