In most of the quantitative research presentations MDRG gives to clients, we hurry through the set-up slides (Research Objectives, Methodology, and Sample) because we know those slides seem boring to clients. It’s a compliment to us, really, because they trust we know what we’re doing. And, of course, we do.
Those slides are really important to us, though, because they are the foundation of the project. One slide, in particular, Sample, has always been the most important slide to me. Quantitative research is only as good as the sample. Let that sink in for a minute. The research is only as good as the sample of people who are representing the attitudes and behaviors of your customers. We cannot confidently draw conclusions about marketing campaigns or brands or new product ideas unless we believe we are surveying a representative sample of the target population.
When I started in the business of quantitative research, the best way to reach a representative sample (no matter the client) was to purchase a list of landline telephone numbers. Even “low incidence” populations could be found on a landline phone. For example, we could find primary grocery shoppers (easy), who live in small towns in the South (no problem), who are non-rejecters of our client’s CPG product (fine), AND who are parents of children too young for school (got it). Everyone had a phone, and, if the budget allowed, we would interview them. When I say “everyone” had a phone, I mean 98% of the US (according to the 2000 census) so we simply did not worry about having a sufficient or representative sample.
As the 2000s rolled along, however, we did start to pick up tools that required respondents to see something. We may have wanted them to see marketing materials, evaluate CPGs “on a shelf,” rank a list of items, or trade-off product features. I’m sure you see (wink) where this is going…yes, online surveys!
When we first acquired the tools that require a visual experience for respondents, we were faced with the challenge of explaining to clients “why online?” We still have a document on our server titled “Why Online.”
Our new challenge is managing the sample. In the days of phone sample, the list you purchased was representative of the target population. You didn’t even ask respondents where they lived because you knew that from their phone number. In the online world, sample providers invite respondents to the survey who they think represent the target, and then MDRG has the task of setting quotas to ensure a representative sample. This is the behind-the-scenes work that clients don’t see, but that is critical to our being able to say…this report represents your customers in every way — geographically, demographically, and when necessary, behaviorally.