New Hope Network is part of the Informa Markets Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


How to avoid common pitfalls of surveying

Jennifer Barney
Follow these tips when surveying consumers.

You can really learn a lot from surveying consumers. You can also spend a lot of money and not learn anything actionable. 
Conducting consumer research has traditionally been the job of consumer insights. These researchers are not marketers—they are the voice of the consumer. They tell marketers what consumers think and how they will behave so marketers can make decisions.
Lately, brands are doing their own consumer insights. Using DIY tools from platforms like PickFu, 1Q, Suzy and others, you can survey consumers fairly inexpensively. What these platforms offer are guaranteed respondents (member audiences) and real-time results. For an extra fee they offer programming (launching on the tool).
But what these DIY platforms don’t do is understand your objectives or develop the right questions to ask.
I’ve seen this go way wrong too many times—brands spending on surveys and coming up empty on insights. Problems usually stem from:

  • Not having a research objective at the outset.
  • Using the wrong methodology. 
  • Making inferences from insignificant data.

 Take the following steps to avoid pitfalls.

1. Determine the research objectives.

This is the biggest input when determining methodology. 

Examples of research objectives:

  • Product naming and packaging.
  • Purchase interest. 
  • Which consumer target will most benefit.
  • What need and occasion best fit the product solution.

2. Use the proper methodology.

A survey may not be the right method! It might be interviews with a few select respondents to understand what even are the questions to ask.

Qualitative: For example, if you want to know what different groups of people think of your product, it’s better not to guess at a bunch of questions and throw them into a survey. It’s better to conduct focus groups. This is called qualitative research and this type of research is verbal and in-depth. It allows for open discussion and investigates the reasons behind behaviors and beliefs. Once you’ve identified themes and patterns from a few small groups, you form specific questions to ask a wider audience in a survey.

Quantitative: Surveys are quantitative—they collect numerical data from a number of respondents that can be extrapolated to a larger population. 

3. Make it statistically significant.

You only need to have passed Intro to Statistics to understand that the larger the sample size the smaller the margin of error. For example, a sample size of 1,000 has a margin of error of 3%, a sample size of 100 has a margin of error of 10% and a sample size of 50 has a margin of error of 14%.

  • Start with screener questions to bring the proper amount of people into the survey, and think about all the different ways you’ll want to cut the data after the results come in. 
  • What often happens is brands field a survey without predicting the incidence rate of the segment breakouts they care about, only to find there weren’t enough respondents to get good data. 
  • Example: A plantain chip brand marketing to gluten free people conducts survey to 500 general population respondents. The results:
    • Grocery shopper decision makers = 70% (350)
    • Chip shoppers 90% (315)
    • Gluten-free chip shoppers = 10% (32) – not statistically significant

To avoid wasting time, energy and money on statistically insignificant, a.k.a. useless results, start with screener questions to bring the proper amount of people into the survey. Then think about all the different ways you’ll want to cut the data after the results come in.
We already have an email list and past purchasers list—can we use them to conduct surveys?
You can absolutely survey from your own lists!

How to use your own audience list in surveys

I once had a brand go to launch on the basis that 70% of their Facebook followers responded positively to a new product concept. First of all, 70% is not impressive considering your FB followers are your most engaged, loyal customers.

But seriously, this is not good data (regardless of the percentage) to make a go/no-go decision on launch. It does not represent what shoppers, most of whom have no recognition of your brand, will do. 

How well a product will do in market can only be tested accurately with general audiences that have no bias. The way to find those audiences is through insights companies that recruit respondents for a fee or DIY platforms that have member audiences as described above.

However, here is when you can, and should, use your own consumer lists:

  • When considering a brand refresh or rebrand.
  • Understanding if your product is meeting expectations.
  • Interest in new flavors or packaging.
  • Brand stretch (limitations or opportunities) into new categories.

Mostly what you are looking for with this kind of testing is guidance or warnings. I’m not sure Colgate followed this advice back in the ’80s.


Yup. Frozen entrees by Colgate. If only they had asked.

The reason why results should be taken only as guidance is that each research objective will have different response bias when coming from your audience list. For example, when presenting a potential new brand name, all you are really looking for is whether or not it alienates or confuses your existing audience.

Alternatively, when asking about new products, you should expect some overinflated level of interest. This is natural because these folks already know you, love you and will be happy to see more of you. Use this questioning as more of a ranking exercise to understand line extension prioritization.

These surveys should be fun and easy and not be too demanding (read: long) on your fans. The biggest challenge will be getting people to opt in. You will need to offer incentives, and even then, plan for maybe a 10% response rate.

The best incentives, and in this order, are:

  1. Cold hard cash.
  2. Free product.
  3. Deep coupon discount on product (40% or more).
  4. Entry into a giveaway.

Happy surveying!

Jennifer Barney is an advisor and consultant to food and ag businesses at 3rd & Broadway. She previously founded the almond butter brand Barney Butter.

Have some big ideas or thoughts to share related to the natural products industry? We’d love to hear and publish your opinions in the IdeaXchange. Check out our submission guidelines.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.