Not all science is good science

Not all science is good science

Even a small misstep during an ingredient supplier's study can put it -- and its customers -- at regulatory risk.

You sell a finished dietary supplement. You see an ingredient featured on "Dr. Oz," where he discusses the great science behind it. You meet with the ingredient supplier at Engredea, and they provide you with the study they say supports their claims. You review the report from the study. Perhaps it is published; perhaps it is not. Perhaps the supplier gave you a full study report, or perhaps just a summary.

From your review the study has all the "bells and whistles" that the FTC expects in a supplement study: It is double blinded. It is randomized. It is placebo controlled. You see that magic number -- p < 0.05 -- indicating that the results comparing the test ingredient to the placebo are statistically significant. The sample size looks pretty good for these types of studies, and the subject population looks like your target audience. You aren't an expert in clinical trials or statistics, but everything seems to check out, and it sure looks a lot better than the studies on the three other ingredients you were considering. Plus, the price is right.

Do you go ahead and put in your purchase order, or should you do more digging?

What if the study used the wrong statistical analysis? What if the data on a few of the subjects was accidently input incorrectly? What if study procedures were not followed exactly right? What if the data was actually fabricated?

If any of those things had happened, not only could you be selling an ineffective product to your valued customers, but you could also be subject to FTC enforcement. The FTC does not need to show that you intended to mislead consumers -- only that you did not, in fact, have adequate substantiation to support your claims.

A supplier of green coffee extract recently paid $3.5 million to the FTC. The company did have a study to support its weight loss claims. However, the FTC charged that the study was "seriously flawed." Among other things, the FTC charged that “the study's lead investigator repeatedly altered the weights and other key measurements of the subjects, changed the length of the trial and misstated which subjects were taking the placebo or GCA during the trial.”

In addition to the fine and future marketing restrictions, the company "must notify trade customers of the FTC's conclusion that the company lacked reasonable scientific support for the weight-loss and fat-loss claims it made."

That, of course, puts the companies that made claims for their products with that ingredient in a difficult situation, and potentially subjects them to liability. The FTC was able to discover the issues through its investigative powers. More and more, we are finding that in an investigation the FTC is now routinely requesting raw data and other information from studies to look for issues, and is scrutinizing everything about a study, including statistical analyses. But as a supplement distributer, you do not have authority to compel an ingredient company to provide you with anything. Of course, you can always ask.

But what do you ask for? How far do you go? Do you have the study report reviewed by an independent expert? Do you ask for the raw data and have your own statistics done? Do you audit the study site? Do you do your own study?

Does it depend on whether the study was published in a peer-reviewed publication? Does it matter if the study was paid for by the company? Does the reputation of the ingredient supplier matter? Does it matter if the study was done in the US or in another country?

These are questions that our clients and responsible marketers (and retailers) everywhere are struggling with, and there are no easy answers. "Due diligence" measures like these can be time consuming and expensive, but they may also provide comfort that the ingredient does what it is supposed to and help defend any challenges down the road.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.