Editorial: Standards of Evidence & Research

By Len Monheit

This past week, a ‘Stop-Sale’ order was issued by Health Canada for kava products. Also this week, a JAMA (Journal of the American Medical Association) article received wide media attention and suggested that ginkgo did not increase mental function in normal, healthy older populations. Despite proactive steps by US industry associations such as ABC, NNFA and AHPA, the media coverage was not supportive of the industry or past research and studies into the efficacy of gingko.

‘Standards of Evidence’ is a common element in both instances and a subject under discussion in Canada as Natural Health Product regulations and guidance documents are issued. In Canada, the dialogue deals specifically with evidence required in order to make claims for natural health products. Dealing with the issue on a broader perspective would include the admissibility and acceptance criteria for research and clinical studies.

“Impressive evidence in the scientific and clinical literature supports the benefits and safety of the popular herb ginkgo, according to the nonprofit American Botanical Council (ABC)”, yet the JAMA article and media coverage did not convey this. There are numerous examples where historical use and research has been questioned by the medical community, the FDA and the media. There are examples of inadequate or improperly designed research and studies. (The St. Johns’s Wart study in April which suggested that neither St. John’s wort nor the antidepressant Zoloft® (sertraline) were more effective overall than placebo in treating severe forms of depression, which received wide media attention, is an example of a questionable, much criticized and potentially seriously flawed study that was supported and administered by the NIH Center for Alternative and Complementary Medicine.) The industry has also been victimized by unscrupulous marketers making claims about study results which have seriously harmed industry and product credibility.

Part of this week’s Health Canada notice goes on to say, “Health Canada reviewed the information currently available and determined that, at this time, the department does not have sufficient evidence to support the safe use of kava-containing products. For example, there is insufficient evidence to determine:

a safe dosage below which liver toxicity is not likely to occur; the way in which kava products are broken down by the body; populations at risk of kava-associated liver problems.

The Stop Sale notice states an “unacceptable potential risk to health”.

The concept of risk implies that the scales would be weighted differently if there was solid, accepted clinical research to support product use, leading back to the concept of standards of evidence and what constitutes acceptable standards.

We have seen movement by industry in the development of programs for quality in ingredients and manufactured products. We have seen discussions regarding the establishment of test methods. We have seen a more direct and concerted effort on the part of industry associations to proactively present research and data to the media and to consumers, and obviously a more collaborative approach. The DSEA has initiated a “Research Outreach Fund” to present emerging research, which is fine, providing the standards under which the research is conducted, and the criteria and measurements are appropriate. And here, we have internal and external forces to deal with. Internally, we need a consensus on the guidelines for conducting the research and study, and standards for result reporting. However, since a large part of the interpretation will be external, we need to understand and deal with these expectations in the presentation as well.

It is obvious that the issues are international, and therefore demand international collaboration and communication. Is also becoming apparent that a self-policing or self-administering mechanism is required, almost a peer review process to show the regulatory agencies and others that the industry (and all of its sub-sectors) is willing to take initiative, and is moving in the right direction based on today’s expectations. Whether we like it or not there is a set of both expectations and convictions that we need to deal with.

Someone once told me that it is a waste of time to challenge someone’s convictions; all you can do is develop a process and evidence that shows the fallacy of the convictions and begins to erode them. Then, like a house of cards, the convictions eventually tumble down.

And we all know some of the convictions we face—the industry is not regulated, the industry is divided, product quality is suspect, research studies are flawed, unscrupulous marketers dominate the industry and are out to make a few bucks, and so many more.

As recently as a few weeks ago, a supplier in the industry was asked for a lower priced material by his customer. The supplier replied that there were two ways in which he could reduce his finished product cost, first, use less of the material, and second, yes he could develop a product that was less expensive. In either case, finished product efficacy would be reduced and all the research would be invalid since the research was conducted on an efficacious ingredient. The customer was unconcerned, declared his intention to cite or refer to the research in his materials and may in fact be knowingly bringing this ineffective product to market in upcoming months, piggybacking on the valid research. Consumers therefore, will either get confused or abused.

So long as this type of situation can occur in our industry, we will face the challenges we presently deal with. It’s like the development of a quality system that only requires you document what you do and do what you document—one dimensional. If you document and follow a flawed process, you will only produce a consistently poor product, it can’t get better than that. Or put another way, in data terminology, garbage in, garbage out.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.