As we discussed in the previous section, science is always evolving, albeit sometimes slowly. One study is not enough to make a guideline or recommendation or to cure a disease. Science is a stepwise process that continuously builds on past evidence and develops towards a well-accepted consensus, although even that can be questioned as new evidence emerges. Unfortunately, the way scientific findings are communicated to the general public can sometimes be inaccurate or confusing. In today’s world, where instant Internet access is just a click away, it’s easy to be overwhelmed or misled if you don’t know where to go for reliable nutrition information. Therefore, it’s important to know how to find accurate sources of nutrition information and how to interpret nutrition-related stories when you see them.

Deciphering Nutrition Information

            “New study shows that margarine contributes to arterial plaque.”

            “Asian study reveals daity coffee can have detrimental effects on the nervous system.”

How do you react when you read headlines like this? Do you boycott margarine and coffee? When reading nutrition-related claims, articles, websites, or advertisements, always remember that one study neither proves or disproves anything. Readers who may be looking for answers to complex nutrition questions can quickly misconstrue such statements and be led down a path of misinformation, especially if the information is coming from a source that isn’t credible. Listed below are ways that you can develop a discerning eye when reading news highlighting nutrition science and research.

• The scientific study under discussion should be published in a peer-reviewed journal such as the Journal of the Acaedmy of Nutrition and Dietetics. Nutrition research is also frequently published in journals like the Journal of Nutrition, American Journal of Clinical Nutrition, and Journal of Nutrition Education and Behavior, as well as medical and behavioral journals. Having gone through the peer review process, these studies have been checked by other experts in the field to ensure that their methods and analysis were rigorous and appropriate. Peer-reviewed articles also include a review of previous research findings on the topic of study and examine how their current findings relate to, support, or are in contrast to previous research. Question studies that come from blogs, popular magazines, etc.

• The report should disclose the methods used by the researcher(s).
  • Identify the type of study and where it sits on the hierarchy of evidence. Keep in mind that a study in humans is likely more meaningful than one that’s in animals. Systematic reviews and meta-analyses often give you the best synthesis of the science to date.
  • If it’s an intervention study, check for some of the attributes of high-quality research already discussed: randomization, placebo control, and blinding. If it’s missing any of those, what questions does that raise for you?
  • Did the study last for three weeks or three years? Depending on the research question, studies that are short may not be long enough to establish a true relationship with the issues being examined.
  • Were there nine or nine hundred participants? If the study was conducted on only a few participants, it’s less likely that the results would be true for a larger population.
  • What did the participants actually do? It’s important to know if the study included conditions that people rarely experience or if the conditions replicated real-life scenarios. For example, a study that claims to find a health benefit of drinking tea but required participants to drink 15 cups per day may have little relevance in the real world.
  • Did the researcher(s) observe the results themselves, or did they rely on self reports from program participants? Self-reported data and results can be easily skewed by participants.
• The article should include details on the subjects (or participants) in the study. Did the study include humans or animals? If humans, are any traits/characteristics noted? You may realize you have more in common with certain study participants and can use that as a basis to gauge if the study applies to you.
• Statistical significance is not the same as real-world significance. A statistically significant result is likely to have not occurred by chance, but rather to be a real difference.  For example, imagine a study reporting that a new vitamin supplement causes a statistically significant reduction in the duration of the common cold. Colds can be miserable, so that sounds great, right? But what if you look closer and see that the supplement only shortened study subjects’ colds by half a day? You might decide that it isn’t worth taking a supplement just to shorten a cold by half a day. In other words, it’s not a real-world benefit to you.
• Credible reports should disseminate new findings in the context of previous research. A single study on its own gives you very limited information, but if twenty other stufies found the same results,  it adds credibility to the study.
• When reading such news, ask yourself, “Is this making sense?” Even if coffee does adversely affect the nervous system, do you drink enough of it to see any negative effects? Remember, if a headline professes a new remedy for a nutrition-related topic, it may well be a research-supported piece of news, but it could also be a sensationalized story designed to catch the attention of an unsuspecting consumer. Track down the original journal article to see if it really supports the conclusions being drawn in the news report.

The CRAAPP Test

While there is a wealth of information about nutrition on the internet and in books and magazines, it can be challenging to separate the accurate information from the hype and half-truths. You can use the CRAAPP Test^1,2 to help you determine the validity of the resources you encounter and the information they provide. By applying the following principles, you can be confident that the information is credible. We’ve added several notes to the traditional CRAAPP Test to help you expand your analysis³ and apply it to nutrition information.

Table 2.1. The CRAAPP Test is a six-letter mnemonic device for evaluating the credibility and validity of information found through various sources, including websites and social media channels. The CRAAPP Test can be particularly useful in evaluating nutrition related news and articles.

Red Flags of Junk Science

When it comes to nutrition advice, the adage holds true that “if it sounds too good to be true, it probably is.” There are several tell-tale signs of junk science—untested or unproven claims or ideas usually meant to push an agenda or promote special interests. In addition to using the CRAAPP Test to decipher nutrition information, you can also use these simple guidelines to spot red flags of junk science. When you see one or more of these red flags in an article or resource, it’s safe to say you should at least take the information with a grain of salt, if not avoid it altogether.

With the mass quantities of nutrition articles and stories circulating in media outlets each week, it’s easy to feel overwhelmed and unsure of what to believe. But by using the tips outlined above, you’ll be armed with the tools needed to decipher every story you read and decide for yourself how it applies to your own nutrition and health goals.

Attributions:

• Lindshield, B. L. Kansas State University Human Nutrition (FNDH 400) Flexbook. goo.gl/vOAnR, CC BY-NC-SA 4.0
• University of Hawai‘i at Mānoa Food Science and Human Nutrition Program, “Types of Scientific Studies,” CC BY-NC 4.0
• “Principles of Nutrition Textbook, Second Edition” by University System of Georgia is licensed under CC BY-NC-SA 4.0

References:

• ¹Blakeslee, Sarah (2004). The CRAAP Test, LOEX Quarterly, 31(3)6-7. https://commons.emich.edu/loexquarterly/vol31/iss3/4
• ²Lumen Learning. (n.d.) The CRAAPP Test. Introduction to College Research. https://courses.lumenlearning.com/atd-fscj-introtoresearch/chapter/the-craapp-test/
• ³Fielding, J.A. (2019). Rethinking CRAAP: Getting students thinking like fact-checkers in evaluating web sources. C&RL News, December: 620-622.

Image Credits:

• Real experts book photo by Rita Morais on Unsplash (license information)
• Figure 2.7. An example of a peer-reviewed journal photo by Heather Leonard is licensed under CC BY 4.
• Table 2.1. The CRAAPP Test by Heather Leonard is licensed under CC BY 4.
• Figure 2.8. “The Red Flags of Junk Science” by Heather Leonard is licensed under CC BY 4.