Basics of Nutrition Research
How do people know things about nutrition?
Questionable nutrition claims have always been around, but in the age of the Internet, they are easier than ever to come across. You name the subject, and there are going to be people out there giving opinions as though they know them to be facts.
Unfortunately, there is a very long list of nutrition questions that do not yet have answers. For example, no one knows whether vegans have higher, lower, or the same rates of cancer as meat-eaters. No one does, because they haven’t been measured.
When I hear a surprising nutrition claim, I ask myself:
- Does this sound too good to be true? If so, it probably isn’t true.
- Does this sound too bad to be true? If so, it probably isn’t true.
- How does the person claiming this know it to be true?
Answering question #3 can save a lot of time. People often just read a popular article or book and then start stating something as though they know it to be true. The information might have been based on test-tube studies (virtually useless for applying to everyday behavior), animal studies (ditto), their own personal experience with their patients (a highly self-selected sample with a lot of bias), or on a handed down tradition. Given the immense number of nutritional factoids and theories making the rounds these days, none of the above forms of knowledge provide enough evidence to cause me much concern.
Self-selection bias occurs when those in the group being studied become part of the study group because they have responded well to treatment. For example, a doctor may put patients on program X. Those who don’t respond to program X stop seeing the doctor and the remaining patients end up with a highly positive response rate. If you only look at the patients who finished treatment, you have a highly self-selected sample and the results will not correlate well to the overall population.
Introduction
The wide range of nutrition claims these days all have some sort of theory to back them up. In this article I discuss what studies provide concrete evidence for a theory. As vegetarians, when we base our claims on concrete evidence, we can prevent others from exaggerating against us while at the same time understanding the facts.
You Don’t Need to be a Biochemist
Before getting into the nuts and bolts of nutrition research, I’d like to point out that answering nutrition questions using only logic (i.e., A decreases B, B causes C, therefore A decreases C) is not enough. Logic can give you ideas as to what might be the case. But you cannot assume that you know all the variables and testing the logic is required.
The unfortunate side of this is that you cannot simply memorize numerous physiological mechanisms that occur in the body, add them together in various ways, and discover truth without first testing it. The body has a staggering number of biochemical pathways all with their own sets of checks and balances, and interactions with other pathways. No one can know them all. This puts people who know biochemistry and physiology on a more even par with those who do not have such knowledge. Everyone’s theories need testing, no matter how much detail they know about the body or food. When those tests are explained well, people without a background in nutrition or biochemistry can understand them.
The issue of iron absorption is an example of relevance to vegetarians. We often hear from doctors, coaches, etc., that you cannot get iron on a vegetarian diet. Iron is an important nutrient for optimal health, and it is not absorbed as well from plant foods as from meat. Since vegetarians do not eat meat, logic would tell us that they must not be as healthy as those who eat meat, right? Not necessarily. Lower iron levels may reduce insulin sensitivity (a risk factor for diabetes) and higher iron levels are associated with some cancers. Knowing this, can we then assume that vegetarians have lower rates of diabetes and cancer? Not necessarily. To know if vegetarians are healthier than meat-eaters, we have to compare their disease rates. Everything else is just guesswork.
So, do not let fancy biochemical explanations fool you. When you hear something new about nutrition, consider what studies have been performed looking directly at the actual outcomes.
Types of Nutrition Studies
Obtaining conclusive results about the impact of eating patterns on chronic diseases, which take a long time to develop, is a complicated venture consisting of many types of information. It often takes numerous studies to verify whether a food, or component of food, is likely to impact the risk of disease.
Studies often focus on only one disease. To really know the impact of a particular eating habit, we need to consider the impacts on the risk of all diseases (and possibly the quality of living, too, though this is much harder to test). Naturally, bits and pieces of information trickle out to the public. Which of these should be taken seriously?
Nutrition studies can be separated into two general categories:
- Those that show what areas of research are worth more time and money but do not serve as concrete evidence themselves.
- Those that serve as more concrete evidence.
Confusion often occurs when results of only a few studies are reported. This is especially true when those studies are not even the type that provide concrete evidence. The following is a brief description of the various types of nutrition studies.
| Table 1: Summary of Study Conclusiveness |
| Not Conclusive - provide info about what variables to study further
In vitro |
| More Evidence-Building
Retrospective (case-control) |
| Most Conclusive
Meta-Analysis of Prospective or RCTs |
In vitro
In vitro studies examine, outside of the body, interactions between food components and cells or other tissues. What happens outside the body can be much different than what happens when food is eaten, digested, and transported to various parts of the body. The food or body may have factors that counteract positive or negative aspects of a food which do not show up in vitro.
Animal
Humans differ from other species in physiology, psychology, size, and lifestyle. That makes the affects of eating on animals too unreliable to guide recommendations for humans. The dietetics profession does not generally, if ever, base dietary recommendations regarding food on in vitro or animal studies. Normal foods that have been used for many years are generally recognized as safe until proven otherwise through studies using humans.
An exception is testing synthetic food additives (i.e., not whole foods) for their cancer-causing potential. The assumption is that if a new substance causes cancer in any organism, it is not worth risking in humans. Some food additives have been banned based on animal studies, though doing so has been controversial.
Beware of those whose nutritional recommendations are based on in vitro and/or animal studies.
Case Study
A case study is when one person’s history, characteristics, and disease outcome is published in a scientific journal. When case studies are not published in a scientific journal, they are considered merely anecdotes. Conflicting anecdotes are widespread and one reason why formal research methods are necessary.
Ecologic (aka Regional)
Ecological studies compare food consumption data and disease rates of groups of people in one geographical region to those in another, or in the same geographical region over time. These studies generate hypotheses that can then be tested by looking at individuals’ habits, rather than the entire group’s.
Migrational studies are ecologic studies that look at what happens to a group of people when they move to a different location and develop new eating patterns (and other lifestyle changes). This gives clues as to whether their diseases are primarily genetically-based.
Ecologic studies are riddled with problems. People’s environment and behavior from region to region often changes along with their diet. It is also possible that a region consumes more or less of a particular food, but the people who get a disease actually do the opposite of the norm for that region.
Disease Markers
Many studies measure markers for a disease, not the actual incidence of the disease. An example would be a study measuring a food’s impact on cholesterol levels rather than on heart attacks. A particular eating pattern can lead to lower total cholesterol, but could increase the “bad” versus “good” cholesterol ratio, or increase triglyceride levels (also thought to be a risk factor for heart disease). Unless you actually measure the disease outcome, the results remain at least somewhat inconclusive.
Retrospective (normally Case-Control)
Retrospective studies find people with a disease and compare their past eating habits to those who do not have the disease. These are normally case-control studies, because a certain number of people already with the disease (cases) are chosen and compared to a similar number of people without the disease (controls).
Case-control studies are relatively inexpensive. No time is required for a long follow-up period while waiting for people to develop a disease. They allow you to study diseases whose rates in the normal population are too small for prospective studies (described below).
The drawbacks are that people’s memories of their previous diet are often influenced by contracting a disease, and the controls may be more health conscious (reflected in their willingness to take part in a study), providing a difference in habits that could falsely appear to be influencing the disease.
Cross-sectional
Cross-sectional studies look at eating patterns and diseases at one moment in time. They are often case-control studies. They can be biased because the cases, especially, might have recently changed their diets due to their current illness.
Prospective (aka Cohort)
Prospective studies follow large numbers of people who are (usually) healthy when the study begins. Diets are assessed at the beginning, and sometimes again during the study to ensure they haven’t changed. As the population is followed, eating patterns of those who eventually get a disease are compared to those who did not.
One benefit of cohort studies is that the participants eat normally, making the results somewhat applicable to normal life.
To be effective, cohort studies must be large and of long duration so that some people will acquire the diseases being studied. They are rarely large enough to study diseases that occur at low rates. The results of cohort studies show associations, but not necessarily causation. Cohort studies can produce misleading results when large numbers of people take the same steps to improve their health. If some of those behaviors lower the risk for disease, all of the behaviors will appear to lower the risk. Another problem works the opposite way: People who know they are at risk for a disease may do things they think will prevent the disease. If they get the disease anyway, those behaviors will appear to be associated with the disease. All these variables should be considered.
Nested Case-Control
Sometimes a case-control study is conducted using a subset of participants from a prospective study. All the people in the study who contract a disease are compared only to a similar number of others in the study who did not get the disease (rather than to everyone in the prospective study who did not get the disease). This frees the researchers from analyzing the diet or blood samples of every person who entered the study. Nested case-controls are not subject to the diet recall bias that exists in other case-control studies.
Randomized Controlled Trial (RCT)
A RCT randomly places people either into a group that is instructed to follow a specified eating pattern or a control group. Outcomes are then compared.
When possible, RCTs should be double blinded to reduce bias. That means that neither the researchers nor the participants know who is in which group. It is important to make things equal among groups. For example, if only one group gets diet instruction, the diet instruction could serve as a placebo for that group only.
The variables can be controlled far better than in cohort studies and can give more insight into cause and effect. Drawbacks are that participants might not follow the instructions and the study might not last long enough to discern a difference between groups.
Review
A review is a survey of the scientific literature on a subject in an attempt to gain some sort of conclusion. They typically do not apply any statistical methods for quantifying the results of the various studies. Reviews can range from extremely thorough to not thorough at all, and are hard to judge on their own.
Meta-Analysis
A meta-analysis surveys the scientific literature and normally applies some sort of method to quantify the body of research. The methods can range from merely providing a chart of study results for easy comparison, to collecting the actual unpublished data from each study and performing a new analysis with the entire data set.
Table 1 summarizes the various studies.
Other Issues in Nutrition Research
Disease Treatment
A successful treatment for a disease might or might not be able to prevent that disease. It is certainly worth considering, but they are two different phenomena and should be recognized as such. An extreme example in which a treatment does not prevent the disease is kidney failure. The diet for someone whose kidneys no longer work (i.e., on kidney dialysis) is high in protein and low in fruits and vegetables. This is the opposite of a good diet for preventing kidney failure.
General Problems Analyzing Eating Patterns
A general barrier to learning about nutrition is the difficulty in determining what people actually eat. The most common methods are:
- Duplicate Portions – Participants set aside equal portions of the foods they are eating for nutrient analysis. This is very expensive, and is rarely done.
- Food Diaries – Participants write down everything they eat for a few days. Analyzing food diaries is a long process, and normally not used for large studies.
- Food Frequency Questionnaires – Participants are asked how often they typically eat each food on a list.
If duplicate portions are not analyzed for nutrient content, foods will be looked up in a nutrient database to determine the levels of various nutrients. This can result in some error because the researchers must match the foods eaten to the foods in the database, not always an easy job. This method also relies on the accuracy of the database which might not be valid for all foods.
When you add up all the potential for error, you can see that researchers are often playing with general trends rather than specific numbers; especially in large studies.
Studying certain groups that do not eat any of a certain food, such as vegans, can provide some benefits for research given that you can be fairly certain that the amounts of food consumed in some categories is zero.
No Findings
It should be expected that with any given subject, some studies will not find the same correlations as found in others. One reason is that in some populations, the eating patterns do not vary enough to produce a correlation. Small sample sizes can also prevent finding a true association.
Researcher Bias
It is always possible that a particular researcher or research group may be biased. They may be convinced that an earlier study they performed was correct and want to provide further proof of it. They may receive funding from a company or organization that monetarily benefits if their results point to a particular conclusion. Generally, I believe that personal biases impact scientific research less than they impact the nutritional ideas in more popular circles.
Publishing Bias
There may be a bias in the scientific literature against studies that find no correlation between variables. Some evidence shows that researchers are less likely to submit studies with negative findings and that journals may delay publishing such studies.
Adjusting Results to Control for Variables
Unless very little is known about a subject, the results will be adjusted for different variables thought to impact the outcome. For example, in a study of fat and obesity, total calories might impact the results, so the results will be adjusted for differences in total calories. Usually, pre-adjustment and post-adjustment results are reported. Because a finding disappears upon adjusting does not necessarily mean a result is invalid.
Statistical Significance
If someone were to do a study looking at 1,000 people and found that the 20 people who got cancer also ate more avocados, then that would mean that avocados cause cancer, right? Not so fast. In any group of people who get cancer, there will be a range of avocado intake. Purely by chance, those who get cancer might eat more avocados than those who did not get cancer. In order to determine whether a finding is due to a true effect or merely random chance, statistical methods are applied. Generally, it is agreed upon that a finding that has less than a 5% chance of being due to random chance is considered statistically significant (SS).
Conclusion
The main message to take home from this introduction to nutrition research is that in order to know something about nutrition with certainty, that thing must be studied directly and thoroughly. Indirect approaches, while often shedding light on a subject, lead merely to educated guesses.
Applying to Vegetarian Diets
Now that you know the basics of nutrition research, you know that to understand the health of vegetarians, you must study the disease rates of actual vegetarians compared to non-vegetarians. Once familiar with that research, you can talk about the diet with authority, and steer clear of traps set by those who argue that it is an unhealthy diet based on a few potential drawbacks. You might be interested in the article, Disease Rates of Vegetarians and Vegans.