The Problem with Epidemiological Studies

pie chart made of people to represent epidemiologyThere are many different types of studies nutrition researchers can conduct to try to understand the world around us, and the epidemiological study is the weakest, least reliable method available. Unfortunately, it is the most common approach used in the field of nutrition.

Here’s an example: When giving nutrition presentations, I am fond of pointing out that there is no proof that vegetables are necessary or healthy. Inevitably, someone asks how I can say this, when there are so many studies out there proving that vegetables are healthy. Now, if someone asked you to design a simple experiment that could tell us whether or not vegetables are healthy, what would you do?

When I asked a 15 year old high school science student this question, here’s what he said: “I would have one person eat vegetables and the other person not eat vegetables and see who was healthier.” Brilliant! Now, of course, he’s 15 and hasn’t studied research methodology, so he doesn’t realize you’d need more than two people to do this study, but other than that, he’s right on the money.

Yes, there have been thousands of studies looking at vegetables and health, but I was not able to find a single experiment that compares one group of people that eats vegetables to another group of people that doesn’t eat vegetables. Worse still, the vast majority of studies looking at vegetables and health are epidemiological studies, and epidemiological studies are not experiments at all.

An epidemiological study is not an experiment!

That’s right—here’s what epidemiologists do. They pick a health problem and try to figure out why some people are more likely to develop that health problem than other people. Let’s take heart disease, for example. The well-known “French Paradox” is a good example. Years ago, it appeared as if French people were less likely to get heart disease than Americans, so epidemiologists compared the lifestyle of the French to the lifestyle of the Americans, and tried to guess what it was about the French lifestyle that was better for the heart.

Of the many differences between the American lifestyle and the French lifestyle they could have chosen (the fact that the French may have eaten more whole foods and less junk food, for example), the one they decided to focus on was red wine. Why red wine? The choice seems kind of random to me. In fact, it may have been even worse than random—the choice may have been biased by the wishes and beliefs of the researchers. After all, wouldn't it be wonderful if the answer to the American heart disease problem turned out to be a simple red wine deficit?

So, epidemiologists noticed that the French seemed to drink more red wine than Americans, and since Americans had more heart disease than the French, they reported in their studies that drinking red wine was associated with a reduced risk for heart disease. This word “associated” is VERY important. Note that they do not, and cannot say, that red wine lowers risk for heart disease. To say this, they would need to conduct an experiment. An epidemiological study can be useful in generating HYPOTHESES (educated guesses) about risk factors for diseases, but it cannot prove cause and effect.

Unfortunately, most reporters (and even many physicians) do not understand this very important limitation of epidemiological studies. Therefore, when reporters write scary headlines like “Red Meat Increases Risk for Death” or promising headlines like “Fruits and Vegetables Reduce Cancer Risk”, they do not realize that these dramatic claims are based on educated guesses that haven’t been scientifically proven yet.

Let's say you were trying to understand why some people become alcoholics while others don't. You interview 10,000 alcoholics and 10,000 non-alcoholics by giving them questionnaires about their daily habits. These questions are based on the researchers' beliefs about what might cause alcoholism in the first place. Note that it is impossible to ask about a risk factor you haven't thought of. A standard type of question would look like this: "How often have you eaten pretzels in the past two years?" If you find that alcoholics reported eating significantly more pretzels over the past two years than the non-alcoholics, the next day the following headline might appear in the Huffington Post: "Eating Pretzels Increases Risk of Alcoholism." The story that follows the headline would advise people to eat fewer pretzels to reduce their risk of alcoholism. Absurd.

epidemiology headline - cause or coincidence

Epidemiological studies, at their best, can only point out a possible connection between two things, but that is only the very first step in trying to figure out whether or not there really is a connection between them:

"Epidemiology is concerned with the incidence of disease in populations and does not address the question of the cause of an individual's disease. This question, sometimes referred to as specific causation, is beyond the domain of the science of epidemiology.”

There is a beautifully written essay about Dr. Walter Willett, the father of nutritional epidemiology, and the shortcomings of the field, on Adele Hite's wonderful site:

If you'd like to read one of my critiques of specific epidemiological studies, take a look at my posts: "New Dietary Guidelines Hazardous to Your Health?," "EAT-Lancet's Plant-Based Planet: 10 Things You Need to Know,"  and "Latest Low-Carb Study: All Politics, No Science."

Go back