The Problem With Epidemiological Studies

There are many different types of studies researchers can conduct to try to understand the world around us, and the epidemiological study is the weakest, least reliable method available. Unfortunately, it is the most common approach used in the field of nutrition.

Here’s an example. When giving nutrition presentations, I am fond of pointing out that there is no proof that vegetables are necessary or healthy. Inevitably, someone asks how I can say this, when there are so many studies out there proving that vegetables are healthy. Now, if someone asked you to design a simple experiment that could tell us whether or not vegetables are healthy, what would you do?

When I asked a 15 year old high school science student this question, here’s what he said: “I would have one person eat vegetables and the other person not eat vegetables and see who was healthier.” Brilliant! Now, of course, he’s 15 and hasn’t studied research methodology, so he doesn’t realize you’d need more than two people to do this study, but other than that, he’s right on the money. Yes, there have been thousands of studies looking at vegetables and health, but I was not able to find a single experiment that compares one group of people who eats vegetables to another group of people who doesn’t eat vegetables. Worse still, the vast majority of studies looking at vegetables and health are epidemiological studies, and epidemiological studies are not experiments at all.

That’s right: an epidemiological study is not an experiment!  Here’s what epidemiologists do. They pick a health problem and try to figure out why some people are more likely to develop that health problem than other people. Let’s take heart disease, for example. The well-known “French Paradox” is a good example. Years ago, it appeared as if French people were less likely to get heart disease then Americans, so epidemiologists compared the lifestyle of the French to the lifestyle of the Americans, and tried to guess what it was about the French lifestyle that was better for the heart. Of the many differences between the American lifestyle and the French lifestyle they could have chosen (the fact that the French may have eaten more whole foods and less junk food, for example), the one they decided to focus on was red wine. Why red wine? The choice seems kind of random to me. In fact, it may have been even worse than random—the choice may have been biased by the wishes and beliefs of the researchers. After all, wouldn’t it be wonderful if the answer to the American heart disease problem turned out to be a simple red wine deficit?

So, epidemiologists noticed that the French seemed to drink more red wine than Americans, and since Americans had more heart disease than the French, they reported in their studies that drinking red wine was associated with a reduced risk for heart disease. This word “associated” is VERY important. Note that they do not, and cannot say, that red wine lowers risk for heart disease. To say this, they would need to conduct an experiment. An epidemiological study can be useful in generating HYPOTHESES (educated guesses) about risk factors for diseases, but it cannot prove cause and effect. Unfortunately, most reporters (and even many physicians) do not understand this very important limitation of epidemiological studies. Therefore, when reporters write scary headlines like “Red Meat Increases Risk for Death” or promising headlines like “Fruits and Vegetables Reduce Cancer Risk”, they do not realize that these dramatic claims are based on educated guesses that haven’t been scientifically proven yet.

Let’s say you were trying to understand why some people become alcoholics while others don’t.  You interview 10,000 alcoholics and 10,000 non-alcoholics by giving them questionnaires about their daily habits.  These questions are based on the researchers’ beliefs about what might cause alcoholism in the first place.  Note that it is impossible to ask about a risk factor you haven’t thought of.  A standard type of question would look like this: “How often have you eaten pretzels in the past 2 years?”  If you find that alcoholics reported eating significantly more pretzels over the past 2 years than the non-alcoholics, the next day the following headline might appear in the Huffington Post:  ”Eating pretzels increases risk of alcoholism.”  The story that follows the headline would advise people to eat fewer pretzels to reduce their risk of alcoholism.  Absurd.

Epidemiological studies, at their best, can only point out a possible connection between two things, but that is only the very first step in trying to figure out whether or not there really is a connection between them:

“Epidemiology is concerned with the incidence of disease in populations and does not address the question of the cause of an individual’s disease. This question, sometimes referred to as specific causation, is beyond the domain of the science of epidemiology.” 1

There is a beautifully written essay about Dr. Walter Willett, the father of nutritional epidemiology, and the shortcomings of the field, on Adele Hite’s wonderful site at www.eathropology.com.

References

[1]
Green MD, Freedman DM, and Gordis L. Reference Guide on Epidemiology. National Research Council. Reference Manual on Scientific Evidence. 3rd ed. Washington, DC: The National Academies Press 2011.
  • Raphael

    Well written and clear. Explained epidemiology and problems associated with it to me in a very simple and understandable method. Thanks!

    • http://diagnosisdiet.com/ Dr. Ede

      Thanks, Raphael–I’m so glad you liked it:)

  • Korteztk

    Epidemiology can be right on, depending on the science. You make it seem as if it has to be a blind guess, but that, in itself, is a bias.

    • http://diagnosisdiet.com/ Dr. Ede

      Hi Korteztk
      You are correct; we all have biases and I am no exception. I do try to acknowledge them whenever possible, or admit them when they are revealed to me. Guilty as charged–my bias comes from reading so many nutritional epidemiology studies that make no sense to me in light of what is known about food, its components, the biochemistry of the body, and the history of the world. I would not say it is useless, or that it doesn’t have the potential for useful application when aimed at less complicated topics than diet. The problem with nutritional epidemiology is that there are hundreds of variables and no real way to measure or track all of them in any meaningful way. I did say educated guess, not blind guess…but in so many nutritional epi studies, the guesses are actually worse than blind–they stem from bias and ignore reality.

Last Modified: Nov 18, 2012 at 5:52pm