LifeTime WeightLoss Logo

« Body Worthy | Main | Should You Strength Train to Lose Weight? »

The Headline Games of Health Research

How many health headlines do you think the average person reads in a given week? How many of these headlines inspire fear, panic, cynicism, smugness, or change? Unfortunately, maybe a better first question is how many should be taken at face value. I’d argue we should view all headlines citing research studies with 1) a level of prudent skepticism and 2) an equal motivation to discern how or why (or if) a given headline should matter to us individually.

Case in point... A  recent flood of headlines covering a British study entitled “Risk of hospitalization or death from ischemic heart disease among British vegetarians and non-vegetarians: results from the EPIC-Oxford cohort study” reminds us that there are massive amounts of detail and commentary hiding behind the eye-catching headlines. A few of these might seem rather compelling. For me, I have a rich family history when it comes to cardiovascular disease. If there are strategies proven to manage and reduce my risk, I’ll be among the first to adopt them. Given that background, you could imagine my potential interest in the following.

  • “A Vegetarian Diet Could Reduce People’s Risk of Heart Disease by up to 32%” –
  • “Risk Of Heart Disease Can Be Reduced By Up To A Third By Vegetarianism” –
  • “Vegetarian Diet Cuts Heart Risk by 32%, Study Says” – Bloomberg
  • “Could going veg lower your risk of heart disease?” –

If I navigate to one or all of the above headlines for the full story, I would likely be grabbed emotionally by statistics identifying heart disease as the leading cause of death in the developed world. Primed this way, I might keep reading, hungry for more information and more answers--as well as some clear and contained solution.

Unfortunately, even when I read the full media summary, I’ll invariably come away with less than an accurate understanding of the study itself. But wait! I sought out the original article and paid my $12 for the opportunity to get to the bottom of this question myself. Should I become a vegetarian for the sake of my heart’s health as the tantalizing headline suggests? Should you?

How many studies do we come across that suggest some similar change (simple or drastic)? In that spirit, let’s take apart this particular study as an example. The vegetarian research is case in point more than point itself today.

The How: Is This a Study or An Observation?

We’re often misled by headlines like those above because of a simple misunderstanding. In nutrition and health science, there are two basic types of research models. The ideal model of scientific research is the controlled trial--i.e. a true experiment. This type of research looks at the outcome of a specific intervention or treatment in a group or groups of subjects. The true “gold standard” of these experiments are the double-blind, controlled, crossover studies in which the participants are subjected to all treatment conditions at different times and serve as their own control. On top of this, neither the researchers nor the subjects actually know which treatment is being given to each group at any point until the experimental period is over and all the data is gathered.  Let’s explore a hypothetical example of such a study.

Let’s say you are a researcher trying to determine if a new super-duper happy pill causes an increase in blood markers for super-duper happiness. You ask for 2000 volunteers to sign up for the study. As you review their applications you discover that 200 of them have a health condition for which you’re unable to predict the potential experimental drug interactions. Consequently, you cannot risk their health by including them. (Darn.) As you examine other factors such as smoking habits or age, you eliminate another 100 potential subjects because they rank themselves as “heavy smokers” and can’t take your experimental drug either. Another 200 are too young to study ethically and only signed up because of the cash you offered. (Youngsters…)

Alas, you now have a sample population of 1,500 subjects that safely and ethically meet the allowable experimental conditions. You randomize them into three groups using fancy computer software. Now you have groups A, B, & C. You gather baseline blood measurements for super-duper happy markers for each individual, who now have been reduced to a number identifying them into their respective group.

Let the experiment begin! Group A will take your drug for a month (without you or them knowing – remember, you’re blinded and they’re blinded). Well, they know they are taking a pill, but they have no idea if it’s real or fake (a.k.a. a placebo). Group B will take the placebo pill for a month, and group C will take a placebo too. All three groups will be given instruction to not change a single habit while they are in the experimental period.

You’ll gather blood samples from all the groups to measure super-duper happy levels at the beginning and end of each intervention.  Not only are you gathering info on the effects of the experimental drug in two similar groups taking the drug, you’re also comparing that data to a third group that never took it in the first place! This model is an example of a double-blinded, controlled, crossover study and is believed to yield the most accurate and dependable scientific data we can obtain in a world with so many variables. In all, you’ve collected a pretty large bunch of data in a secretive, “double-blinded” manner. This way, you are unable to edit the findings at any point because of your own “blind” position within the set up itself.

That said, you just performed an experiment. And it was tiring. And expensive. People who read about your methods, protocols, and results will say you should do it again, but this time get more people and try different doses of super-duper happy pills. This is the world of research. When done well, a study of this nature presumably earns researchers the confidence to say, “This intervention likely caused these outcomes, and we have reason to believe it because our data demonstrate it.”

A second (usually less expensive) form of research is known as observational research. The British study that made such compelling vegetarian-focused headlines is an example of observational research. While there are opportunities for observational research, sometimes called epidemiological studies, to help us identify patterns from which we can learn, these studies cannot claim to show us causation. At best (if well designed and analyzed properly), these large sets of data can uncover associations between one or more characteristics being assessed and a given outcome. 

By reading the headlines highlighted earlier, one would believe if everyone became vegetarian it would cause us to see fewer cases of heart disease. It is a general possibility that this suggestion may be true. The study in question did produce data that demonstrated an association between being vegetarian and having a lower relative risk of heart disease. However, the study does not and cannot show that being vegetarian causes less heart disease. Because the study is observational in nature, no cause can be assigned. The headlines, however, get as fast and loose as they want with their claims.

Just for fun, let’s apply this logic to another situation. If we gathered a large sample of basketball players that included amateur, collegiate, and professional players to compare their average heights, we may find the average height of all players is taller than the rest of the population. We may even be able to observe that the professional players are taller on average than the collegiate players.  Makes sense.  Although we could say that being taller is associated with being a professional player, we have no right to say that being taller than average causes a player to become professional, nor could we say being a professional player causes one to be taller.  Observational studies have definite limits.  Headlines [definitely] do not.

The Who: Meet the Subjects

Beyond the design of any research study, knowing who the subjects are is the most important thing to deciding whether the results have any relevance. Does the population studied have any similarity to you or me as individuals? Should we care about the outcome? Does the population represent larger populations well enough to make claims about overall health? 

Let’s look at how these questions play out in our case in point. Study authors Crowe, Appleby, Travis, & Key selected a total study population of 44,561 British adults between 1993 and 1999, and followed the individuals for an average of eleven years or so until September of 2009. This seems like a reasonable amount of time to follow the characteristics of a group this size for a certain outcome. Basically, researchers followed their groups’ British health records for evidence of hospitalization or deaths coded for cardiac-related incidents in accordance with the World Health Organization’s International Classification of Diseases (ICD) – a standard practice for epidemiological studies.

I was a bit confused, however, by the fact that the vegetarian & non-vegetarian samples were not more alike. For example, there were more than twice as many non-vegetarian subjects as vegetarians (n=~30,000 non-vegetarians, ~15,000 vegetarian-76% of whom were women). Additionally, “[v]egetarian subjects tended to be younger, more towards the age of 50 than 70” at the time of heart incident. In an ideal or even a better controlled research setting, studies should only draw conclusions from two or more very similar populations with just a single variable differing amongst the groups.  We’re already observing two: significantly different sample sizes and significantly different sample ages. No statistical analysis is bullet-proof enough to control for this in an epidemiological study. Add to this point, the authors’ own statement in the final paragraphs: “Given that the cohort (study population) was not a representative sample of UK adults, the absolute rates of IHD (ischemic heart disease) in the general population may differ from that of the 6.8% reported for non-vegetarians aged 50-70y.” Ummmmm...

The What: A Look at Information Gathered

Let’s continue to persevere here.... Compounding this ambiguity is how researchers allowed subjects to self-report all the data to classify themselves as vegetarian or not.  Subjects merely filled out a Food Frequency Questionnaire one time during study recruitment between 1993 and 1999.  These questionnaires are notorious for producing margins of error that can skew results of any study, especially those being conducted in uncontrolled environments over an average of eleven years. (Bingham, 1994) (Kristal, 2005). Imagine this: if you were asked to fill out a single survey covering your use of over 130 food items in the past twelve months, would you be confident those characteristics would be consistent for the next eleven years, or do you think your choices might change slightly-or perhaps drastically-in that time frame? 

For clarity, the vegetarian subjects were self-classified and included those who reported to abstain from eating meat and fish but may still consume dairy milk, dairy cheeses, and eggs as animal proteins. In fact, vegetarian subjects actually consumed more dairy milk and cheese than their non-vegetarian cohorts. At the end point, if a subject was hospitalized for or died of heart related condition, there was no way to tell if any of the individuals would still be classified as vegetarian or non-vegetarian! Yet, these are the factors that determined the “association,” which was then simplified and blown up by the headlines.

The “So What?” Factor: What Does This Mean for Me?

Now let’s look at the actual outcomes. 

  • Researchers followed 44,561 subjects, recording incidences of IHD if an individual was either hospitalized for or died of a condition related to heart disease classifications
  • There were a total of 1,235 occurrences of Ischemic Heart Disease reported (2.8% of total study population)
  • 21% of these 1,235 IHD cases involved acute Myocardial Infarctions (Heart Attacks) and 50% of them were classified as chronic IHD
  • Of the 1,235 IHD cases, there were 169 deaths due to IHD (13.7% of IHD cases, and just 0.38% of total population!)

As significant as any loss of human life is, from a statistical standpoint, the argument to take up vegetarianism might not seem as compelling as the headlines immediately suggest. Let’s look now at a tricky research calculation. Remember the reported statistic highlighting a 32% risk reduction for vegetarians? Let’s break it down a little further.

The 32% risk reduction stat is not a straight number representing absolute risk reduction. It’s calculated using relative risk, which makes it appear more significant. Let’s read the authors commentary to discover where this 32% came from.

First, there’s this statement: “The cumulative probability of IHD between ages 50 and 70years was 6.8% for non-vegetarians compared with 4.6% for vegetarians.” If we subtract 4.6 from 6.8 to come up with the difference in absolute risk of IHD between the two groups you get a value of 2.2. This 2.2% happens to be 32.352941% of 6.8. Here we have the source of the 32% reduction in relative risk. In other words, the data collected in this study showed that for every 100 British non-vegetarians between ages 50 and 70years, about 6.8 of them may be hospitalized for or die of conditions related to heart health. For every 100 British vegetarians between ages 50 and 70 years, about 4.6 will be hospitalized for or die of conditions related to heart health. Yes, it comes down to an absolute risk of 2.2%--as it was calculated within the format of an observational study that included (as noted earlier) some fairly loose group profile comparisons and inevitably fallible (however common) self-report information techniques.

Could there be legitimate health related reasons for choosing vegetarianism? I leave the door open to that possibility, particularly when care is taken to eat real (i.e. not processed) food and ensure proper intake of nutrients. (This said, I understand many vegetarians choose this diet because of other, non-health motivations.) In my experience as a dietitian, the vast majority of people who decide to give up animal proteins have a difficult time achieving optimal health on plant-based foods alone. Without rich protein sources like meat, poultry, and fish, many people find it difficult to maintain resilient immune systems, stable blood sugars, or lean body mass.  The majority of vegetarians I’ve worked with did not eat significantly more vegetables by adopting the “veg” lifestyle; they’ve ended up eating more processed meat substitutes or starchy foods that provide far less nutrient density than animal proteins. Eating a healthy, nutrient dense, vegetarian diet that provides for your body's full nutritional range is difficult but can be done. Certain nutritional monitoring can be especially helpful for anyone attempting to accomplish this, and the Healthy Way of Eating can be a good guide for everyday reference.

Does the study give me reason to question my non-vegetarian diet? When it’s all laid out--no. Although I'll continue to follow any well designed studiesthat look at this  question--particularly those characterized by the best controlled research--or now I'm confident going vegetarian simply isn’t a surefire way to improve health. In the end summation, it's important to remember a headline may give us pause, but in too few cases does the full story give us accurate and compelling direction.

What do you think of this study (and research process) taken apart--headline to calculation? I hope you’ll share your thoughts on what goes into health headlines--as well as the reactions and questions you commonly take away from them.

Written by Paul Krieger - Corporate Registered Dietician

 This article is not intended for the treatment or prevention of disease, nor as a substitute for medical treatment, nor as an alternative to medical advice. Use of recommendations in this and other articles is at the choice and risk of the reader.


PrintView Printer Friendly Version