Nutrition science is hard, but influencers make it seem so easy.

Got a question you want answered in a hurry? Checkout the FAQ.

Confused about what you should be eating? You’re not alone.

In a recent survey, conducted by the International Food Information Council, eight out of ten people said that they found conflicting advice about nutrition that made them doubt their food choices. Sound familiar?

Take something like alcohol. A simple Google search for ‘is alcohol good for me’ will tell you that no amount of alcohol is good for you but also that moderate drinking is good for your heart health. Well, which is it?

The scientific literature seems equally confused. Study after study offers contradictory results about salt, red meat, alcohol, eggs and anything else we humans consider food.

Not helping things is the constant din of influencers and brand spokespeople on social media. Self-styled experts handing out nutritional advice that is wrong almost 50% of the time.

So how did we get here? This is the 21st century, we’ve got all the data, how can we be so confused?

There are plenty of answers to these questions but in great part the confusion stems from a failure to understand the nature of nutrition science and its reliance on observational studies.

Ignorance, and sometimes a wilful ignorance, of the challenges in nutrition science fuels misunderstanding and provides a loophole for all manner of snake oil salesperson.

So, lets have a look at how nutrition scientists do their thing and see if this helps us better evaluate the dietary advice that seems to be coming at us from all angles.

The Controlled Experiment

Most of us have some type of idea of what entails a scientific study. Even if they are some hazy memories from high school. When most of us think of a scientific experiment we think of a controlled experiment.

In this type of experiment the only thing that is varied between two or more experimental groups is the thing you are testing.

A simple example would be an experiment to see whether carrots grow better on whisky than water.

All you have to do is grow two groups of carrots under the same conditions but water one group with whisky and the other with water (yes, a terrible waste of good whisky but it’s only a hypothetical example). After some time, you would just weigh the carrots and see which grew better.

In this controlled experiment there is a dependent variable, the weight of the carrots, and an independent variable, what the plant was watered with, whisky or water.

It would be better if the carrots where genetically identical, but we can always apply for a grant to refine our study if results are promising.

The Randomised Clinical Trial

In nutritional studies, if we want to run a controlled experiment, we need to use something called a randomised clinical trial (RCT).

In a RCT we would randomly select people for a control and a test group then administer, say, saturated fats to the test group and a placebo to the control group.

Then we would monitor the participants health over a defined period of time.

Preferably the study is double blinded, that is neither the participants or the researchers know which group someone is in.

It should also be prospective. That is, the participants are followed forward over time and various dependent variables measured periodically; blood pressure, cholesterol levels and weight for example.

This type of study is considered a gold standard in nutritional studies because the random selection of participants, hopefully, distributes any confounding factors (more on these below) equally across both groups. And the double blinding reduces bias and reduce placebo effects that could influence the results.

Why Randomised Clinical Trials Rarely Work for Nutrition Science

So, you might be thinking, let’s just use RCTs and sort out all this nutritional confusion. Well, it’s not that easy.

An RCT might be good for say a drug that is meant to lower blood pressure over the short term. You can grab some participants, administer the drug and measure blood pressure over a relatively short period of time.

With nutritional studies though we want to know if a particular diet is good over a long period of time, potentially decades, and these studies are enormously expensive to conduct.

We also can’t lock the participants up to make sure they haven’t started smoking, suddenly become a jogger, experienced unusual stress or done anything that normally comes with having a life.

Participants will often drop out of long running studies and we also can’t be sure that they are keeping to a specific diet.

We also can’t do anything that isn’t ethical, if we wanted to look at the effects of drinking, we can’t force people to start drinking a bottle of whisky a day.

There are a lot of challenges when we want to use this kind of study to look at nutrition (and I’ve just touched the surface if you want to get in the weeds this is a good place to start).

The Observational Study

What normally happens is that nutrition researchers fall back on what are called observational studies.

In an observational study the researcher makes no interventions at all. All the researcher does is observe a group of individuals and record outcomes.

There are a couple of different types of observational studies so let’s start with a hypothesis, say, that potato chips cause high blood pressure and design some simple observational studies.

A Prospective Cohort Study

In a prospective cohort study, we would select a group of people and then follow them over time.

Periodically, we’d meet with the participants and ask them about their potato chip consumption and measure their blood pressure.

Over time, as people develop high blood pressure, we could correlate their potato chip consumption with their blood pressure.

Alternatively, we could have a test and control cohort, people who eat chips and people who do not eat chips, that we would treat the same way.

A Retrospective Cohort Study

Now let’s say we hear about a nursing home that is really cost-conscious and its accountants have been recording what everyone in the home ate for the past 20 years.

As they are a nursing home, they also keep detailed medical records on their residents.

We are now in a position to conduct a retrospective cohort study.

We can take this data, that has been pre-collected for other reasons, tease out the potato chip consumption from the overall food data and assess its relationship to blood pressure by looking at the medical records.

A Cross-Sectional Study

Probably the least reliable study you can conduct is a cross-sectional study.

In a cross-sectional study we just select a group of people and ask them how many potato chips they have eaten over their lifetime and measure their blood pressure.

We then use this data to construct a statistical model correlating reported potato chip consumption to blood pressure.

Why Observational Studies Aren’t Considered Gold Standard Evidence

I have greatly simplified things here, but it is enough for us to consider how observational studies aren’t considered solid evidence on the level of an RCT.

Participant Selection

The first problem is how you select your participants. In an RCT you select them randomly, but that is not always possible in an observational study.

In a retrospective study you are constrained by what data is available, elderly people in a nursing home for example. The makes the study only really applicable to elderly people in a nursing home.

In a cohort study your test and control groups are dictated by the participants; do they eat chips or not. You need both to make comparisons which means you are no longer randomly selecting participants.

As I said, this can limit the applicability of your study, but more seriously it opens up the possibility of confounding factors.

Confounding Factors

A confounding factor is something that influences both the independent and dependent variables of your study which can lead to a spurious association.

A classic confounding factor is smoking when looking at the effects of alcohol on heart disease. Smokers are more likely to drink alcohol so if you don’t control for smoking then alcohol will be associated with heart disease not smoking.

Observational studies are susceptible to confounding factors because, often, you can’t randomly select your participants.

In our potato chip example, because our test and control cohorts were selected on the basis of whether they ate potato chips or not, they aren’t randomised.

It is quite possible that people who don’t eat potato chips exercise more.

So, in our study a finding that potato chips cause high blood pressure is actually confounded because the real cause of high blood pressure is the lack of exercise not the potato chips.

Although there are strategies for dealing with confounding factors they are still a huge problem for nutritional observational studies.

You need to know about possible confounding factors to be able to handle them and they can be difficult to identify.

For example, recent discoveries about the gut microbiome are a massive confounding factor for older nutritional studies where researchers didn’t know much about the microbiome or its effects on our health.

Likewise, many studies focus on a single type of nutrient, say omega-6 fatty acids, and ignore the synergistic effects of a complete diet which introduces a whole host of potential confounding factors to be found in the overall dietary context.

Humans are Forgetful and Sometimes They Lie

Another major problem for these types of studies, especially retrospective ones, is they can rely on self-reporting by participants and unfortunately humans don’t always tell the truth.

Participants in alcohol studies, for example, may be susceptible to under-reporting their actual alcohol consumption. Who wants people to know how much you drink?

Sometimes people are also just forgetful and can’t remember exactly how many potato chips they have eaten.

More damaging is the misreporting by those at the extremes of a group.

For example, under-reporting of energy intake increases with body mass index and under-reporting of alcohol intake increases in heavy drinkers. That is the more overweight someone is the more they will under-report what they eat.

I imagine at the other end of the scale people with border-line or full blown eating disorders will over-report their food intake.

Any observational study that relies on self-reporting may have problems with its data and so it’s conclusions.

Conclusion

You’ve probably heard that correlation does not mean causation.

Observational studies, by their nature, show correlation. And, for all the reasons we’ve gone through here, they can easily be confounded.

Most nutritional studies are observational. Not because nutrition scientists are too lazy to do a proper study. It’s because human nutrition is hard, timelines span decades and you can’t ethically do too much experimenting on humans.

There’s no getting around it though, because of nutritional science’s reliance on observational science a lot of somewhat less than solid evidence gets waved around by various interests groups.

So, you may ask, how do nutrition scientists establish anything? Here, I’m afraid, I’ve only told half the story.

Nutrition scientists can bridge the gap between correlation and causation. They do it by looking for a convergence of evidence from different directions, they use meta-studies to examine many different observational studies and they apply frameworks like the Bradford Hill criteria.

But all this is a story for another post.

For now if you see a newspaper report heralding the result of a single observational study or an influencer hyping a single study that backs their personal agenda, you can be pretty sure that they don’t know what they are talking about.