The Science of Food

In the past I’ve been critical of both observational studies and the reporting of these studies in the media and yesterday the Independent published an article called “Food that makes up more than half of western diets linked to lung cancer” which really pushed my buttons. The title is clearly link-bait and it had me sputtering and spitting my morning tea over my keyboard because The Independent should know better (and I guess I should know better than getting excited about stuff I see on the internet, but anyway). The problem is that the article does a reasonable good job of conveying the results of the study but it does a terrible job connecting the dots for people. The connected dots spelling out that the study they were reporting on had significant flaws and in no way supported the title they chose to run with.

Apart from being an observational study with all the caveats that implies, the study took data from a long running trial that was designed for a different purpose, to evaluate the effect of screening on cancer mortality. This study included data on diet for some of the participants allowing the researchers to look at food intake and cancer outcomes. When a participant entered the study, 12 years prior, they filled out a food questionnaire and from that the participants were grouped according to levels of ultra-processed food (UPF) consumption according to the NOVA classification. Using this data the researchers looked for associations between this food data and cancer diagnoses.

I’ve covered why surveys like these are not a good tool to base your research on but in this case the food survey was conducted exactly once, at the beginning of the trial. Even worse, the people were recruited from 1993 to 2001 and the NOVA classification wasn’t even defined till 2009. So what we have is a retroactive fitting of food categories to a single historical survey for each participant. A lot of interpretation would have been required for this, in my opinion, let alone that it doesn’t capture differences over time in the consumption patterns of the participants or the composition of the foods that were being eaten.

The reported effect was also very small. Both The Independent article and the research paper only reported relative risks and stated that people who ate a lot of UPFs had a 44% greater risk of getting lung cancer. That sounds pretty bad. But there were only 1,706 cases of lung cancer diagnosed out of 101,732 people included in the trial. Doing some rough calculations this means that the absolute risk for getting lung cancer was 1.8% for high UPF consumers versus 1.3% for non-consumers. So the increase in absolute risk for lung cancer in the people with the highest UPF consumption was 0.5 of a percent, sounds a little better than 44%. For the baseline in this calculation I just used the diagnosis rate for the lowest UPF group from the paper, which is rough and probably a little wrong, but the point is that 44% of a small number is still a small number.

There were some other issues, acknowledged by the authors, like a lack of data on smoking intensity, but the bottom line is that this is a long way from a randomised clinical trial and so confounding factors are always going to be a problem. The authors even acknowledge this in The Independent article, and in the research paper, but claim that because UPFs are a risk factor for things like obesity they could be causing lung cancer. I agree with them but this makes this study little more than opinion, in my opinion.

I’m not really having a go at the authors of this paper, they freely acknowledge almost all the concerns I have mentioned here. But what this work amounts to is a bit of data doodling looking for associations that can be further investigated. But in concert with some link-baiting by The Independent we are all now going through our day thinking we and our children are going to get lung cancer from our food or worse we just start ignoring these kind of reports all together.

I think we need to ask whether we publish these types of papers at all if they are going to be interpreted so poorly (and I’m not the only one asking this question). But scientists rely on papers for their livelihood and papers like these are the result of that pressure, preliminary data dressed up as a proper study. Almost every scientist is guilty of this, we need to publish after all, but when I was publishing it didn’t really matter because there wasn’t really anything that could scare people in my field. But nutritional studies are different, they can freak people out. So maybe we need a different way of reporting this type of stuff? I don’t know, I don’t have the answers but I do find it very annoying while reading the news over breakfast.

One last thing. If you want to read the original research report it is here and it’s open access. I need to give you this link because the Independent committed another crime against science and didn’t include a link to the original paper in the article it wrote about.