Tuesday, March 13, 2012

Unpacking the meat data

So, this story from Harvard on how red meat is bad for you deserves some unpacking. First, allow me to say that all meat is not created equal: the cows that graze on the farm around the corner from where I live make meat that is quite different from that reconstituted slime used by fast-"food" restaurants. Cows that are raised on CAFOs and fed corn-based diets are practically different species from those guernseys down the street.

But putting that aside, let's just look at what the paper reports and what the numbers add up to. The investigators examined two large observational cohort studies totaling over 100,000 subjects and tried to estimate the risk of death associated with red meat consumption. Now, first, it has been widely acknowledged that dietary habit surveys are a difficult beast, and that is how these two studies got at the food history. Next, let us look at some of the numerators and denominators. The paper reports 23,926 deaths among these >100,000 subjects over 22 to 28 years of observation. The denominator for this type of a study is person-years, where you simply multiply the number of persons observed by the corresponding number of years of observation. In this instance, this value is 2,960,000 person-years. So, the roughly 24,000 deaths occurred over 2.96 million years of observation, simplifying to 24,000/2,960,000 = 8 deaths per 1,000 years overall. If we were to translate this to an individual's risk for death over 1 year, it would be 0.008, or under 1%.

The study further reports that at its worst, meat increases this risk by 20% (95% confidence interval 15-24%, for processed meat). If we use this 0.8% risk per year as the baseline, and raise it by 20%, it brings us to 0.96% risk of death per year. Still, below 1%. Need a magnifying glass? Me too. Well, what if it's closer to the upper limit of the 95% confidence interval, or 24%? The risk still does not quite get up to 1%, but almost. And what if it is closer to the lower limit, 15%? Then we go from 0.8% to 0.92%.

Does this effect size matter, even if statistically significant? What if this were a randomized controlled trial for a statin? What would we say to this result? Even if this is a real signal, which is questionable given the observational design (yes, despite holding a special affection for observational studies, I don't think that this cause-effect is completely unconfounded; and this matters greatly in view of this minuscule magnitude), I am far more likely to die next time I get into my car than from eating burgers, even if I do indulge in one a couple of times per week. I am certainly not advocating eating red meat 7 days per week, though this view is driven more by practical concerns for sustainable beef farming than by the data presented in this paper.

There are a few political issues to disentangle. I despise CAFOs and their product; I despise their contempt for the animals and for the environment; and I despise their disregard for human health. I would love to see a study that shows that CAFO-raised beef kills people, as my cognitive biases tell me it must. I would love to see them all shut down, period. And this goes for the meat packing and distributing oligopoly as well. This venom notwithstanding, the current paper gives us no ammunition to this end: it failed to explore this pivotal question. Pity!

Furthermore, a study like this is likely to feed extremist marketing messages to suit someone's agenda that will likely drag us farther away from the moderation that is conducive to our health. But my local farmers should rest easy knowing that this is not by any means a game changer, that there is nothing in this paper to make us any less enthusiastic about their product, and that our New England pastures will not any time soon be devoid of the beautiful sight of these lovely ruminants. Moderation in everything including meat consumption, is probably still the best course of action. If we focus our energies on what is genuinely good for our health, we will do right by the environment.      

13 comments:

  1. Marya,

    I'm planning to quote part of your item at GMO Pundit blog, with a link back to this piece.

    regards

    David

    ReplyDelete
  2. Thank you, Marya. One thing that also bothers me is the fact that I know comparatively few people who eat red meat every single day -- and, according to this study, even if they did, the increased mortality risk would be miniscule.

    One of the reasons this study bothers me so much is that the numbers, and the methodology, are transparently ridiculous and poor quality. I am, like, nobody important at all, and I can see how poor the analysis was. Which tells me that the people who did it are not labouring under any delusion that they are going to impress anyone who knows how to do basic math with this study.

    I believe that this is a ploy for media reporting, for alarmism, and ultimately to push some agenda - and that agenda could be many different things. But the point to me is that this is more a piece of marketing than it is a serious piece of scientific research. That is really unsettling to me.

    ReplyDelete
  3. Michelle, thank you for visiting and commenting. I'd like to think that the investigators were pursuing a legitimate scientific hypothesis, though we come at all science with our political and cognitive baggage. I would have preferred for the press to have evaluated the data much more rigorously than they did. Essentially just about everyone copied from the press release, it seems. And now it is near-impossible to get ahead of that load of misinterpretation with the real story.
    Really this is why I wrote "Between the Lines," so that a savvy consumer of medical news can at least articulate some of these questions, if not get at the actual answers.

    Marya

    ReplyDelete
  4. Bittman's take today. Hits the right notes, not the piece he would have written the day of study release though--cooler (wiser) heads prevail:
    http://opinionator.blogs.nytimes.com/2012/03/16/no-surprise-meat-is-bad-for-you/

    brad

    ReplyDelete
  5. Thanks, Brad, I am thrilled to be referenced in his column! These headlines have been a runaway train, and, despite several measured analyses on the web, it has been impossible to get ahead of the original message.

    ReplyDelete
  6. www.lewisfamilyfarm.com will set you free...

    It is not the red meat - it is what Conagra et al infuse along the way.

    When it tastes good, it is good: Duke Ellington... ?

    ReplyDelete
  7. The difference between 0.008 and 0.0092 in one's chances of dying each year is an expected number of years of 87 if you don't eat meat and 74 if you do. So, if it is correct it is very important.

    ReplyDelete
  8. Thanks Anonymous! I'd like to see you math. Ultimately, though, it s also important to remember that such a tiny effect size in an observational study with so many other potential confounders and effect modifiers is highly unlikely to be due to this primary exposure.

    ReplyDelete
  9. Dear, Anonymous, me again. The rason I am skeptical of your math is this, from the BBC' Ruth Alexander (found here: http://www.bbc.co.uk/news/mobile/magazine-17389938)
    "The easiest way to understand it is to think of how this might affect two friends who live very similar lives, according to David Spiegelhalter, a Cambridge University biostatistician, and the Winton Professor of the Public Understanding of Risk.
    Imagine that the two friends are men aged 40, who are the same weight, do the same amount of exercise and do the same job.
    The only difference between them is that one eats an extra portion of red meat every day - an extra 85g, or 3oz.
    "Let's say that every work lunchtime one of them had a hamburger and the other didn't.
    "What the study found is that the one who likes the meat had a 13% extra risk of dying. They're both going to die in the end, but one has got this extra annual risk of dying."
    But what does that extra risk amount to in practice - for these two average people? The paper doesn't say.
    Spiegelhalter has been working it out.
    "The person who eats more meat is expected to live one year less than the person who doesn't eat so much meat. You'd expect the 40-year-old who does eat the extra meat to live, on average, another 39 years, up to age 79, and the person who doesn't eat so much meat, you'd expect him to live until age 80."
    So all those headlines, and it turns out we are talking about whether you might live to age 79 or 80."

    I could not have put it better myself.

    ReplyDelete
  10. But somehow vegetarians live several years longer than meat eaters (see linked summation of various studies), so the cumulative effect is likely greater than the above estimate of one year difference.
    http://www.ajcn.org/content/78/3/526S.long
    " The findings from one cohort of healthy adults raises the possibility that long-term (≥ 2 decades) adherence to a vegetarian diet can further produce a significant 3.6-y increase in life expectancy."

    ReplyDelete
  11. Comparisons with vegetarians should be made between populations of similar health consciousness, age, income etc. When that is done no advantage is seen.

    ReplyDelete
  12. Mary,
    Could you please explain how you converted a rate to a risk in your calculations. Based on the information you provided it seems that these cohort studies used the rate as the measure of disease frequency (#disease/ person time at risk- where the numerator and the denominator have different units ), however somehow you convert this directly to a risk (#diseased/ # at risk- where the numerator and the denominator have the same units). I thought it was well established that the rate is a measure of the disease in the population not at the individual level, yet you have done this. If the study has used cumulative incidence then risk at the individual level would have seem appropriate.
    I would appreciate if you could clarify why you think its is correct to convert the measure of disease frequency that is clearly a rate to a risk as you've done here. Thanks

    ReplyDelete
  13. Dear Anonymous,

    Thank you for visiting (again?) and commenting. I am not sure what you are referring to, but the units in the numerator and denominator did not change -- still death per unit of time. I took the incidence density and reduced it to 1 person-year in the denominator, ergo an individual risk over a year. Granted population risk is very difficult to apply to a single individual's risk, but how else would you interpret death per person-year? And how else do we hope to impact behavior and other modifiable risk factors if not at the individual level?

    Two things:
    1. It would be great to know who you are, and
    2. My name is Marya, not Mary.

    Thanks again.
    Marya

    ReplyDelete