That's a bad rep for a science, which has no other
aspiration than that of making sense from data, of discovering an association
between salt intake and stroke, of proving that the former causes the latter. Statistics
is above lies. Those who interpret it are not.
To be a skeptic isn't about habitually disbelieving. It is about asking the right questions. And there is a method behind this questioning. Unsurprisingly it is called statistics. The good news is, you don't need to be a statistician to become a methodical skeptic. You only need a little help on how to ask those questions and, more importantly, on how to find the answers for yourself. Which is what we are going to do.
On April 12 this year, Hannah Gardener and her colleagues
published their findings about the associations between salt intake and risk of
stroke in a community of people residing in northern Manhattan [1]. This appropriately named
Northern Manhattan Study, or NOMAS, enrolled 2657 residents with a mean age of
69 years, roughly two thirds of whom were aged between 59 and 79 years. Participants
completed a food frequency questionnaire with which the researchers attempted
to assess the participants' dietary patterns over the one-year period, which
preceded this investigation. I'm not going into discussing the potential
pitfalls of using a 12-months recall to calculate how many grams of salt you
consume every day. Let's just take those numbers as if they were accurate
reflections of participants' salt intake. The researchers categorized
participants into four groups according to how much sodium they had in their
food:
1.
1.5 grams/day or less
2.
> 1.5 grams - 2.3 grams per day
3.
> 2.3 grams - less than 4 grams per day
4.
4 - 10 grams per day
These categories didn't just occur to them from a close look
at the tea leaves. The American Heart Association (AHA) recommends not to take
more than 1.5 grams of sodium per day. Before 1.5 grams became the order of the
day, it was 2.3 grams in the previous recommendation . Which is why the
researchers used group number 1 as the reference, to which they compared the
remaining 3 groups. By the way, only 12% of the study participants were in
group 1.
The researchers then checked how many stroke
"events" occurred in each group over an average period of 10 years. And
they also checked whether there was a significant difference when comparing the
groups of higher salt intake with the reference group.
When you do this type of
comparison, it pays to keep in mind that salt is not the only potential cause
of stroke. Age is too, because the older you are the more likely it is that
you'll suffer a stroke within the next ten years. So the researchers had to
adjust for age. That's a statistician's way of asking "what would the rate
of stroke events be if all participants were equal in age?". They did this
adjustment thing not only for age but also for sex, ethnicity and education.
Simply, because we know that these factors have an influence on stroke risk,
too. This demographic adjustment was the researchers' first model of
adjustment. They went a step further with a second and a third model, in which
they additionally adjusted for behavioral and then, on top of that, for
biological risk factors. In other words, they were very thorough in isolating
the stroke risk that associates with eating salt, irrespectively of what else
you do to your health. That's good statistics work. Now let's look at the
results.
Only in the group of people with a daily sodium intake above
4 grams per day was the rate of stroke significantly higher than in the group
of people who had reported to take no more than 1.5 grams. The take-home point in
this case is, that consuming 4 grams or more of sodium per day was associated
with a significantly increased risk of stroke in this population. Now here is
the first question which you should ask:
What do you mean with "significant"?
To a statistician it does NOT mean what it probably means to
you - "substantial". "Significant" is statistician speak
for "probably not due to chance". In this case it means, there is
some association between eating more than 4 grams of sodium per day and the
risk of suffering a stroke within 10 years from now. Now you can shoot the
second question:
How large is this "significant" difference in
risk?
Let's take the AHA's demarcation line of 1.5 grams of
sodium: Of those whose salt consumption was not more than that, 7.5% suffered a
stroke. Whereas 9% of the sodium delinquents did. While this is not the correct
way of comparison, it puts things into a clearer perspective. So, let's do it
the correct way, and look at the risk in terms of event rates, that is in
events per thousand person-years. There you have 7 strokes per thousand
person-years in the group of low-sodium consumers vs. 8.9 strokes per 1000
person years in the rest. With these figures you can ask one very important
question:
How many strokes could have been prevented among the sodium
delinquents if they had gone easy on the salt?
The answer is: one in five. That is, 80% of the strokes that
did occur in this group would have occurred even if they had consumed salt
according to AHA recommendations. The picture is quite a bit different if you
look only at the group of highest salt consumers, those who reported consuming
more than 4 grams of sodium per day. In that group, "only" 60% of the
strokes that did occur would have happened if they had lowered their salt
consumption. Now, here comes your (almost) final question:
How relevant are these data to me?
Not at all if you are below the age of 40. That was the
threshold for enrollment. Which obviously doesn't mean that you should go on a
salt rampage until you hit 40 and then cut back to a daily dose of 1.5 grams of
sodium per day. It simply means, the data from this study are not applicable to
you, because your profile doesn't match the profile of the study participants.
Now let's assume, you are on the wrong side of 60, and let's
also assume, that you measured your sodium intake to be more than 1.5 grams per
day (and mind you, to get 1.5 grams of sodium you need to put 3.75 grams of
salt on your food). Your next question would be:
What's MY stroke risk for the next 10 years?
About 9%. That is, of 11 guys who have exactly the same
profile as you do, one will suffer a stroke over the next 10 years. Whereas, if
you had found yourself to consume less than the 1.5 grams of sodium, that ratio
would still be 1 out of 14. That's
a 20% reduction. And who says that cutting down on salt will get you this 20%
risk reduction? Which amounts to your last question:
Does high salt consumption cause stroke?
Who knows? The study of Hannah Gardener and colleagues CANNOT answer
this question. Their study design can only show you that there is an
association. It CANNOT show causation. Which is why Dr. Gardener and colleagues are not correct to
conclude that "The new American
Heart Association dietary sodium goals will help reduce stroke risk."
That's an assumption of causality, which would require a different study
design. For example, a study in which one group of participants is given sodium
at the AHA recommended level and at least one other group is given sodium in
excess of those 1.5 grams. For ten years, mind you. And without the
participants or their physicians knowing who gets what. It's called a double
blinded, randomized, controlled trial. It's the gold standard to prove
causality. Try to do that with salt in a real life setting.
Naturally, Reuters blared out on 25th April "High salt
intake linked to higher stroke risk". As usual, the media types gleefully
dramatize studies like these. They feed you the bits and pieces which sell
print.
But statistics are above the razzle-dazzle. Those who
interpret them are obviously not. That's why it pays to be a skeptic and to take
those statements literally with a pinch of salt.
Print
PDF
Excellent analysis. I agree that there are serious concerns with the methods, specifically the data collected from the food frequency questionnaire. There's enough data on the perceptions of people trying to lose weight on what they eat to show that there's a terrible bias when we have to report even to ourselves.
ReplyDeleteThis is one of the best analyses of I study I have ever read. Thank you!
ReplyDelete