by Jerome Burne
Last week I was called by a friend who opened the conversation by saying: “So you were wrong about statins.” He was referring to the correction that the British Medical Journal (BMJ) has had to make in two papers that claimed statins have a high level of side effects. But I certainly haven’t recanted and started popping those pills.
Given the huge amount of attention this has attracted, the actual point being “corrected” is remarkably small. It is the kind of thing that would normally be dealt with by publishing a response to the article in the journal. I’ll come to the specifics in a moment but first a bit of context.
Even though statins have been used for about 20 years there is still a big dispute as to your chance of suffering a side effect and that matters because if it’s bigger than is commonly claimed, then the balance between risk and benefit changes.
The complaint about the two papers was made by Professor Sir Rory Collins, a very senior medical researcher, the author on a number of respected, detailed papers showing that statins are effective and have few side effects.
Why RCTs can be unreliable
In recent weeks Sir Rory has given media interviews containing widely different estimates of side-effect risk. He’s put serious muscle problems at 1/10,000 and he’s said the overall risk is 1/100. In contrast the claim made in the two BMJ papers was 1/5. Interestingly the patient leaflet for the most lucrative statin ever – Lipitor – says that: “common side-effects may affect 1/10. So experts disagree but knowing which side is more accurate is important with the proposal to put another 7 million on the drugs.
So how good is Sir Rory’s evidence? It is based on large randomised controlled trials (RCTs), generally considered the evidence gold standard. But in recent years there has been growing criticism of the reliability of these trials, especially when it comes to assessing side effects. Here are some of the factors that can lead to side-effects being under estimated:
Firstly they are virtually all paid for by the drug companies which have little interest in checking carefully for possible side effects. For instance diabetes is now recognised as a risk but just three out of 29 statin trials reported on new cases of diabetes.
Second, many trials don’t give any details about how side-effect reports were actually collected or how often.
Third, some trials exclude patients with disorder such as severe diabetes, kidney failure or hypertension, many of who would be likely to be given statins in the real world.
Fourth, people who volunteer for trials are often chosen because they are enthusiastic and so may be less likely to report side effects and less likely to stop taking the drugs than real world patients.
Fifth, many trials start with a period when all subjects get the drug. Those that show a reaction to the drug can be excluded.
It’s a scandal we can’t see full trial data
For this reason it is important that other researchers are able to get access to the full data collected in during any trial – known as the Clinical Study Report. (CSP) The CSP’s for statins however are considered a commercial secret and not made available to independent researchers. See recent HealthInsightUK post[Link].
This is not an eccentric view of mine, among those concerned about the reliability of RCTs is Mark Wilson of the Cochrane Collaboration, a body dedicated to establishing an evidence base for treatments: “It is a scandal that we still do not have access to all trials’ data so that we can be confident in our conclusions,” he said last year.
It also relevant here that a declaration of interest form, published on Zoe Harcombe’s site’here, shows that Sir Rory together with another top statin researcher received £91 million pounds from Merck and an associated company, makers of simvastatin along with other funding. There is no suggestion of impropriety but funding has not been publicly known before.
So with that background what was the problem with the two papers? Both referred to the same paper – a review of the medical records of more than 107,000 patients taking statins over 8 years, which reported that 17.4% had a ‘statin related event documented’.
Is this a joke?
And what had they got wrong? One said that “side effects of statins occur in about 18 -20% of cases” while the other claimed there were: ‘unacceptable side effects… in 20% of participants.’
Can you see what they got wrong? The actual figure was 17.4%. I am not joking. The further charge is that the statements did not take ‘sufficient’ account of the fact that this was not a RCT but an ‘observational’ study – considered less reliable because a various factors – age, social class, existing health – could all affect the outcome.
It seems frankly incredible that this narrow statistical point, the sort of thing that is the bread and butter of responses to articles in hundreds of journals every day, now needs a committee to decide if the each of the two articles should be withdrawn rather than corrected. (Dr John Briffa has written an angry satirical response to this on the BMJ site here)
This is despite the fact that it’s admitted that in neither case did the correction change the main argument of the article. One was challenging a big study done by the organisation headed by Sir Rory (the CTT in Oxford). It argued that the study didn’t show that giving statins to patients without heart attacks cut the rate at which they died. The other article challenged the accepted wisdom that saturated fat was a cause of heart disease and recommended much greater awareness of the benefits of the Mediterranean diet.
This could cause unnecessary deaths
That leaves the other part of the charge – that both articles failed to mention the shortcomings of observational trials, which given that they had been written for a professional audience, might be thought unnecessary.
But even if it were appropriate, then surely warnings about the considerable shortcomings of RCTs should be part of any claims about statins’ lack of side effects that are based on published results from RCTs?
Such caveats were certainly not part of Sir Rory’s interviews with the media. Here’s one example from the Mail: “We have really good data from over 100,000 people that show that the statins are very well tolerated. There are only one or two well-documented [problematic] side effects.”
He went onto confidently assert that the BMJ had overstated the dangers of taking statins by 20 times – his figure of 1% of patients suffering side effects vs the 20% from the BMJ’s papers. Not only was this a “huge error” but it could “cause unnecessary deaths.”
Statins make some less likely to exercise
This is eminence based medicine, which is quite different from the evidence based version. The data from the trials, at best, has a big question mark over it, even the drug companies themselves admit to a 10% risk and clinicians regularly report a much higher figure. This is not the cut-and-dried issue that Sir Rory aggressively presents.
In fact there is a randomised controlled trial of the side effects of statins published in 2012 in the Archives of Internal Medicine. It involved over 1000 patients compared reports of fatigue between those on statins and those getting a placebo. Not only did 20 to 40% of those on statins report fatigue but this affected their level of activity which “could lead to an increase in cardiovascular events”.
The high-powered attack on the authors and the BMJ seems to have little to do with constructing an evidence based picture of the risk and benefit of statins but rather to defend the status-quo by publicly denigrating any researcher who disagrees, in remarkably offensive terms.
“I would think the papers on statins are far worse in terms of the harm they have done,’ declared Sir Rory “than the paper on the MMR vaccine by Andrew Wakefield.” Interestingly this is a phrase that Sir Rory used when criticising a draft of the Cochrane report on statin use on healthy patients several years ago, suggesting he regards it is an effective debating point.
However Sir Rory is not the only one using heavy-handed tactics to intimidate critics of statin benefit. The same thing as been going on in Australia in recent months and there are some striking similarities – challenges to both statins and to official dietary advice as well as a critique on a minor point. It involved a TV science reporter for ABC (Australia Broadcasting Corporation) called Dr Maryanne Demasi who had made two programs for the science program Catalyst; they aired at the end of October beginning of November.
Not a single charge was upheld
The station was deluged with complaints from professional bodies claiming that the two programs had broken cardinal principles of balance and fairness. A committee was set up to assess them.
Earlier this month, after committee had produced at 49-page report, ABC decided that both programs should be withdrawn. So she’d got something really wrong, right? Well again no. It’s worth pointing out just what a challenge it is to maintain a balance while explaining disputed and quite complicated science and make a lively TV program.
And for the saturated fat program that is just what Demasi did. Not one of numerous charges made by her critics was upheld. The verdict on the statin program was equally impressive. The committee had examined 17 charges of violating standards of fair and honest broadcasting and upheld just one of them.
The criticism that was accepted, like the one against the BMJ authors, was narrow. It was a failure to make it clear that statins for patients who already had heart disease has wider benefits, even if it doesn’t make them live longer.
Repressing intelligent and informed debate
What this ignores is that without full knowledge of the trial data (unavailable) such an overview of statins risk and benefit can’t be done. However that single point was sufficient to have both programs withdrawn, leaving the popular impression that low fat diet and widespread statin use had triumphed. For a far more detailed account of this saga see Dr Malcolm Kendrick’s latest blog
What’s on display here is an authoritative system attempting to repress an intelligent and informed critique in the name of science. The flexibility of the way this ‘science’ was wielded in both the UK and Australia is illustrated by how the rules were relaxed for the establishment.
Sir Rory felt no need to issue caveats about his own research model while the National (Australian) Heart Foundation, which uses observational trials to support its advice on low fat seemed happy to use belief to support its position. The Committee’s evidence included this comment: “Notwithstanding the lack of definitive proof… the NHF believes there is enough good quality evidence to recommend a diet low in saturated and trans fats.”
Science relies on questioning and testing not eminence based bullying, I for one will continue to view statins with a sceptical eye.