by Jerome Burne
There is a whiff of the Wizard of Oz about the advice on statins that we’ve been getting for years. It comes in the form of ringing pronouncements from behind a curtain that all is well; that these drugs have been tested and found to be greatly efficacious and totally safe. Unfortunately it is now clear that no one, other than the Wizard, his assistants and the munchkins who make the statins, have been able to see how they arrive at that conclusion since the curtain conceals the data.
What actually happened to individual patients during the big trials run by the drug companies is for wizard eyes only. This was the topic of a HeathInsightUK post a couple of weeks ago (The house of statin secrets) and followed up last week in the Daily Mail (Millions of healthy Britons to be prescribed drugs GPs say they won’t take). Others bloggers such as HIUK contributor Malcolm Kendrick and cardiologist Dr Aseem Malhotra have also been raising questions about the reliability of the pronouncement issuing from behind the curtain.
This proved too much for a very senior wizard Professor Sir Rory Collins who warned in the Guardian on Saturday that thousands would die if doctors kept “creating misleading levels of uncertainty” about statins. Making people “unjustifiably suspicious” of these drugs was going to lead to deaths because people would stop taking them. This is authoritarian medicine at its most magisterial.
We need transparency
It is certainly a long way from the evidence based approach to medicine advocated by Dr Ben Goldacre, set out in great detail in his book Bad Pharma. He rightly believes that we need transparency from drug companies so that independent researchers can see all the information about patients they gathered in their trials because, as he makes very clear, the companies are far too ready to juggle with statistics and spin or simply hide unfavourable results.
This would of course mean independent researchers such as Drs Kendrick and Malhotra picking over the data and quite possibly “creating levels of uncertainty” when the raw data and the conclusions didn’t match up. But then that is what science is all about.
Unfortunately there is not much chance of that happening at the moment as far as the data on a large number of statin trials held by the Oxford-based Clinical Trialists (CTT) Collaboration is concerned. This is the research organisation Professor Sir Rory Collins heads. The evidence that underpins his pronouncements about the safety and effectiveness of statins is firmly behind the curtain. It is of course possible that all his conclusions are built on rock-solid data and totally valid, but while nobody else can see it, this is faith-based medicine.
The European Medical Association (EMA), which licences drugs in Europe, certainly doesn’t believe that we should rely on faith in the pharmaceutical companies’ integrity. It bases its licencing decisions on the full results of trials run by drug companies, not just the published ones. In 2010 it announced that it was going to encourage transparency by making clinical trial data available on demand for every drug it had reviewed. Since then it has released over 1.9 million pages of this detailed information on how patients responded.
It’s hard to disagree with this initiative – although a couple of drug companies disagree so strongly that they are taking the EMA to court over it, but that is another story – and many top UK doctors have signed up a campaign called Alltrials that is pushing for something similar here. Dr Goldacre is of course heavily involved. So there were raised eyebrows when his name showed up as one of the authors on a widely reported paper published last week which found that statins had virtually no side-effects.
The surprise – schadenfreude in some quarters -was due to the fact that the evidence for this comforting conclusion all came from published trials run by the drugs companies with no attempt to gather the kind of detailed data his book was vehement in demanding. So has Dr Goldacre now transmogrified into a junior wizard behind the curtain?
On the one hand he has put up a blog which admits that the data on which his trial’s conclusions were based were “flawed” but defends its publication on the grounds that it is “a useful illustration of how… to gather side-effects data from trials”.
On the other hand, despite the flawed data, the paper makes a pretty radical recommendation that can only be of considerable benefit to the manufacturers, and possibly harmful to patients. It says that patient information leaflets for statins should be rewritten to make it clear that your chance of suffering a genuine side effect is very small. This is a recommendation based on unexamined drug company data that has been shown many times in the past to be highly unreliable.
How to get favourable results
What is useful about both Goldacre’s blog and the published paper is the detailed list they contain of ways in which trials can be set up to produce favourable results, largely by excluding many of the patients who would actually be getting the drugs in the real world. This is a dejargonised version of what the paper says:
First, the drug companies paying for the trials may have little interest in checking carefully for possible side effects. For example a biomarker for liver damage is checked in most trials although it rarely indicated a problem, but only three out of 29 trials reported new cases of diabetes.
Second, many trials don’t give any details about how side-effect reports were actually collected or how often.
Third, some trials exclude patients with disorder such as severe diabetes, kidney failure or hypertension, many of who would be likely to be given statins in the real world.
Fourth people who volunteer for trials are often chosen because they are enthusiastic and so may be less likely to report side effects and less likely to stop taking the drugs than real world patients.
Fifth many trials in our meta-analysis had a period before the trial had actually started when patients were just given a placebo to find out which ones took their pill regularly. Those who didn’t were excluded from the real trial. The result would be a those remaining would be much more highly motived than those who would get the drug in the real world.
Finally, another group who were often excluded from trials, also making them unrepresentative, were those already on a drug that affected the liver in a similar way such as certain antibiotics – just the ones who could be expected to report more side effects.
Time to go back to Kansas
In his post-publication blog Dr Goldacre also mentions the various ways patients can be selected for trials to produce more favourable results (Statins have no side-effects? This is what our study really found). He then goes on to clarify the information you need to make a judgement if the published results from a trial reflect what actually goes on.
This is something called the “Clinical Study Report (CRP) about a trial: these are very long and detailed documents that give a huge amount of detail about the methods and results of a trial, and they’re important, because methodological flaws can often be glossed over in the brief report on a clinical trial that appears as an academic journal paper.”
Dr Goldacre then illustrates the importance of CRPs with some German research which found that 87% of the Clinical Study Reports gave a complete account of the side-effects found in trials compared with just 26% in the published versions of those trials.
So a simple question to Sir Rory Collins: Please sir when can we see the CRPs of the 29 clinical trials you have behind the CTT’s curtain? If they were brought out it would be likely to cause a rapid return to Kansas and the real world.