Keep statin supremo away from the missing side-effect data

By Jerome Burne

The statin saga – do they help or harm? – took a fascinating new turn on Sunday when statin supremo Professor Sir Rory Collins confessed to the Express that he hadn’t actually done the analysis needed to uncover the true side-effect rate.

If you’ve been following this pharmacological soap, your response when you heard this was probably first amazed laughter, followed by outrage at the breath-taking hypocrisy and then, after a brief reflection, alarm at the implications.

The hilarity was prompted by the fact that for years Sir Rory has been telling everyone that statins are virtually side effect free. The outrage is because he has defended this belief, which now turns out to be far from firmly based, in ways that have nothing to do with the norms of scientific disagreement. (For background detail, go to the drop down menu “categories” at the top of this page and select “statins”).

Last summer, when a couple of articles in the British Medical Journal suggested the true side-effect figure was much higher than he was claiming, he gave an interview to the Guardian warning that thousands would die if doctors kept “creating misleading levels of uncertainty” about statins. Making people “unjustifiably suspicious” of these drugs, he said, was going to lead to deaths because people would stop taking them.

Reports of side-effects ignored for years

In fact he had no data other than what the drug companies had published – a suspect source of information. His team had only looked at the risk of cancer and serious muscle damage. They had so far ignored years of reports from doctors and patients of serious muscle pains, diabetes, cataracts, memory loss, brain fog and declining libido. So the basis for those charges of killing patients was….?

The alarm was then triggered by Sir Rory’s assumption that he was a suitable and fit person to head a new and belated analysis of the data on side effects. A telling reason why he shouldn’t do it is that he thinks the purpose “is to convince the public that statins were safe.” Now there’s a guarantee of impartiality. Later I’ll come to several others reasons why he should be kept well away from any attempt to assess statin risks.

Sir Rory heads a large and very influential, but little known outside medical circles, unit at Oxford called CTT (Cholesterol Trialists Collaboration) which holds a unique and enormous collection of drug company data. It has all the raw material gathered in the 28 or 29 large statin trials (just one trial can generate 100,000 pages) and it regularly publishes complex meta-analyses on their effects. It nearly always finds that statins are beneficial.

In fact it was one of these big CTT meta-analyses published in the Lancet in 2010 that persuaded NICE to recommend prescribing statins for another seven or eight million people in the UK last summer. The critical articles in the BMJ formed part of a campaign objecting to this extension. Other experts have made serious criticisms of the way the CTT estimated statin benefits, described below.

Risk of diabetes greater than heart protection

The issue of side-effects is a crucial part of any decision to recommend drugs for prevention, such as statins or blood pressure pills, because lots of people have to take them for one person to benefit. There is no point in handing out a drug to prevent heart attacks if only 1 in 200 of the people taking it will benefit when 1 in 50 or even 1 in 100 will suffer serious side effects.

In fact the risk, in certain patients, of developing statin induced diabetes is larger than the chance that the drug will protect you from a non-fatal heart attack, according to a new independent analysis – details below.

So Sir Rory’s admission makes it very hard to understand how his Lancet paper could have formed the basis for the decision to double the number of healthy people eligible for the drug, who were at even lower risk of heart disease than those already getting them, without having a realistic picture of the risk of side effects.

If Disney made a movie of the statin saga the Oxford CTT unit would be a vast, turreted Dracula style castle, the retreat of Sir Rory, a strict and austere man who conducted fiendishly complicated calculations on the mounds of data he kept stored in locked underground chambers. Occasionally the thick studded main door would creak open and another pronouncement on the benefits of statins would be handed out to be published in the local journal.

Lower the drawbridge at Castle Collins

The astonishing thing about Castle Collins, which has only very recently become widely known, is that the data it holds on statins really is kept secret. Now everyone from the editor of the BMJ to the head of the Commons health committee are calling for the drawbridge to be let down and the shutters opened to allow light and independent inspectors into its secret chambers.

The secrecy dates back to an agreement between CTT and the pharmaceutical companies in 1995 – in return for being given the raw data on the trials (what is published in a scientific journal is actually only a very bald summary) the CTT would not allow any other researcher to see them on the ground of commercial confidentiality. It is a deal that is no longer acceptable, given what we now know about the ability of companies to fudge and spin trials to produce favourable results.

Another result of the secrecy is a great deal of confusion about where the side effect data on statins actually is. Two years ago – before the great NICE expansion of prescribing – there had been an opportunity to do an assessment of side effects of the sort Sir Rory is at last proposing. The respected Cochrane Collaboration was doing a review of statins and buried in the foot notes is revealing exchange about the gap between the super low CTT estimate of side effects and the higher rate reported by others.

Side-effect data goes missing

Professor Harriet Rosenberg of the Health and Society Program at York University in Toronto thought one way to find out which side was right would be for the Cochrane Reviewers to examine the raw patient data but they said they couldn’t get it. “It’s not clear if the AE (adverse events) data was withheld from the Cochrane reviewers (by CTT) or were not collected in the original trials,” she wrote.

According to the lead author of the review Dr Shah Ebrahim, the CTT didn’t actually have the data. “Full disclosure of all the adverse events by type and allocation from the RCTs is now really needed,” he replied “as the CTT does not seem to have these data.”

What more damning indictment of the secrecy surrounding Castle Collins can there be? Over 20 years after these powerful drugs were introduced and doctors encouraged to keep handing them out to more and more people, not only has a proper analysis of the side effects found in clinical trials still not been done but senior people in one of the top drugs watchdogs doesn’t even know where the relevant data is.

However the mists that constantly swirl around the turrets of Castle Collins needn’t prevent us getting a reliable fix of the risks and benefits of statins. One useful study has just come out that tells you how many people need to take a statin for one person to benefit (known as NNT- numbers needed to treat) and how many for one to be harmed (NNH – numbers needed to harm).

Diabetes risk greater than heart protection

It is posted on a site called theNNT.com which is run by Professor David Newman director of clinical research at Mt. Sinai School of Medicine in New York, together with doctors at St. Luke’s-Roosevelt Hospital Center in New York City. Last month they put up an analysis which set out the risks and benefits of statins taken for five years by people who hadn’t had a heart attack but who were at high risk – most had conditions such as high blood pressure and/or diabetes or smoked.

Their risk of a heart attack (25% over the next 10 years) was much higher than the risk level that NICE decided should make people eligible (10% over ten years). That means that you would expect them to show up as getting more benefit than the lower risk NICE group.

The results were stark and simple.

NNT: Lives saved – 0

         Preventing a heart attack – 1/104

         Preventing a stroke – 1/154

NNH: Developed diabetes – 1/100

           Muscle damage – 1/10.

In other words you are more likely to develop diabetes if you take a statin for five years than you are to avoid a heart attack and it won’t make you live any longer.

Statin benefits over-estimated

Now this is far from the final word. As Professor Newman sets out very clearly there are always problems with reviews like these that combine the data from lots of trials (meta-analyses) but it is just the sort of analysis that CTT relied on.

There is the issue of how good the individual trials were, how well they recorded side effects and assumptions made by the reviewers and so on. This study also doesn’t deal with the side-effect question in any detail – nothing about rate of muscle pains (as opposed to damage), memory problems, cataracts, brain fog and such like.

Newman concludes that deciding to take a statin when you haven’t had a heart attack “may be best left to individual preference,” because, he goes on to say, ”we believe the benefits are best-case and the harms may well be underestimated.” As far as the apparent greater risk of diabetes is concerned, he says it is more important to avoid diabetes “a chronic conditions with serious long term morbidity …than a single event such as a heart attack or stroke.”

Newman acknowledges that the conclusions of NNT.com don’t square with the CTT results but explains that it is because the Oxford group uses a unique method of calculating benefit, which his group doesn’t agree with, and which, he says, “provides potential advantage to the groups taking statins.”

Statistical magic gives statins their shine

Certainly the way Newman describes CTT approach makes it sound distinctly odd. The standard way of analysing a clinical trial is straightforward. You look at the benefit in the group getting the pill and see if it is better or worse than those getting the placebo. What Sir Rory and his CTT team do is to compare the placebo group with those getting a statin but, and this is the important bit, only with those who lowered their cholesterol the most.

In other words the placebo group is compared with those who respond best to the drug. This could be because of their genes, or because they take the pill very regularly or because they exercise more. Whatever the reason, this method seems very unlikely to tell doctors and patients anything useful about how people in general are going to respond to statin drugs.

These days when regulation of institutions needs to be seen to be tough, independent and done in the interests of customers, the idea that Sir Rory himself is the best person to ferret out the side effect data from the dungeons of Castle Collins, or possibly to prise it out of the grip of the drug companies, is very implausible.

Let’s have a clean sweep. Hand the job over to the Cochrane Collaboration possibly headed by Professor Peter Gotzsche, whose forensic and dogged examination of Tamiflu provided a much clearer evaluation of the drug than the company was providing.

And finally…

Just after this post went up a paper was published showing yet another way statin researchers manipulate statistics to make very iffy trial results look really positive. The authors, David M. Diamond, a professor of psychology, molecular pharmacology and physiology at the University of South Florida, and Dr. Uffe Ravnskov, an independent health researcher and an expert in cholesterol and cardiovascular disease, show how the minimal actual benefit – one fewer heart attack among a 100 on statins – can be presented as a benefit of 30-50%. An explanation of the magic can be found here:

Diamond and Ravnskov conclude that while a a pill that promises a longer, heart attack-free life has great appeal, statins only have a small impact on heart disease while their adverse effects ‘are far more substantial than is generally known.’

 

 

 

Jerome Burne

Jerome Burne

Jerome Burne is the editor of HealthInsightUK. He is an award-winning journalist who has been specialising in medicine and health for the last 10 years and now works mainly for the Daily Mail. His most recent book “The Hybrid Diet” was written with nutritionist Patrick Holford, published 2018. Award: 2015: Finalist for 'Blogger of the Year' Medical Journalists' Association.

31 Comments

  • Regarding CTT’s 2010 paper: , , “Retraction!!!”

    If this paper should indeed be based on false / falsified data, what would be the most effective way of making sure that

    a) it is documented for everybody to see what has happened
    b) this information stays accessible and doesn’t vanish behind a pay-wall

    As much as I dislike it, retraction may not be the most sobering option to choose. Some journals allow for “corrections”, however as far as I’ve seen, these corrections are not always “advertised” aggressively, they’re easy to miss.

    In such an important case (public health), if found guilty of cheating, it might be advisable to not only require a correction, but to also permanently mark the original publication (online & inside PDF document) as invalid or even fraudulent.

    The more I ponder this issue, the more I think that total retractions are not a good thing at all. Weeding out crap is necessary, but thereby creating a white-washed appearance as if nothing had happened doesn’t feel right.

    P.S.

    I think there is a typo in “It is a deal that is no long acceptable,…” [drawbridge paragraph]

    • Editorial

      Thanks for typo alert. I have no idea whether the CTT’s special way of analyzing data would count as falsifying. It certainly looks odd but performing mata-analyses is an arcane art and perhaps this twist is OK??

      What it does highlight, it seems to me, is that evidence based medicine has reached a point similar to what was happening in banking before the crash, which was relying on what turned out to be toxic debt packages tied up with complex mathematics. They bought the system down because certainly no ordinary investor and very few of those in the industry could actually understand them, let alone query them. And of course their one comprehensible message was what everyone wanted to hear.

      I think it is fair to say that few doctors would feel competent to critique most meta-analyses let alone spot what CTT was doing with the data. What else is going on in those data bundles that the rest of the profession rely on to make prescribing decisions and who is responsible for verifying them?

      • I do agree with your point comparing evidence based medicine (EBM) and matters in banking, which became so complex that few could understand what was going on – contributing to the crash.
        EBM sounded so simple when it came in: Surely doctors and patients all agree that we should only use treatments which have a good evidence base ? Seems obvious .However if the evidence is to come from trials we have to ask who will pay for these trials – mainly drug companies. Which explains the bias towards drug treatments and the marginalization of lifestyle improvements I have witnessed in 30 years of medicine.
        The cholesterol debate illustrates this point so well.
        Dr David Unwin

      • Your comparison of EBM with mathematical investment banking really hits the mark! I hope you can somehow float this idea to a larger audience.

        After my brush with Simvastatin, I was amazed that I could just collect statin horror stories from friends and acquaintances – handfuls of them – yet officially side effects are (or maybe were) supposed to be rare!

        Mathematics has in effect stopped doctors doing what they used to do – using their experience with patients to flag up general problems. I imagine the older ones may even grumble privately about their statin side effects, and still not do anything about the problem professionally!

        One particular problem with heavily mathematical treatments is that they often contain hidden assumptions. For example, buried deep down is often the assumption that distributions are Gaussian – but many things simply can’t be because they contain an obvious cut-off at zero (nobody has negative BP).

    • TO THE STORY ON STATINS, STATINS ARE THE SAFEEST THING ON THE PLANET,ON A PIECE OF PAPER..ONCE MADE AND TAKEN, THERE HORRORS.. I KNOW I TOOK FOR 5 YRS AND STILL SUFFER YRS LATER. NO ONE WANTED TO HEAR WHAT I WAS SAYIN, FDA,WHITEHOUSE,CDC,STATE SEN.STATE GOV. NO ONE… THAT WAS 2000-2005 WHILE ON,2015 STILL OFF STILL SUFFER… NOT AS BAD BUT ALL THIS PILL WAS SUPOSE TO DO WAS EXEND MY LIFE,IT REALY INCREASE MY CHANCE OF HEART ATTACK BY 1 OF MANY TROUBLES IT GAVE ME.. STRESS FROM LOOSEIN A 100,000$ CHANCE AT THAT JOB.THE MONEY I GOT FROM MY DADS DEATH. GONE.5MONTHS OF A YR I WORKED THE REST AT HOME OR DR OR HOSPITALS WITH TROUBLES, NOW TELL ME THERE SAFE… I CAN PROVE U WRONG..KB GOOGLE MY BOOK THATS NOT EVEN ALL I SUFFERED.

  • BRAVO Mr. Burne! Glad to see Sir Rory’s neglect of statin side-effect data is getting some coverage in the UK. Unfortunately, to date, it appears the latest developments in this saga have gone completely unnoticed by both the medical press and the lay press in the US and Canada.

    Today, however, the ScienceDaily website has reported a new bombshell study which critically reassesses existing statin research. The study was conducted by Dr. David M. Diamond, University of South Florida, and Dr. Uffe Ravnskov. Their findings have just been published in the Expert Review of Clinical Pharmacology.

    Here are relevant links:

    1) SAFETY AND LIFE-SAVING EFFICACY OF STATINS HAVE BEEN EXAGGERATED, SAYS USF SCIENTIST; Press Release from University of South Florida with additional links to other current statin-related breaking news, Feb. 20, 2015…
    http://www.eurekalert.org/pub_releases/2015-02/uosf-sal022015.php

    2) HOW STATISTICAL DECEPTION CREATED THE APPEARANCE THAT STATINS ARE SAFE AND EFFECTIVE IN PRIMARY AND SECONDARY PREVENTION OF CARDIOVASCULAR DISEASE PREVENTION, David M. Diamond and Uffe Ravnskov, Expert Review of Clinical Pharmacology, March 2015…http://informahealthcare.com/doi/abs/10.1586/17512433.2015.1012494

    3) SAFETY, LIFE-SAVING EFFICACY OF STATINS HAVE BEEN EXAGGERATED, SAYS
    SCIENTIST; ScienceDaily Featured Research, Feb. 20, 2015…
    http://www.sciencedaily.com/releases/2015/02/150220110850.htm

    Hope the foregoing is of interest. Cheers.

    • Editorial

      Many thanks for links to that post. Great to see serious critiques of the statin data being done from several sources. Wonder if the ability to ignore contrary evidence shown by the dietitian fraternity will be shared by the cholesterol brothers. Seems very likely unless you think the companies could possibly encourage statin doubts to provide a better launch for their new statin products that are already under starters orders.

      • Mr. Burne,

        Respectfully, just a couple of TYPO ALERTS:

        ~ The sentence “such statins or blood pressure pills” should read “such as statins or blood pressure pills”.

        ~ The sentence “there are always problems reviews like these” should read “there are always problems with reviews like these”.

        ~ The sentence “You look at the benefit in the group getting the pill and see if it better or worse than those getting the placebo” should read “You look at the benefit in the group getting the pill and see if it is better or worse than those getting the placebo”.

        ~ The sentence “provides potential advantage to advantage to the groups taking statins” should read “provides potential advantage to the groups taking statins”.

        BTW, you made a brilliant suggestion: out with Sir Rory and in with an honest broker like the Cochrane Collaboration, headed by Professor Peter Gotzsche who got to the bottom of the Tamiflu debacle.

        • Editorial

          Many thanks for taking the time to do that. My proofing has always been poor and my friendly sub-editor didn’t get to see this one before it went up.

      • Nah, it’s a pure coincidence. Nothing whatsoever to do with the fact that statins are off patent and cheap as chips (and about as valuable for our health) while PCSK9s are being talked about with eye-watering prices (US$7000-12000 per person/year). Let me see now, 7M uk, 6M fr, 6M? de, 30+M USA, oh, say 60M worldwide, price ‘generously negotiated’ down to, say, a mere $5k = $300,000,000,000 per annum – should be a nice little earner that…

  • Comparing placebo group with with a sub-set of the verum group is most definitely NOT meta-analysis practice. As in this case, it is very likely to introduce massive bias into the meta-analysis. As you have described it (and not having read the paper myself) it sounds like a straightforward piece of scientific fraud.

    For that reason alone I agree with your conclusion that the whole paper ought to be withdrawn. Mere alteration of this methodology is likely to alter the conclusions so profoundly that there are likely to be other ramifications throughout the paper. Tweaking it a bit will not solve the inherent problems.

    And you are also right to suggest that Sir Rory Collins should be taken as far away as possible from further interference from a complete re-analysis of the data.

  • Oh, look at this!

    Yet another paper from CTT. Women rejoice, you may now take statins as well.

    “Efficacy and safety of LDL-lowering therapy among men and women: meta-analysis of individual data from 174 000 participants in 27 randomised trials.”

    http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(14)61368-4/abstract

  • Please ask the BHF for their view, in particular from their statin supremo Prof. Peter Weissberg.

    • Editorial

      I would bet that the response would be something like:
      Stains are the most extensively tested drugs and the consensus of all the research is that they are safe and effective and have made a valuable contribution to saving lives and reducing the burden of heart disease.
      Individual findings that challenge statins’ benefit are always looked at carefully but they don’t change the high degree of expert agreement.
      The BHF will continue to monitor findings about statins and modify their advice if necessary in the light of ongoing scientific developments.

  • “An explanation of the magic can be found here:”

    Unfortunately that link seems to be dead.

  • Even a child can find out that Newman’s “analysis” is a complete b*****it. He cites a 10% value for myopathy or muscle damage when in fact it’s a proportion of participants reporting MUSCLE-RELATED PROBLEMS in an UNCONTROLLED observational study. So, this is not MYOPATHY (muscle damage with CK more than 10xULN) but MUSCLE PAIN/STIFFNESS etc. The study was uncontrolled so we can’t estimate what number of participants would report muscle problems without taking a drug. I think it is not just an innocent error. It is a deliberate misrepresantation.
    Then he completely miscalculates the number of people diagnosed with diabetes. The relative risk is not 1.81 as he claims but something around 1.25. The absolute risk translates to 1 in 66, not to 1 in 25. Including the numbers from AFCAPS/TEXCAPS it goes up to 1 in 91. But anyway, his method of adjusting for the time of observation in JUPITER trial is completely unscientific. He feels free to adjust the side effects but not benefits.
    You know how we call the type of person who misrepresents the data to prove his point? I’m sure you do.

    • Editorial

      Adam T – thanks for your detailed comment and apologies for being so slow in responding, didn’t pick up alert that it had been posted. I received a response from Professor Newman which I have included with the original text:

      ”Submitted on 2015/04/06 at 6 Apr 2015
      Even a child can find out that Newman’s “analysis” is a complete b*****it. He cites a 10% value for myopathy or muscle damage when in fact it’s a proportion of participants reporting MUSCLE-RELATED PROBLEMS in an UNCONTROLLED observational study. So, this is not MYOPATHY (muscle damage with CK more than 10xULN) but MUSCLE PAIN/STIFFNESS etc. The study was uncontrolled so we can’t estimate what number of participants would report muscle problems without taking a drug. I think it is not just an innocent error. It is a deliberate misrepresantation.

      I encourage anyone to examine this (http://bit.ly/1Iinrv7), the most comprehensive available review on the topic of muscle damage (myopathy) among those taking statins. The paper’s authors are transparent about definitions, and differences in approach. We agree with their definition of myopathy because it is patient-centered—it includes what patients experience (pain, weakness, exercise intolerance, etc.), not just what can be measured in a CPK assay. The commenter here is absolutely correct that these estimates are from observational data. Please see our ‘Philosophy’ page, see heading “Adverse Effects Data”, where we explain that from our inception we have preferred observational data for estimates of adverse effects, again because this is patient-centered. Patients typically want to avoid adverse effects whether they are due to placebo or active ingredient. The FDA agrees, using almost exclusively observational data for post-marketing surveillance and safety decisions. If side effects occur that is a problem for patients, whether the context is a randomized trial or real life.

      Note also, please, that even Rory Collins has conceded (http://dailym.ai/1ziYM88) the adverse effects from statin RCTs have been under-reported and are as yet unreliable. Indeed most large studies perform ‘run-out’ periods where those who do not ‘tolerate’ the drugs are excluded, as are patients with renal and other diseases, making RCT estimates unusable for projections about real life use of statins. Finally, we used 10% as our estimate, at the most conservative end of the published estimate range (9-20%).

      Then he completely miscalculates the number of people diagnosed with diabetes. The relative risk is not 1.81 as he claims but something around 1.25. The absolute risk translates to 1 in 66, not to 1 in 25. Including the numbers from AFCAPS/TEXCAPS it goes up to 1 in 91. But anyway, his method of adjusting for the time of observation in JUPITER trial is completely unscientific. He feels free to adjust the side effects but not benefits.
      You know how we call the type of person who misrepresents the data to prove his point? I’m sure you do.”

      We extrapolated the diabetes data from the 2013 Cochrane review (see Analysis 3.7 on p72) to match the 5-year endpoint reported for all benefits. The benefits we use are from meta-analyzed data, therefore these cannot be extrapolated forward by selecting out the JUPITER trial data in the fashion suggested above, because the median 5-year time periods for benefit reporting have already done this extrapolation, usually by regressing the results against a time variable. (Note also the JUPITER trial was stopped early, a maneuver well known to exaggerate, not attenuate, reported benefits, thus extrapolating these data forward, even if one could do this, would be deeply misleading). The diabetes data, however, are separated out using only the two trials for which the Cochrane authors had direct access to adverse effects reporting, making it possible to extrapolate to a 5-year endpoint, which we did.

      • Many thanks for posting the response but I still think that Professor Newman is seriously wrong about statin drugs:

        “I encourage anyone to examine this (http://bit.ly/1Iinrv7), the most comprehensive available review on the topic of muscle damage (myopathy) among those taking statins. The paper’s authors are transparent about definitions, and differences in approach. We agree with their definition of myopathy because it is patient-centered—it includes what patients experience (pain, weakness, exercise intolerance, etc.), not just what can be measured in a CPK assay.”

        Exaggerating the side effects of potentially life-saving medications is not “patient-centered”. It’s actually harmful. In the same way you could call every allergy an anaphylaxis or bradycardia a cardiac arrest. By calling benign symptoms such as muscle pain or stiffness a myopathy/muscle damage you are clearly misinforming the patients.

        “The commenter here is absolutely correct that these estimates are from observational data. Please see our ‘Philosophy’ page, see heading “Adverse Effects Data”, where we explain that from our inception we have preferred observational data for estimates of adverse effects, again because this is patient-centered. Patients typically want to avoid adverse effects whether they are due to placebo or active ingredient.”

        Sorry, but that’s just silly. So if in observational study 2% of patients taking a statin have a traffic accident, we should count this as an adverse event regardless of causality and advice people to not take the medicine? This type of reasoning would eliminate every single drug we use, because most people experience some kind of “side effect” while taking a placebo pill.

        “The FDA agrees, using almost exclusively observational data for post-marketing surveillance and safety decisions. If side effects occur that is a problem for patients, whether the context is a randomized trial or real life.”

        I seriously doubt that FDA would agree with your opinion. FDA recognizes the limitations of the observational data. For example, in VAERS (Vaccine Adverse Event Reporting System) one can read: A report to VAERS generally does not prove that the identified vaccine(s) caused the adverse event described. It only confirms that the reported event occurred sometime after vaccine was given.
        Just because something happened during therapy doesn’t imply that the therapy is to blame. Causality is crucial for FDA’s decisions to withdraw a drug.

        “Note also, please, that even Rory Collins has conceded (http://dailym.ai/1ziYM88) the adverse effects from statin RCTs have been under-reported and are as yet unreliable. “

        Professor Collins did not say that. He said that his previous meta-analyses of cholesterol trials didn’t investigate all side effects and in the light of public concerns it should be done by his team. Collins never stated that adverse effects in RCT have been underreported and unreliable. He believes that RCT proved that statins are safe and he performs the analysis solely because of aggressive anti-statin movement that has recently emerged.

        “Indeed most large studies perform ‘run-out’ periods where those who do not ‘tolerate’ the drugs are excluded, as are patients with renal and other diseases, making RCT estimates unusable for projections about real life use of statins.”

        Many statin trials did not use any “run-out” periods and included patients with renal diseases or other chronic conditions. See for example AURORA, CORONA, SHARP. They also found statins to be safe, contrary to your statements. For example I strongly encourage you to take a look at data on muscle side effects in CORONA trial. No difference between rosuvastatin and placebo (225/2514 vs 207/2497). But according to your “philosophy” we should count all 225 patients with muscle pain as experiencing side effects from treatment. By the way, the rate in the placebo group (8,3%) is surprisingly close to your observational estimate.

        “Finally, we used 10% as our estimate, at the most conservative end of the published estimate range (9-20%).”

        Irrelevant. All these estimates are baseless and misleading.

        “We extrapolated the diabetes data from the 2013 Cochrane review (see Analysis 3.7 on p72) to match the 5-year endpoint reported for all benefits. The benefits we use are from meta-analyzed data, therefore these cannot be extrapolated forward by selecting out the JUPITER trial data in the fashion suggested above, because the median 5-year time periods for benefit reporting have already done this extrapolation, usually by regressing the results against a time variable.”

        Sorry, but you did the same for the diabetes data. You took out the JUPITER trial from the meta-analysis of diabetes outcome and extrapolated the results to 5 years. So I can’t see why you couldn’t do it for benefits e.g. non-fatal heart attack.

        “(Note also the JUPITER trial was stopped early, a maneuver well known to exaggerate, not attenuate, reported benefits, thus extrapolating these data forward, even if one could do this, would be deeply misleading).”

        FDA determined that premature discontinuation of JUPITER trial had minuscule effect on the reported benefits. Look at page 11 in this document: http://www.accessdata.fda.gov/drugsatfda_docs/nda/2010/021366s016StatR.pdf

        “The diabetes data, however, are separated out using only the two trials for which the Cochrane authors had direct access to adverse effects reporting, making it possible to extrapolate to a 5-year endpoint, which we did.”

        Thanks to these researchers: http://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2809%2961965-6/abstract we now have access to many other trials and we can evaluate the diabetogenic effect of statins more precisely. Statin therapy increases your chances of diabetes by 9%. Anyway, the calculations you provided are completely erroneous. Let’s do it once again. Here are the numbers in Cochrane report:

        JUPITER Statin: 270/8901 Placebo: 216/8901 OR=1.26 AFCAPS/TEXCAPS Statin: 72/3304 Placebo: 74/3301 OR=0.97, which in summary gives OR = 1.18

        Now let’s extrapolate the numbers from the JUPITER trial to 5 years: Statin: 675/8901 Placebo: 540/8901. Odds ratio remains essentialy the same – 1.27.

        Then we combine the data from JUPITER with AFCAPS/TEXCAPS: Statin 747/12205, Placebo 614/12202. This gives odds ratio OR= 1.23 (1.10-1.37) not 1.81 as you claim and translates to NNH = 91 and ARI = 1.1%.

        If my calculations are OK, I trust that you correct your errors publicly. In summary, I think that you seriously misrepresent statin safety evidence and that your “patient-centered” position is in fact extremely harmful.

        And just in case: I do not have any financial conflict of interest to report. I’m a student of pharmacy but (unfortunately) I do not receive any funding from pharmaceutical companies.

        • Any side effect is benign or even irrelevant, eh?

          Unless, of course, you get it yourself!

          Even mild aches, ones you could probably tolerate for a few days or weeks, will wear you down if they persist for months or years.

          Do not belittle the destructive power of constant, relentless insults.

          I do not know what your true agenda is, but I’m disliking it already.

          • It’s obvious that you didn’t understand what I wrote. I only meant that you can’t confuse severe, possibly disabling side effect (which is myopathy or rhabdomyolysis) with muscle pain. These things are just not equal. Just like allergic rash and anaphylaxis. I also wrote that the 10% estimate is probably wrong because it comes from an uncontrolled observational study. Better designed studies show that statin-related myalgia is not so common.
            And of course I agree – constant pain, even a mild one, can be very destructive.
            Believe me, there is no hidden agenda here. I just don’t like when someone twists the data.

        • Editorial

          I was so impressed at this level of debate between Adam T and Professor that I emailed my thanks to him and made this comment:
          David – many thanks for that – do appreciate your readiness to engage.
          Have to confess the two of you have left me behind and most GPs I would guess, which seems a problem for evidence based medicine. Someone has to say what the evidence means and if you two, with your hugely impressive grip on the data and rules of evidence, can be so totally at odds over the implications of that data,, who decides what the evidence actually shows?
          Seems like whoever has hold of the microphone.

          Professor Newman quickly replied so here is another installment. I suspect it won’t be the final word but it seems to me to be a valuable and honest acknowledgement of the subjective element in what is normally portrayed a purely objective issue. From empiricism to epistemology.

          I have to confess my own bias here which is that I too share Professor Newman’s suspicion of the way much of medical data is presented and I feel he does a valuable job in exposing just how misleading it can be.

          Perhaps I didn’t emphasize this enough but the strange thing is that we’re not disagreeing on almost anything meaningful. He is not disputing the benefit numbers, only the harms. He disputes the muscle pain assertion (the least important harm, I’d say) on the grounds that if you can’t see differences between placebo and statin groups reported in industry-sponsored trials then he doesn’t believe the muscle pains and problems exist.

          Everything else he seems to agree with, although he wants to argue strenuously about a 0.9% difference in our estimates of how often statins cause diabetes (despite the fact that with either estimate diabetes remains a more common side effect of statins than any benefit).

          Bottom line is that the argument here is ideological, which is what readers should, in a perfect world, understand. We are so close to perfect agreement on fact and numbers that the differences are nominal. But he wants to characterize statins as beneficial because he believes people should take them. He wants to accept and believe industry data on their face. He wants people to be excited about statin therapy.

          I, on the other hand, want people to be skeptical of any therapy because I believe most modern medical therapies have been largely presented and offered in a fashion that ignores or minimizes the harm side and mischaracterizes (ie exaggerates) the benefit side. Industry data is famous for this, but it’s true for most therapies regardless of source. The use of The NNT highlights this history no matter how one interprets the balance of benefit//harm. NNTs piss people off, it’s a common reaction, because it’s an agnostic presentation, not the rosy version that most doctors use in the exam room, and that most supporters and believers are used to and expect.

          Generally speaking how many doctors or patients know and can explain out loud that for statins, or BP drugs, or daily aspirin, even for the highest risk person there is roughly a 99% chance that they will see no benefit? How many truly know how to prioritize diet vs exercise vs pills to keep them healthy? How many have a true and informed sense of the potential adverse effects of the pills they take? How many can place the expense and inconvenience and risk of the pills in the context of the potential benefit?

          Many of these therapies are quite reasonable choices and if needed I would consider every one of them, but ideologically he and I are different. I want informed discussion and debate, starting from a position that is balanced in its presentation, he wants to recruit people to one side. Anything else makes him conspicuously angry.

        • Editorial

          I did put up a reply to this post from Professor Newman last week but had to remove it because of formatting problems. Essentially it involved three different typefaces – to indicate who is making the points – and something went wrong and they were all lost leaving an unreadable wodge. Anyway am trying again – unfortunately settings don’t allow cutting and pasting copy with all type faces in place.

          Submitted on 2015/04/29 at 29 Apr 2015 | In reply to Editorial.
          Many thanks for posting the response but I still think that Professor Newman is seriously wrong about statin drugs:
          “I encourage anyone to examine this (http://bit.ly/1Iinrv7), the most comprehensive available review on the topic of muscle damage (myopathy) among those taking statins. The paper’s authors are transparent about definitions, and differences in approach. We agree with their definition of myopathy because it is patient-centered—it includes what patients experience (pain, weakness, exercise intolerance, etc.), not just what can be measured in a CPK assay.”
          Exaggerating the side effects of potentially life-saving medications is not “patient-centered”. It’s actually harmful. In the same way you could call every allergy an anaphylaxis or bradycardia a cardiac arrest. By calling benign symptoms such as muscle pain or stiffness a myopathy/muscle damage you are clearly misinforming the patients.
          A patient-centered approach makes efforts to see issues from the perspective of the patient. We will leave it readers to decide whether we have accomplished this, and whether our summary exaggerates or misrepresents side effects. Note, however, that for the use of statins in patients at high risk we list the same side effect profile yet classify the intervention as ‘Green’, because we believe that the benefit value trumps the harm value.

          “The commenter here is absolutely correct that these estimates are from observational data. Please see our ‘Philosophy’ page, see heading “Adverse Effects Data”, where we explain that from our inception we have preferred observational data for estimates of adverse effects, again because this is patient-centered. Patients typically want to avoid adverse effects whether they are due to placebo or active ingredient.”
          Sorry, but that’s just silly. So if in observational study 2% of patients taking a statin have a traffic accident, we should count this as an adverse event regardless of causality and advice people to not take the medicine? This type of reasoning would eliminate every single drug we use, because most people experience some kind of “side effect” while taking a placebo pill.
          As you can see in our Philosophy section, here, we do not agree. We also do not list side effects that have no biological or plausible relationship to an intervention. Indeed most of the side effects we list come from RCTs (because this is often the only source available) despite the fact that RCTs notoriously under-report and underestimate them. There is a long and important history of poor and unreliable reporting of adverse effects (one important reason we and the FDA feel compelled use alternatives). RCTs performed with the harm endpoints as co-primary endpoints, and properly defined, are often adequate. This, alas, is rare, and statin data has often been used as the case-based example of how AE reporting has been deficient. We do not trust the industry database, knowing the foibles of the included trials and their methods, to be the informant for AEs in statins. Of course, it is entirely an individual decision how to treat these data. We try hard to make these choices transparent, and allow the arguments to continue, as they have here so that readers can decide what they would want were they the patients.

          “The FDA agrees, using almost exclusively observational data for post-marketing surveillance and safety decisions. If side effects occur that is a problem for patients, whether the context is a randomized trial or real life.”
          I seriously doubt that FDA would agree with your opinion. FDA recognizes the limitations of the observational data. For example, in VAERS (Vaccine Adverse Event Reporting System) one can read: A report to VAERS generally does not prove that the identified vaccine(s) caused the adverse event described. It only confirms that the reported event occurred sometime after vaccine was given.
          Just because something happened during therapy doesn’t imply that the therapy is to blame. Causality is crucial for FDA’s decisions to withdraw a drug.
          Causality data would be wonderful to have. It is rarely available, as noted above, for AEs reported from RCTs. This is the primary reason that drug withdrawals have become much more common in the past 10 years than in the preceding decades. Contrary to the implications above observational data have often trumped RCT data for FDA-mediated withdrawals, and the lack of adequate or reliable reporting of AEs in the original trials has consistently been a point of broad regret and retrospective hand-wringing in these withdrawal cases.
          “Note also, please, that even Rory Collins has conceded (http://dailym.ai/1ziYM88) the adverse effects from statin RCTs have been under-reported and are as yet unreliable. “

          Professor Collins did not say that. He said that his previous meta-analyses of cholesterol trials didn’t investigate all side effects and in the light of public concerns it should be done by his team. Collins never stated that adverse effects in RCT have been underreported and unreliable. He believes that RCT proved that statins are safe and he performs the analysis solely because of aggressive anti-statin movement that has recently emerged.

          It seems we can all agree: even the most zealous advocates of statins feels that the current state of evidence on statin AEs is inadequate.

          “Indeed most large studies perform ‘run-out’ periods where those who do not ‘tolerate’ the drugs are excluded, as are patients with renal and other diseases, making RCT estimates unusable for projections about real life use of statins.”

          Many statin trials did not use any “run-out” periods and included patients with renal diseases or other chronic conditions. See for example AURORA, CORONA, SHARP. They also found statins to be safe, contrary to your statements. For example I strongly encourage you to take a look at data on muscle side effects in CORONA trial. No difference between rosuvastatin and placebo (225/2514 vs 207/2497). But according to your “philosophy” we should count all 225 patients with muscle pain as experiencing side effects from treatment. By the way, the rate in the placebo group (8,3%) is surprisingly close to your observational estimate.

          Please re-read all three studies cited: AURORA, CORONA, and SHARP are all industry funded trials that explicitly used pre-enrollment run-out/run-in periods, and all use very strict exclusion criteria that remove the great majority of patients with significant chronic conditions other than those defining the target inclusion criterion. The lack of myopathy findings in one such trial is not broadly reassuring on the issue of statins and adverse effects.

          “Finally, we used 10% as our estimate, at the most conservative end of the published estimate range (9-20%).”

          Irrelevant. All these estimates are baseless and misleading.

          “We extrapolated the diabetes data from the 2013 Cochrane review (see Analysis 3.7 on p72) to match the 5-year endpoint reported for all benefits. The benefits we use are from meta-analyzed data, therefore these cannot be extrapolated forward by selecting out the JUPITER trial data in the fashion suggested above, because the median 5-year time periods for benefit reporting have already done this extrapolation, usually by regressing the results against a time variable.”

          Sorry, but you did the same for the diabetes data. You took out the JUPITER trial from the meta-analysis of diabetes outcome and extrapolated the results to 5 years. So I can’t see why you couldn’t do it for benefits e.g. non-fatal heart attack.

          The outcomes for benefit in the Cochrane and virtually all other major statin reviews constitute a dataset with a median follow-up of 5-6 years, typically 5. There are two trials reporting new onset diabetes in the Cochrane review, AFCAPS/TexCAPS and JUPITER. The former is a 5.2 year trial and the latter is a 1.9 year trial. JUPITER is also three times the size (over 17K subjects compared to roughly 5K for AFCAPS). In order to convert this combination of two trials into an estimate of new onset diabetes risk during the same 5 year duration that benefits potentially occur (i.e. a patient-centered estimate that allows a side-by-side estimate of benefits and harms over a single time period) one needs to extrapolate the diabetes data forward.

          “(Note also the JUPITER trial was stopped early, a maneuver well known to exaggerate, not attenuate, reported benefits, thus extrapolating these data forward, even if one could do this, would be deeply misleading).”

          FDA determined that premature discontinuation of JUPITER trial had minuscule effect on the reported benefits. Look at page 11 in this document:
          http://www.accessdata.fda.gov/drugsatfda_docs/nda/2010/021366s016StatR.pdf

          There isn’t much debate on this topic: early stoppage exaggerates potential benefits. Moreover, when JUPITER is placed in the context of 115K more statin subjects from trials the median time periods equalize out to 5 years. As above, it would be both unnecessary and misleading to ‘correct’ for this in one such trial. In the case of harm data, as above, it is necessary to perform the correction outlined.

          “The diabetes data, however, are separated out using only the two trials for which the Cochrane authors had direct access to adverse effects reporting, making it possible to extrapolate to a 5-year endpoint, which we did.”

          Thanks to these researchers: http://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2809%2961965-6/abstract we now have access to many other trials and we can evaluate the diabetogenic effect of statins more precisely. Statin therapy increases your chances of diabetes by 9%

          There are a number of groups that have attempted to quantify the diabetes risk increases. Unfortunately these data are difficult to ascertain, as most trial publications do not report new onset diabetes. The true increase is unclear to us, which is the reason we have classified the possibilities as being within a credible interval (rather than a confidence interval which would suggest statistical precision).

          Anyway, the calculations you provided are completely erroneous. Let’s do it once again. Here are the numbers in Cochrane report:
          JUPITER Statin: 270/8901 Placebo: 216/8901 OR=1.26 AFCAPS/TEXCAPS Statin: 72/3304 Placebo: 74/3301 OR=0.97, which in summary gives OR = 1.18
          Now let’s extrapolate the numbers from the JUPITER trial to 5 years: Statin: 675/8901 Placebo: 540/8901. Odds ratio remains essentialy the same – 1.27.
          Then we combine the data from JUPITER with AFCAPS/TEXCAPS: Statin 747/12205, Placebo 614/12202. This gives odds ratio OR= 1.23 (1.10-1.37) not 1.81 as you claim and translates to NNH = 91 and ARI = 1.1%.

          These numbers look correct to me, though we will have to go back and formally review the sources and original calculations from our review. If correct we will be happy to adjust. If the numbers calculated here are correct, it also seems they fall squarely within the credible interval we describe (0.4-4%) and ultimately represent a 0.9% total difference between our published point estimate and that described here. Also note that, regardless of point estimate used, the likelihood of developing new onset diabetes due to a statin appears numerically similar to, or larger than, the likelihood of avoiding a nonfatal CV event. This again supports the notion that transparency in these numbers can hopefully facilitate more informed patient and physician choices about using statins in this population.

          If my calculations are OK, I trust that you correct your errors publicly. In summary, I think that you seriously misrepresent statin safety evidence and that your “patient-centered” position is in fact extremely harmful.
          And just in case: I do not have any financial conflict of interest to report. I’m a student of pharmacy but (unfortunately) I do not receive any funding from pharmaceutical companies.
          Outstanding.

  • wow thanks so much for the well-articulated responses, Dr Newman. You have learned to be considerate and not attacking and that attitude (with good arguments) will win over lots more ppl to become skeptical

  • I watched Dr Newman’s Ted talk on NNT and it makes so much sense.

    After watching this talk I visited my GP who recommended blood pressure medication. I asked her what the NNT was? She had no idea, but NICE said medication and that was enough for her. I said no thank you.

    As for statins, do doctors care about informed consent? Does Adam truly believe a patient would take statins if told he had a 1 in 100 chance of benefitting? The figure is more usually quoted at 140 – 300. And with a possibly greater chance of harm? Who would give informed consent to take these drugs?

    CTT operates much like FIFA. Prof. Sir Rory Collins and Sepp Blatter become ever more synonymous.

  • For Adam T

    Many of my patients have come off statins since I became a bit more enlightened about drug company scams. The most common cause was simple muscle pain. The CPK wasn’t raised so Sir Rory would have dismissed that as not significant.
    Most of these patients have told me how amazing it was to be free of pain that they had suffered for years, simply by stopping their statins. Many years of mild pain can grind you down just as effectively as severe pain, and make your life arguably more miserable.

    When informed that they might possibly live from -9 to +10 DAYS longer, they are understandably outraged. [BMJ Open June 2014]

    Try seeing it from a patient’s point of view.

Leave a Reply


WP-Backgrounds by InoPlugs Web Design and Juwelier Schönmann