Clinical trials – getting it right

Astrazeneca reported a failure in Phase III clinical trials for their non-small cell lung carcinoma (NSCLC) candidate Selumetinib, which was being tested for its use in combination with docetaxel. This comes in the wake of a prior failed PIII trial of  selumetinib with dacarbazine for metastatic uveal melanoma (a form of eye cancer). Given each PIII clinical trial is estimated to cost 30-40 million USD, which makes up 60% of all clinical trial costs, that is an incredible amount of money to be investing on something with a chance of coming to naught.

Just how many of these trials are failing you ask? From the FDA website, PI trials have a success rate of 70%, PII trial success is reportedly lower at 33% and the success rate of a PIII trial ranged from 25-30%.

What are the reasons for failure? There are worryingly too many but I list a key few here:

  • A mouse is not a little man. Its a problem scientists face all the time, poor translation from animal models to humans. Numerous drugs have worked in mice but failed disastrously when put in humans.
  • Heterogeneity of disease. NSCLC is a great example. It is actually made up of ten different mutation-specific diseases so patients tend to have rather varying disease etiologies making them harder to treat with the same approach. The PIII success rate for NSCLC is just 26.1%!
  • Poor trial design. Designing a trial is not a walk in the park. Choosing the right patients to test, randomizing treatments, blinding treatment, getting the dose right, having good end-points and surrogate measures are all factors that can severely affect trial  outcome. See Dr Richard Chin’s blogpost for a good overview.
  • False discovery rate. Performing a large number of measures on a small number of patients (oftentimes the case in PII trials) has a tendency to increase the number of false positives. E.g. If 10 tests are performed per patient in a trial with 100 patients (i.e 1000 measurements), there will be 50 positive results that are actually false (if a significance level of 0.05 is used). The FDR is a correction measure proposed by Benjamini and Hochberg (1995) – it looks only at significant results and calculates the proportion of false positives within this (see here and here for more info). Though it reduces the level of false positives overall, it still lets in a small proportion of false positives which may increase when using too many or inappropriate measures. This of course may create the impression that a drug appears to be working when it is not (which is later found out in the more highly-powered PIII trial)
  • Risky re-positioning. The success rate of drugs in trials for diseases which they were not designed to treat, i.e non-lead/secondary indications, are often lower (about 1.5-3X less likelihood of approvals). This is often due to institutional bias where less time and effort go into monitoring patient selection or establishment of scientific rationale.

These problems are not easy to solve but at least the last 3 points are immediately addressable.

To counter poor trial design , adaptive clinical trial design is now gaining popularity as it allows for modification of the treatment regime/dose and patient selection/sample size based on observing the results as they come out. They also allow early trial terminations when the drug is seen to be futile, which could save lots of money. However, there is an inertia to adopt this due to logistic/operation concerns, so only an estimated 20% of clinical trials are following adaptive design, but hopefully this grows.

To reduce the FDR, we should adopt more stringent significance criteria, e.g. a p value < 0.001. It also helps to look at what the multiple measures are telling you as a whole, rather than singling out one which gives a statistically significant result. Testing a hypothesis that is very unlikely to be true tends to increase the number of false positives, so it is advised not to include this in your battery of tests. Benjamini has also now come out with a new weighted FDR approach, that assigns importance to the measured endpoints which enforces better control on the overall error rate.

Finally, the repositioning of drugs for non-lead or secondary indications is something pharma has to closely regulate. Though it makes sense to use drugs that have cleared all the initial safety hurdles and have well-established pharmacokinetics, how suited it is to treat another disease should be properly established. Costly, failed clinical trials are known to bankrupt biotechs and set pharmas well-behind in the development of other drugs that would otherwise have created significant impact in the lives of patients. That said, there have been several cases of successful repositioning  – thalidomide previously a sedative is now used for pain/inflammation in leprosy, dapoxetine an anti-depressant apparently also works for premature ejaculation. There are a growing number of biotechs focussed solely on repositioning, with several targeted strategies. This presents a new market niche altogether which might improve the great waste we currently see in drug development.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s