A masterclass in selection bias
The original study was a pair of multicentre, randomised, double-blind, placebo-controlled phase 3 trials investigating the safety and efficacy of daridorexant in adults with insomnia disorder, published in The Lancet Neurology in 2022. The headline finding was that daridorexant improved both night-time sleep measures and daytime functioning compared with placebo.
Top banana.
The study ticks all the boxes we are taught to look for. Randomised. Double blind. Placebo controlled. Published in a prestigious journal.
Unfortunately, it is also one of the best examples of selection bias I have come across in a long time.
Background
Insomnia is a nightmare.
It can eat away at every aspect of your life. When patients present with it, they are usually exhausted, frustrated and desperate for a solution.
Back in the day I was trained that there was no simple fix for insomnia. Good sleep habits, behavioural interventions, addressing underlying causes and accepting that there was rarely a magic pill.
Apparently I got that wrong.
NICE has now approved daridorexant, a drug that blocks orexin, one of the chemicals in the brain involved in keeping us awake. It arrives with plenty of excitement and plenty of marketing.
Why is this relevant to General Practice?
GPs sit on the frontline of insomnia management, and it often feels like the problem is getting worse.
We live in a hyper-stimulated society where work follows us home in our pockets. Social media is full of people with suspiciously perfect teeth telling you to wake up at 4:30am and seize the day. If you don’t have a side hustle, and instead enjoy going the park with your daughter, expect some grifter to shout “DON’T WASTE A SECOND, BRO. YOU ONLY GET ONE SHOT AT THIS.”
If you throw in energy drinks, a crumbling economy and perpetual vaping, it is hardly surprising that people struggle to switch off.
Don’t worry though, if a societal problem exists, there is always Super Pharma ready to put on their cape and solve the day
Who is going to be making money from this?
Daridorexant is manufactured by Idorsia. A typical three-month prescription costs around £126 based on NICE pricing.
Daridorexant is also one of the company’s few realistic “blockbuster” products. Unlike highly specialised medicines prescribed to relatively small numbers of patients, this is a drug aimed at a condition affecting millions.
Given that, it perhaps isn’t surprising that I receive a steady stream of invitations to educational dinners. Maybe they’re trying to rescue me from a never ending schedule of play dates?
And speaking of potential influences, the declarations of interest section is Proustian.
Is the population similar to mine?
In a word, no.
The study excluded many of the patients we see every day in General Practice.
Patients with significant psychiatric illness were excluded (this included those on antidepressants).
Patients with important chronic medical conditions were excluded.
They were also recruited through specialist sleep centres rather than General Practice.
Think about how many patients presenting with insomnia in your surgery have anxiety, depression, chronic pain, diabetes, COPD, cancer, heart disease, or are taking antidepressants. Now think about how many don’t.
This trial was not conducted in a typical GP population. It was conducted in a carefully selected population with relatively uncomplicated insomnia.
That matters.
What did the study find?
The primary outcomes included Wake After Sleep Onset (WASO) and Latency to Persistent Sleep (LPS).
After three months:
- Patients receiving daridorexant spent approximately 18 fewer minutes awake during the night compared with placebo.
- Patients fell asleep approximately 12 minutes faster than those receiving placebo.
Those results were statistically significant.
Whether they are clinically significant is a different question.
Critical appraisal
This is where the real lesson lies.
Selection bias occurs when the patients included in a study are not representative of the patients to whom the treatment will ultimately be prescribed.
The more selective your population becomes, the more likely you are to demonstrate a treatment effect.
The problem is that those impressive results may not translate into routine clinical practice.
The question is not:
“Does daridorexant work in carefully selected patients attending specialist sleep clinics?”
The question is:
“Does daridorexant work in the exhausted fifty-year-old sitting in front of me with anxiety, chronic pain, type 2 diabetes, three repeat medications and a teenager who keeps coming home at 2am?”
This study does not really answer that question.
In fact, even in this highly selected population, patients only experienced around 18 fewer minutes awake during the night and around 12 minutes less time getting off to sleep compared with placebo.
Interestingly, the placebo group improved as well.
A useful reminder that symptoms often improve with time, expectation, support and natural recovery.
The takeaway
The critical appraisal lesson is simple.
Whenever you read a trial, don’t start with the results.
Start by looking at who was excluded.
Because before asking whether a treatment works, you should first ask:
“Are these patients anything like mine?”
This is essentially a very expensive medication, that wasn’t studied in your average GP patient and even then, only demonstrated minimal improvement.
Leave a comment