Why do two papers on SURMOUNT-1 report different headline figures?

This is a question about interpretation, not about whether to act — I will take action questions elsewhere.

This is one of those things that everyone repeats and nobody derives.

This matters practically, not just academically, because it changes what I would do next.

Is the standard explanation correct, and if so, what is the evidence for it?

Answer first: read the primary endpoint, the comparator and the population before you read the effect size. Almost every argument on this site about a trial is really an argument about one of those three.

Placebo arms in this class are not nothing. Lifestyle-intervention placebo arms in the major obesity trials commonly lose two to three per cent of body weight, so an active-arm figure quoted without its comparator overstates the drug effect by roughly that much.

Non-inferiority and superiority designs are not interchangeable. A non-inferiority result says the new agent is not meaningfully worse against a pre-specified margin — it does not say it is as good, and it certainly does not say it is better.

One qualification: absence of a signal in a trial of this size is not evidence of absence for a rare event. It is evidence that the event is rarer than the trial could detect.

Quote the interval alongside the estimate and half the disagreements on this site would not start.

Before comparing two trials, check whether they share an endpoint definition. Frequently they do not, and the numbers then are not comparable in any sense.

Duration decides what can be seen. A 68-week trial can measure weight and glycaemia; it cannot measure anything whose event rate is one per cent per year without enrolling tens of thousands.

Trial populations are selected. Exclusion criteria in this class routinely remove people with significant renal impairment, prior pancreatitis and unstable psychiatric illness, which is exactly the population the results are then quoted for.

I am not a clinician and this is not medical advice; it is a reading of a published protocol.

Read the protocol and the statistical analysis plan if the result matters to you. Both are usually published alongside.

Look at the discontinuation rate alongside the efficacy figure. A large effect in the two thirds who stayed is a different result from a large effect in everyone.

Confidence intervals matter more than point estimates when two trials disagree. Two studies reporting fifteen and twenty per cent whose intervals overlap heavily have not disagreed about anything.

Meta-analyses in this area are dominated by whichever trial contributed the most participants, so read the forest plot rather than the summary estimate.

The caveat is that trial evidence is about licensed product administered under supervision. None of it transfers automatically to research-grade material of unverified content.

If a claim cannot be traced to a named trial with a named endpoint, treat it as a claim rather than as evidence.

edited 13 May 2024 by tabular_nums — clarified the distinction between purity and content

The trial answers a narrower question than the headline suggests, and the narrowing is where the useful information is.

Intention-to-treat and per-protocol analyses answer different questions. ITT asks what happens if you offer the treatment; per-protocol asks what happens if it is taken as directed. The gap between the two is a measure of how tolerable the protocol was.

The cardiovascular outcome programme in this class runs to several large randomised trials — LEADER for liraglutide, SUSTAIN-6 and SELECT for semaglutide, REWIND for dulaglutide — and they are the reason the class is discussed as more than a weight intervention.

When two sources disagree, the answer is almost always in the methods section of the one you have not read.