Explainer – Shot in the dark: Early COVID-19 vaccine efficacy explained

By John Miller

ZURICH (Reuters) – This week has seen a flurry of good news from COVID-19 vaccine developers, with Pfizer Inc. and BioNTech SE trumpeting early data indicating their mRNA candidate is more than 90% effective.

A Russian project came out a day later, touting 92% efficacy for the Sputnik V candidate, named after the Soviet-era satellite, based on a smaller data set.

HOW DO MANUFACTURERS ARRIVE AT EFFICACY NUMBERS?

In Pfizer’s case, it waited until 94 volunteers in its late-stage clinical trial of more than 43,500 people – half got the vaccine, the other half got a placebo – tested positive after developing symptoms.

For 90%-plus efficacy, no more than eight people among those who tested positive had received the vaccine, with the rest having received the placebo.

“Roughly speaking, it’s probably around eight to 86 cases in the treated and placebo groups,” David Spiegelhalter, a Cambridge professor of risk and an expert in statistics, told Reuters.

“You don’t need a lot of fancy statistical analysis to show that this is deeply impressive. It just hits you between the eyes.”

In Russia, Sputnik V-developer Gamaleya Institute reached its preliminary 92% efficacy figure based on 20 illnesses in 16,000 volunteers as its late-stage trial progresses. It aims to reach 40,000 people.

Of the 16,000 people, about quarter got the placebo.

“It suggests that there is some effect, but it’s insufficient to estimate the magnitude of it,” Spiegelhalter said.

HOW MANY PEOPLE MUST GET SICK IN BIG VACCINE TRIALS?

Some experts say that, ideally, 150 to 160 people in a trial of tens of thousands of participants must get sick before making a reliable assessment of a vaccine’s efficacy. That’s a bit of a rule of thumb, though, open to interpretation.

“There is no such regulatory standard requiring X number of events for making a reliable decision,” the government-funded Swiss Clinical Trial Organization said. “The amount of (infections) has to be seen in relation to the disease and its risk profile. It’s rather a case-by-case evaluation.”

Typically, regulators strive to have at least 95% certainty that the trial read-out is not the result of random variations with nothing to do with the tested compound.

For trial sponsors there is safety in numbers as a large enough trial can ensure that 95% reliability hurdle is cleared. But the larger the underlying clinical benefit, the fewer trial participants needed to create that clarity.

In Pfizer and BioNTech’s trial, they planned a final analysis when 164 people had become sick, with multiple, pre-planned interim analyses along the way. They skipped an analysis at 32 patients, and once they were ready to release a look at the 62-person mark, 94 had come down sick.

Details from the Russian trial are unclear, without access to its protocol.

HOW DO THESE RESULTS STACK UP TO OTHER DRUGS, OR VACCINES FOR OTHER ILLNESSES?

In normal drug trials, for diseases like terminal cancer, benefits of new medicines may be less apparent, with survival benefits of just a few months sometimes revolutionary for patients at death’s door.

For vaccines, however, marginal protection is inadequate, and the World Health Organization ideally wants to see at least 70% efficacy in trials, while the U.S. Food and Drug Administration wants at least 50%.

The 90% efficacy reported in the Pfizer and Russian trials beats those, and appears to exceed that of typical flu vaccines, which the U.S. Centers for Disease Control and Prevention (CDC) estimate reduce the risk of sickness by 40%-60%.

For other shots, the CDC estimates the efficacy of a two-shot measles vaccine at 97%, and a two-dose chicken pox vaccine at 90%. Two doses of polio vaccine are 90% effective, rising to nearly 100% with a third.

CAN WE EXPECT EFFICACY RATES TO HOLD UP AS TRIALS ADVANCE?

Pfizer acknowledged on Monday that its final vaccine efficacy percentage may vary. Still, Spiegelhalter said the study’s design seems likely to generally hold up, based on the 94 sick participants.

“In this case, the effect is so huge, even if there is a little bit of fallback – if the effects become slightly smaller over time – that is very unlikely to be significant.”

WHAT ABOUT REAL-WORLD EFFICACY, SHOULD THE VACCINES BE APPROVED?

The interim data is promising, since it appears to demonstrate that a vaccine can be effective in preventing COVID-19.

The jump to mass vaccinations, however, presents new hurdles, in particular for an mRNA vaccine like Pfizer and BioNTech’s that must be stored and shipped at minus 70 degrees Celsius (-94°F).

Moreover, the Pfizer-BioNTech vaccine requires two doses, ideally 21 days apart. If people do not stick to the timetable, it may affect the vaccine’s efficacy.

Protection against the mumps, for instance, drops from nearly 90% to 78%, if people don’t get a follow-up shot.

Swiss epidemiologist Marcel Tanner, president of Switzerland’s Academies of Arts and Sciences and one of the government’s top COVID-19 science advisers, expects possible variations in efficacy among older people, whose immune systems wane with time, or those with immune disorders.

“Efficacy says, ‘Does it work?’ Effectiveness says, ‘Can it be applied? Can you carry the efficacy to the people?'” Tanner said. “But no question: 90% efficacy, at that stage, is a pretty good result.”

(Reporting by John Miller in Zurich, Kate Kelland in London, Ankur Banerjee in Bengaluru, Julie Steenhuysen in Chicago and Polina Ivanova in Moscow; Editing by Josephine Mason and Nick Macfie)

Russia says its Sputnik V COVID-19 vaccine is 92% effective

By Polina Ivanova

MOSCOW (Reuters) – Russia’s Sputnik V vaccine is 92% effective at protecting people from COVID-19 according to interim trial results, the country’s sovereign wealth fund said on Wednesday, as Moscow rushes to keep pace with Western drugmakers in the race for a shot.

Russia’s results are only the second from a late-stage human trial, following on swiftly from data released on Monday by Pfizer Inc. and BioNTech, which said their shot was also more than 90% effective.

While experts said the Russian data was encouraging and reinforced the idea the pandemic could be halted by vaccines, they warned that the results were only based on a small number of trial volunteers who had contracted COVID-19.

The analysis was conducted after 20 participants developed the virus and examined how many had received the vaccine versus a placebo. That is significantly lower than the 94 infections in the trial of the vaccine being developed by Pfizer and BioNTech.

“I assume there was political pressure after the press release from Pfizer and BioNTech earlier in the week to now draw level with their own data,” said Bodo Plachter, deputy director of the Institute of Virology at the Mainz University. “What is missing for now is an analysis of statistical significance.”

To confirm the efficacy rate of its vaccine, Pfizer said it would continue its trial until there were 164 COVID-19 cases.

The Russian Direct Investment Fund (RDIF), which has been backing Sputnik V’s development, said the Russian trial would continue for six months.

Alexander Gintsburg, director of the Gamaleya Institute which developed the vaccine, said the interim results demonstrated that Sputnik V was effective and mass vaccinations would be rolled out in Russia in the coming weeks.

European stocks and U.S. stock futures extended their gains slightly after Russia’s announcement though the reaction was far more muted than after Pfizer’s results.

China’s Sinopharm, which is running large-scale late-stage clinical trials for two COVID-19 vaccine candidates, said on Wednesday that its data was better than expected, though it did not give further details.

‘NOT A COMPETITION’

Successful vaccines are seen as crucial to restoring daily life around the world by helping end the pandemic that has killed more than 1.26 million people, shuttered businesses and put millions out of work.

However, experts said knowledge about the Russian trial’s design was sparse, making it hard to interpret the data.

Scientists have raised concerns about the speed at which Moscow has worked, giving the regulatory go-ahead for the shot and launching mass vaccinations before full trials to test its safety and efficacy had been completed.

“This is not a competition. We need all trials to be carried out to the highest possible standards and it is particularly important that the pre-set criteria for un-blinding the trial data are adhered to avoid cherry picking the data,” said Eleanor Riley, a professor of immunology and infectious disease at the University of Edinburgh.

“Anything less than this risks a public loss of trust in all vaccines, which would be a disaster.”

The results are based on data from the first 16,000 trial participants to receive both shots of the two-dose vaccine.

“We are showing, based on the data, that we have a very effective vaccine,” said RDIF head Kirill Dmitriev, adding that it was the sort of news that the vaccine’s developers would talk about one day with their grandchildren.

The so-called Phase III trial of the shot is taking place in 29 clinics across Moscow and will involve 40,000 volunteers in total, with a quarter receiving a placebo shot.

The chances of contracting COVID-19 were 92% lower among people vaccinated with Sputnik V than those who received the placebo, the RDIF said.

That’s well above the 50% effectiveness threshold for COVID-19 vaccines set by the U.S. Food and Drug Administration.

The RDIF said data from the study would be published in a leading medical journal following a peer review. The results of the early-stage Russian trials were peer reviewed and published in September in The Lancet medical journal.

Experts said that as with the Pfizer results, it was not yet clear how long immunity would last after taking the Russian vaccine, nor how efficient it would be for different age groups.

“We certainly need longer-term observations to draw valid conclusions about efficacy and side effects. The same goes for Pfizer’s and BioNTech’s numbers,” said Plachter in Mainz.

As Moscow seeks partners abroad to boost output, China’s Tibet Rhodiola Pharmaceutical Holding announced a deal soon after the results to released to make, sell and test the shot in China.

SPUTNIK V

The Russian drug is named Sputnik V after the Soviet-era satellite that triggered the space race, a nod to the project’s geopolitical importance for Russian President Vladimir Putin.

Russia registered the vaccine for public use in August, the first country to do so, ahead of the start of the large-scale trial in September.

So far, it has inoculated 10,000 members of the public considered at high risk of contracting COVID-19 such as doctors and teachers, outside of the trial.

The vaccine is designed to trigger a response from two shots administered 21 days apart, each based on different viral vectors that normally cause the common cold: human adenoviruses Ad5 and Ad26.

The Pfizer and BioNTech vaccine uses messenger RNA (mRNA) technology and is designed to trigger an immune response without using pathogens, such as actual virus particles.

Russia is also testing a different vaccine, produced by the Vector Institute in Siberia, and is on the cusp of registering a third, Putin said on Tuesday, adding that all of the country’s vaccines were effective.

RDIF said as of Nov. 11 no serious side effects had been reported during the Sputnik V Phase III trial.

Some volunteers had short-term minor adverse events such as pain at the injection site, flu-like syndrome including fever, weakness, fatigue, and headache, it said.

In late October, the vaccination of new volunteers was temporarily paused due to high demand and a shortage of doses.

Russia’s deputy prime minister said on Wednesday that the Vector Institute vaccine was expected to start post-registration trials on Nov. 15.

She also said that Russia would produce 500,000 doses of Sputnik V in November, lower than a previous forecast of 800,000 doses given by Trade and Industry Minister Denis Manturov.

Russia reported 19,851 new coronavirus infections in the past 24 hours and a record high of 432 deaths. At 1,836,960, its overall case tally is the fifth largest in the world, behind the United States, India, Brazil and France.

(Reporting by Polina Ivanova; Additional reporting by Kate Kelland, Ludwig Burger, Josephine Mason and Thyagaraju Adinarayan; Editing by David Clarke)