What does success look like for the Covid-19 vaccine effort?

A version of this story first appeared in D.C. Diagnosis,  STAT’s weekly newsletter about the politics and policy of health and medicine. Sign up here to receive it in your inbox.

WASHINGTON — Administration of Pfizer-BioNTech’s Covid-19 vaccine officially began on Monday. It was a joyous kickoff to a massive, arduous effort to vaccinate over 100 million Americans. But how will we know if the vaccination effort is actually going according to plan?

STAT posed that question to a handful of experts in the fields of public health, emergency preparedness, and bioethics. From our conversations, one thing was clear: Judging the success of a mass vaccination campaign is not as simple as making sure the number of people being vaccinated keeps on increasing. Here are a few answers that stood out to us.

How convenient is the process?

Convenience will be a big factor in determining how successful the vaccination effort will be, multiple public health experts told STAT.

There’s a lot of potential pain points when it comes to a vaccination campaign, from long-lines to anger over wanting a vaccine and not being able to get it. All of those issues must be managed. After all, nightly news stories about long lines or frustrated groups being denied vaccines could hurt the government’s credibility and dissuade people from ever getting vaccinated.

“I’m going to be watching to see if the average person who gets their vaccination finds that this is no big deal — no different from getting their flu shot, no different from getting their tetanus shot,” said Georges Benjamin, executive director of the American Public Health Association. “We will know we won if that’s the case.”

How many shots are sitting on the shelf?

A growing delta between the number of doses delivered to a state and the number of doses being administered could be a sign of multiple red flags, multiple experts argued.

It could suggest logistical snafus, like a state struggling to administer doses on schedule, or doses being wasted due to errors in preparation or handling, argued Eric Toner, a senior scholar at Johns Hopkins Center for Health Security.

Doses sitting on the shelf could also be an early warning sign that Americans are reluctant to get the vaccine, argued Alison Bateman-House, an associate professor and bioethicist at NYU Langone Health.

How is the general public talking about vaccine safety?

Michael Osterholm, the head of the University of Minnesota’s Center for Infectious Disease Research and Policy and a member of President-elect Biden’s Covid-19 task force, told STAT he will be watching to see how the government manages the tendency for the general public to attribute unrelated health emergencies to adverse vaccine reactions.

“Any time you have an effort like this, people will associate any possible health event that occurs … as likely being vaccine associated,” he said. Osterholm underscored that if 10 million people aged 55-64 were vaccinated today, nearly 800 would have heart attacks and another 700 would have strokes within a week, all unrelated to the vaccine, just based on general statistics for that population.

Are people still wearing their masks?

No one knows whether the Pfizer-BioNTech vaccine prevents individuals from spreading Covid-19, but public health officials are already worried that Americans who are vaccinated will nonetheless, inappropriately, throw their masks in the trash. Benjamin of APHA and Michael Fraser, the executive director of the Association of State and Territorial Health Officials, highlighted the vaccination campaign’s ability to keep people in masks even after they get vaccinated as one of the metrics they deem essential for judging its success.