Data problems hurt California’s COVID-19 vaccine distribution efforts

Omitted doses, uploading errors, lag times and software mishaps. California’s vaccine rollout has been plagued by data issues, leaving the state unable to keep track of how many doses of the lifesaving COVID-19 vaccine are available at any one time.

The implications are far-reaching: Gov. Gavin Newsom has pushed to speed up inoculations, in part because the state’s data appeared to show vaccine providers were sitting on doses, prompting the governor to threaten to take supplies from those not moving quickly enough. Now county officials say they are worried the data accuracy issues will cause future allotments to be curtailed based on flawed conclusions from faulty figures.

“We’ve been pointing out that their data is bad since the end of December,” said Fresno County Supervisor Ernest Mendes.

After being pushed to make public more information on the progress counties were making in their vaccination efforts, the state published a dashboard to “make vaccine data transparent and accessible to all Californians.”

But the dashboard was so riddled with errors — including displaying a handful of doses from counties in Arizona and other states — that Kat DeBurgh, executive director of the Health Officers Assn. of California, said she initially told the state it should be taken down. Officials in smaller California counties also reported that the dashboard was drastically underreporting doses for them.

“When it first went up it was extremely inaccurate,” DeBurgh said, “but [the dashboard] looks much better now.”

The accuracy of the state’s vaccine data overall is improving, which has helped California raise its national ranking for doses administered, but the effort has often been a manual and painstakingly slow process, said California Department of Public Health spokesman Darrel Ng.

And the work is not done. Aimee Sisson, Yolo County public health officer, told lawmakers during a hearing Wednesday that uploading errors had caused the state’s database to undercount the vaccines administered in that county by nearly 30%.

“Neither the number received nor the number administered being reported by the state matches what we know to be true in Yolo County,” Sisson said. “Yolo County staff have been working with a state contractor to troubleshoot the discrepancies, to no avail.”

With California warning providers that a failure to administer vaccines fast enough could result in the state taking back doses, Sisson emphasized that the data the state relied on needed to be accurate.

“If the state uses its current data to determine future allocations,” Sisson said, “Yolo County could be penalized for what the state sees as an administration rate of 51%, but is actually an administration rate of 74%.”

The data problems have made it appear that counties have been slow to get doses out to the public, Sisson said, prompting the state to look for new ways to speed up administration. Newsom said the state will sign a contract with Blue Shield of California to overhaul the way the state allocates vaccines and to improve data collection. That contract is expected to be released this week.

Some county officials have expressed concerns about retooling the state’s vaccine delivery system, saying the issues are with the state’s data collection, not with how doses are distributed.

“The system isn’t broken,” Sisson said. “It just looks like it is because doses being administered aren’t showing up.”

Part of the issue with the state’s data is the large number of vaccine providers entering information using different types of software. At times, the errors and delays have been the result of the software itself, some county officials have said. Other times the state’s immunization registry indicates the dose information has been uploaded, but then fails to update the totals, according to county officials.

“I’m hearing from multiple local health departments in the greater Sacramento region that they’re experiencing the same issue with missing doses [in the state registry],” Sisson said during the hearing. Dr. Paul Simon, chief science officer for the Los Angeles County Department of Public Health, added that “we’re seeing the same thing here in Los Angeles.”

The Urban Counties of California legislative advocacy organization wrote in a letter to Newsom this month noting that its member counties were “significantly challenged by the lack of accurate data at the state level.”

Last month, Newsom touted how far the state had come in increasing daily vaccination numbers, but said the “two-, three-day data lag ... is killing us in terms of some of the national numbers.”

California’s significant vaccine data collection problems were first exposed when Newsom told Californians to “hold me accountable” to a goal of administering 1 million doses of COVID-19 vaccine in 10 days. After that deadline passed, state officials said coding errors and data lags made it difficult to say whether Newsom met the goal.

The state’s pandemic response has been hampered in other ways by poor or outdated data systems, including the August discovery that a public health computer database failure rendered numbers unreliable and prompted second-guessing about actions taken to stem the spread of the coronavirus.

Facing criticism from county officials about the persisting vaccine data issues, Dr. Tomás Aragón, director of the California Department of Public Health and state’s public health officer, told lawmakers during Wednesday’s hearing that there is an “incredibly intense focus” on “cleaning up the data, improving the quality.”

“We have to improve the data,” Aragón said.