CDC’s top disease modeler courts controversy

Atlanta • Last fall, when Martin Meltzer calculated that 1.4 million people might contract Ebola in West Africa, the world paid attention.

This was, he said, a worst-case scenario. But Meltzer is the most famous disease modeler for the nation's pre-eminent public health agency, the Centers for Disease Control and Prevention. His estimate was promoted at high-level international meetings. It rallied nations to step up their efforts to fight the disease.

But the estimate proved to be off.

Way, way off.

Like, 65 times worse than what ended up happening.

Some were not surprised. Meltzer has a lot of critics who say he and his CDC colleagues have a habit of willfully ignoring the complexities of disease outbreaks, resulting in estimates that over-dramatize how bad an outbreak could get — estimates that may be skewed by politics. They say Meltzer and company also overestimate how much vaccine is needed and how beneficial it has been.

Overblown estimates can result in unnecessary government spending, they say, and may further erode trust in an agency that recently has seen its sterling reputation decline.

"Once we cry wolf, and our dire predictions turn out not to be the case, people lose confidence in public health," said Aaron King, a University of Michigan researcher, who in a recent journal article took Meltzer and others to task for making what he called avoidable mistakes.

Meltzer, 56, is unbowed. "I am not sorry," he said.

He dismisses his peers' more complicated calculations as out of touch with political necessities, telling a story about President Lyndon Johnson in the 1960s. Johnson was listening to an economist talk about the uncertainty in his forecast and the reason a range of estimates made more sense than one specific figure. Johnson was unconvinced.

"Ranges are for cattle," Johnson said, according to legend. "Give me a number."

Meltzer does not shy away from providing a number.

Calculated risk • What Meltzer does is not particularly glamorous. He and others use mathematical calculations to try to provide a more precise picture of a certain situation, or to predict how the situation will change. They write equations on chalkboards, have small meetings to debate which data to use, and sit at computers. Meltzer spends a lot of time with Excel spreadsheets.

But modelers have become critical in the world of infectious diseases.

Epidemics often have a "fog of war" aspect to them, in which it's not clear exactly what just happened or what's about to happen next. That's true both of common infections and rare ones.

Take flu, for example.

Each winter, flu is so common that it's impossible to test and confirm every illness. It's also difficult to determine every flu-related death — it's often not clear flu was responsible for the demise of everyone who had flu-like symptoms when they died. So when the CDC cites an average of 24,000 flu-related deaths in the U.S. each year, that comes from modeling, not an actual count.

Ebola is another example. CDC leaders came to Meltzer early last August, when the epidemic was spiraling out of control and international health officials were quickly trying to build a response. Meltzer was asked to project how bad things could get if nothing was done, as well as to estimate how stepped-up aid could bend the curve of the epidemic.

Meltzer and his colleagues created a spreadsheet tool that projected uninterrupted exponential growth in two countries, Liberia and Sierra Leone.

His prediction — published last September — warned that West Africa could be on track to see 500,000 to 1.4 million Ebola cases within a few months if the world sat on its hands and let the epidemic blaze.

About 21,000 cases materialized by mid-January — a terrible toll, to be sure, but also just a tiny fraction of the caseload Meltzer and his CDC colleagues warned about. Today, the epidemic is considered to be on its last legs.

No modeler claims to be 100 percent correct. Indeed, modelers have a saying: "All models are wrong, but some are useful."

They mean that a model's mathematics can be correct, but the resulting predictions can still prove to be terrible if the wrong kinds of data are used or key assumptions are off. Unexpected intangibles, like a change in the weather, can also mess things up. (Of course the math can also be wrong — as in a ballyhooed 2004 CDC estimate of how many Americans die annually from obesity. It later proved to be over-inflated, with officials blaming a computational error.)

During last year's Ebola crisis, the World Health Organization made its own set of projections for the epidemic's course, released at about the same time as the CDC's. But the WHO chose to project cases only as far out as early November, saying 21,000 people could be infected in Guinea, Liberia and Sierra Leone by then.

Also, the WHO decided not to make a key assumption Meltzer did — that Ebola cases were being under-reported by a factor of 2.5.

Did Meltzer blow it? Many say no. He and his colleagues clearly stated they were providing a worst-case scenario of how bad things could get. They also predicted a far lower number of cases if more help was sent — which already was happening when the model estimates were released.

He tries "modeling" • Meltzer was born in 1958 in Southern Rhodesia, a British colony in Africa — a white, Jewish boy growing up in a privileged enclave in a country that was 99 percent black. Drafted into the military at 18, he went on reconnaissance missions in the Zambezi Valley during the later stages of a civil war that led to 1980 elections that brought independence and created the nation of Zimbabwe.

His early scientific interest was in the health of animals, not humans. He earned a degree in agricultural economics in Zimbabwe, then wrote a doctoral dissertation at Cornell University on control of tick-borne diseases in African livestock. He joined the CDC in 1995, when disease modelers were still a tiny group on the margins of public health.

Influential position • There is no doubt some envy among modelers for the influence Meltzer holds. Modeling-produced numbers become valuable currency in debates about what public health measures to take and what programs to fund; they can drive policy decisions.

Many modelers go into the field because "it has real implications you can see in your lifetime," said Irene Eckstrand, who until last year was scientific director of a government-funded modelers network called MIDAS — Models of Infectious Disease Agent Study.

CDC is supposed to prepare America for the worst, so it makes sense for CDC modelers to explore extreme scenarios. If Meltzer's estimates push policymakers to bolster public health defenses, it's all to the greater good, they say.