When Hurricane Maria hit Puerto Rico last fall, it killed power to most of the island. The toll was catastrophic. Damages have been estimated at $90 billion. For months, entire cities had no electricity, sanitation, and access to health care. Which is why the government’s official death toll from the storm—64 people—always seemed suspect.
On Sunday, a team from Harvard confirmed those suspicions. In a study published in the New England Journal of Medicine, they showed that the hurricane killed more than 4,600 – 70 times the government’s number.
Why the difference? How can one entity count 64 people and another 4,600? When what they’re counting is something as important as dead American citizens? It turns out counting disaster deaths is harder than it sounds.
To be fair, the government and public health researchers used different definitions for what counted as a “hurricane death.” The government only counts those deaths it can attribute directly to Maria. The academics looked for something called excess deaths: the total number of deaths beyond what should have happened if there’d been no hurricane. To get at this number, researchers don’t even require a complete count of deaths, they just do a survey.
In an ideal world, when disaster strikes, we’d always know exactly how many people died and what from. However, that’s an ideal we never get. As the government response in Puerto Rico shows, it’s just not realistic to expect officials combating a hurricane to accurately track all individual deaths and find each root cause. And sometimes, getting to an individual root cause is impossible. In the wake of a natural disaster, more people die from almost every cause you can think of. Certainly people drown, but rates of stroke also increase. Rather than trying to tease out if a given stroke was caused by the hurricane, an excess death estimate simply notes how many more there were than would have been expected.
And that’s an important number. Humanitarian relief efforts targeting survivors need to know how many died. Such estimates can also be crucial for analyses of the economic economic and social effects of disasters. Basically, estimating excess deaths is an imperfect but practical solution to an awful problem.
To come up their estimate, the researchers conducted a survey. Puerto Rico contains 900 barrios. Researchers divided the barrios into eight categories based on how close they were to a city, and then sampled barrios from each category and households in each selected barrio. Not everyone agreed to be interviewed, but in the end they got responses at 3299 houses. Much the way a random digit dial telephone poll lets researchers estimate Americans’ political views, this process let the researchers use just a small number of households to estimate the number of deaths in the entire population of Puerto Rico.
What they found was a death rate of 14.3 deaths per 1000 people per year – six points higher than during the same time period the previous year. On an island of just over 3 million, that means an extra 4,645 Puerto Ricans died in the three months between September 20 and December 31, 2017.
Excess death estimates are usually larger than counts of individual deaths. But not 70 times larger. The difference between 64 and 4,645 can’t be explained by the use of different definitions.
Puerto Ricans are clearly concerned. The Washington Post noted in a story last Sunday,
The Center for Investigative Journalism in Puerto Rico is suing for access to the mortality data and the Puerto Rico Institute of Statistics (a politically independent agency) has begun an independent review, and plans to use subpoenas to get the death count.
That institute has corrected government statistics in the past; however PRIS is facing its own crisis. The government of Puerto Rico has been trying to eliminate the independence of the statistical institute since January. Puerto Ricans have resisted the move, and the American Statistical Association (of which I’m a member) and 46 other international scientific organizations have publicly opposed the government’s efforts. Eliminating the independence of a statistical agencies is always a bad idea, at least to those who believe in independent facts. However, this would be particularly terrible time: Puerto Rico needs good numbers now more than ever.
Robin Mejia runs the program in statistics and human rights at Carnegie Mellon University’s Center for Human Rights Science.
What error bars would you put around that factor of 70 difference, or why wouldn’t you? The analogy to polling makes me think quantifying the uncertainty would be apt.