Due to a mix of scarce resources, methodological difficulties and poor incentives, much of the data collected on the world’s poor is either inaccurate or missing. That’s one of the findings of a study being released today by the Overseas Development Institute (ODI) at the Cartagena Data Festival in Colombia.
While the trend of falling poverty is genuine, the ODI think that numbers in poverty may be being understated by up to 350 million, more than the entire population of the US.
Take estimates of the rate of maternal mortality: Most might assume that the data are collected by keeping a record of every woman who dies in childbirth and comparing it to the total number of women who give birth but the reality is very different.
Estimates are created for most countries using a regression model, which uses gross domestic product per head, the fertility rate and how likely births are to be attended by someone skilled as predictors.
This is a reasonable strategy for gathering statistics where many women are giving birth outside hospitals and states lack consistent registration systems for births and deaths. But it does mean that the estimates come with huge bands of uncertainty. In Nigeria, for example, the report says that while the maternal mortality ratio was estimated at 560 deaths per 100,000 women in 2013 it could have been anywhere between 300 and 1000.
This is just one example of the problems listed in the report, there are myriad others. Largely the problems stem from sampling bias – the errors that occur when a sample non-randomly excludes some particular portion of the population.
As researchers find it much harder to contact the poor, those living in conflict zones, the homeless, informal migrants and other disadvantaged groups, they are likely to be excluded from many of the surveys that researchers rely on, thus underestimating poverty.
But this isn’t the only problem. Most national statistics agencies lack capacity. Data is infrequently collected – with many big surveys carried out only every three to five years. Many important statistics aren’t collected at all — the ODI report lists the number of people living in cities, girls married before the age of 18 and the number of people living in cities as all examples of basic facts that we still don’t know.
To further complicate things, incentives are skewed by political interference and the actions of donors. The ODI highlight the cases of Myanmar’s census and poverty reduction in Pakistan. In Myanmar the government decided that the Rohingya minority would not be allowed to self identity and so many of the census takers would simply move on to the next house whenever someone identified themselves as Rohingya. In Pakistan, the ODI say, there is continuing debate over whether or not the government has underweighted food in measures of inflation thereby exaggerating people’s real income and understating poverty.
Big foreign donors can sap the agencies’ capacity even further as staff at statistical offices are hired out to carry out surveys on a consultancy basis, earning more than they could from doing government work. In other cases, for example school enrolment, data might be exaggerated in order to increase grants and budgets.
However, the report argues that much can still be done. Elizabeth Stuart, the lead author, says “Even when people are counted, the counting is not good enough. We need better surveys, better use of the data we have, underpinned by more investment so governments know the size of the problem, and how well they are tackling it.”
Including using big data. For example, satellite technology to track proxies for economic development like light emissions or smartphone use so that those in remote areas could send data instantly to a central office.
All this sounds a bit over optimistic about technology in the face of major political constraints and the lack of basic infrastructure so it remains to be seen whether any significant changes will be made. However Ms Stuart says “From Columbia, Kenya, to India, we’ve heard of data being harnessed to help citizens with everything from growing crops, to mobile banking, to claiming land rights. The data revolution isn’t in the future, it’s already here.”