Statistical Risk Assessment Results 2016 Map —Data from the Early Warning Project
Last month, we shared the results of our Early Warning Project’s latest Statistical Risk Assessment (SRA)—a list of 163 countries ranked by their risk for onset of state-led mass killing.
As we’ve taken our results on the road, we’ve found that we are commonly asked some variation of this question: This is all very interesting, but what am I supposed to do with it?
Our answer: Through this modeling approach, we seek to assess which governments are most likely to systematically kill more than 1000 noncombatant civilians in any 12-month period. The results provide an overview of worldwide risk, and are presented as a ranking so that policymakers, NGOs, and governments can prioritize and distribute resources effectively. (For more details on why forecasting is an effective policy tool, see this report from the Atrocity Forecasting Project).
But how exactly can one use the risk assessment to prioritize? And what to do when the statistical assessment is at odds with country experts’ judgments?
We see the statistical results not as the conclusion, but the starting point for discussion and further analysis. We see it as an interactive thought process, combining statistical results with other kinds of analysis to provide the most accurate and useful assessments. This post will explain how we interpret results, providing examples from our most recent assessment.
When we consider the results of the SRA, we first look at four types of countries:
Top risks
Surprises
Big Movers
Consistently medium- to high-risk
Top risks
As with all country rankings, one’s attention is first drawn to the highest ranking countries. From the 2016 results, for example, Sudan and Burma/Myanmar ranked among the three highest risk countries for the third year running. The first way to use the SRA results is to ask, Is there adequate attention on the risk of mass atrocities in the countries at the very highest risk of mass killing?
We consider all countries in the top 30 to be high risk, but note that “high” in this assessment is relative. Mass killings are low likelihood events and our highest risk countries this year are around 7%. While mass killings are unlikely, they are so horrific and destructive that it is worth taking steps to prevent them.
Surprises
Normally, the top 10 countries in our statistical risk assessment tend to confirm people’s intuitions: mass killings are most likely in countries with ongoing civil wars and/or mass killings. But sometimes those just outside of the top 10 can come as a surprise. For countries whose ranks are surprisingly high or surprisingly low, key questions are: What accounts for the discrepancy between the statistical results and experts’ expectations? What additional analysis would help shed light on the level and nature of atrocity risk in the country?
In this year’s results, Bangladesh is ranked 16th. To many, Bangladesh is seen as a success story. Its economy is on the rise, with sustained increasing per capita income, rising women’s employment, low inequality, high financial inclusion, etc. It does well on development indicators including life expectancy and access to water and sanitation and is moving quickly towards middle-income status as a country.
Despite its economic progress, our risk modeling points to several attributes of Bangladesh’s politics that are associated with mass killing risk: growing authoritarianism, sharp and sometimes violent competition between national political factions, and discrimination against vulnerable minority populations. Discussion with country experts have emphasized longstanding feuds between political parties, increasing extremism, and security forces that have committed abuses with impunity. This led us to choose Bangladesh as a focus of more extensive, field-based research on the scenarios that could lead to mass atrocities, in collaboration with a country expert working as an Early Warning Fellow. This is just one example of how the SRA can draw attention to issues in a country that are outside the “normal” center of focus.
Big Movers
We also look at how much countries move up and down in the rankings from year-to-year. While some input data, such as infant mortality or ethnic fractionalization, stay relatively consistent year to year, others, such as a coup attempt or onset of civil war, can cause a country to jump in the rankings. Key questions for big movers are: What events or changes explain the big shift in estimated risk? Have there been additional events or changes, not yet reflected in the data, which are likely to further shift the risk? Does the increase or decrease indicate an ongoing trend?
Burundi is the most significant mover in this year’s SRA, jumping from 56th to 6th. This dramatic shift reflects a failed coup attempt, a slide back into more openly authoritarian rule, and the resumption of violent civil conflict. A United Nations inquiry concluded in September 2016 that while open violence has declined from 2015, repression in other forms is more systematic and is increasing.
In more positive news, Burkina Faso moved the opposite direction, going from 16th to 32nd. Though it still bears watching, consultation with experts indicates that this decrease appears to reflect the reality in country. We will look to the next SRA to determine if this is a downward trend or an aberration.
Consistently Medium- to High-Risk
Finally, we look for countries that are consistently medium- to high-risk. The key questions for these countries are largely the same as others: Is there adequate attention on the risk of mass atrocities in the countries at the very highest risk of mass killing? If the risk is seen as surprisingly high, what accounts for the discrepancy between the statistical results and experts’ expectations?
If a country ranks relatively high for multiple years running, we take this as a strong indication that the models are detecting real, significant risks, not just some statistical peculiarity. One example is Mali, which has ranked 9th, 13th, and 8th in the three years of this project. Starting with this assessment, we consulted multiple country experts, all of whom suggested the risk for violence in Mali is increasing, though they mostly didn’t see the state as the main threat or expect killings to reach the 1,000 person threshold. At the same time, conversations with the policy community indicated that attention to Mali comes almost entirely from a counterterrorism perspective, which we see as related but distinct from an atrocity prevention perspective.
This combination of consistently high statistical ranking; open analytical questions about the nature, scale and trajectory about the risk; and limited policy attention to the risk of mass atrocities led us to pursue deeper research on Mali, in which we will conduct field work to map out scenarios that might lead to mass killing (see our similar work on Zimbabwe).
In sum, we believe the SRA is a useful tool for identifying countries at high or rising risks of mass atrocities. Used together with other kinds of analysis, the SRA can help generate atrocity risk assessments and warnings necessary for effective preventive action. As we continue to conduct assessments we will refine our process, drawing on the latest developments in forecasting and mass atrocity assessment to improve our precision and accuracy.
If you have examples about how you have used the SRA, or questions about how you might, please contact us at (mailto: ewp@ushmm.org).