Big data and “data communities” are key in crisis response and prevention

Nishant Shah, Analyst, Government Technology

Using connection technologies to facilitate response to humanitarian emergencies has become a well-established and rapidly evolving practice by governments, international organizations, non-profit organizations, community groups, and individuals. Using such technologies in the preventionof violence and catastrophe has been more difficult, owing to lack of political will, operational gaps between receiving warnings and mounting responses, and the difficulty of measuring the results of prevention efforts in the face of more austere budgets.

The first generation of network technologies used in crisis response were primarily designed and deployed top-down by governments. The generation that followed added technical, bottom-up hackers and volunteer groups, such as Ushahidi.

The next wave is being influenced by a broader humanitarian technology community of practice and the growing popularity of open source approaches. These embrace open source tools, put a premium on effective visualizations and maps, and use crowdsourcing techniques to improve response and enable prevention. Such approaches will increasingly require skills in sentiment analysis, analytics, and artificial intelligence, as well as a more granular understanding of the hallmarks of specific “data communities” that form before, during, and after events.

Big Data is useful for both structural and operational crisis prevention

Thanks to the “Internet of Things,” a growing body of data is being collected which tracks the movement of people, the operation of things, and the state of the environment. With emerging open source technologies, such data – often considered “Big Data” because of its volumes, varieties (in structure), and velocity (its speed of occurrence and processing) – can now be analyzed in reasonable timeframes, allowing rapid response.

Furthermore, much of this data is becoming publicly available and accessible through APIs; it can be regarded as “digital breadcrumbs” that point to the occurrence of larger phenomena.

Emmanuel Letouzé, Patrick Meier, and Patrick Vinck – experts and practitioners in the humanitarian technology space – concentrate specifically on conflict prevention using Big Data in their essay Big Data for Conflict Prevention: New Oil and Old Fires. They distinguish between structural (medium to long term) prevention and operational prevention (conflict early warning/response systems and diplomacy), and Big Data’s ability to uncover new insight into these endeavors.

For structural prevention, their major use cases include the following:

  • Understanding migration patterns via remote sensing, mobile phone data, and email data.
  • Monitoring community concerns and stresses using social media such as Twitter.
  • Studying human dynamics in informal and hard-to-reach settlements, such as slums, to inform interventions and poverty reduction programs.

For operational prevention, surveillance is the major use case. Examples provided in the essay include using public health early warning systems, using satellite imagery to reveal war crimes and mass movements, and detecting rising tensions using sentiment analysis, for example in Kenya’s 2013 presidential elections or during potential ceasefire violations in Syria’s civil war.

Not all crises are conflict-related, however; for disaster-related crises, the physical sensor component is of utmost importance in detection and tracking, with the digital “data exhaust” of humans particularly useful in response and assistance.

Each crisis has three major data communities

Ovum posits that each crisis situation has three major information imprints, or “data communities” of spiking incoming data.

The first is pre-event data, characterized by everything from average seismograph readings to Twitter chatter and status quo machine-to-machine (M2M) communications. Small changes in pre-event data can indicate upcoming shifts from the norm.

Analyzing past pre-event data can help responders understand which types of data are important to continually monitor and which characteristics to watch out for; the key is in knowing which data trails are the most important to monitor. UN Global Pulse, for example, is using SAS analytics to conduct research aimed at understanding online text content in Indonesia related to food and fuel prices, to see if it can indicate or predict trends in the Consumer Price Index (CPI).

The second is data that is collected during an event – this is the spike that is traditionally focused on, as it provides the information necessary for effective tactical response and resource allocation. This data must be collected, analyzed, and distributed in realtime to be effective, and, in some cases, prompts for data that cannot be collected by physical sensors or passive data scraping, such as the specific locations of stranded individuals in floods, must be pushed to the public to fill in data gaps.

As the Google Crisis Response team notes, three types of information are particularly necessary for individuals: what is happening; where are the resources they need; and where are the people they care about.

The third is post-event data, which represents the gradual shift back to the status quo. This type of data is particularly important during recovery. Social network analysis, for example, can be used to better understand how volunteer networks are forming and where help is still needed.

Those in the crisis response and prevention community of practice should evaluate past disaster data and collect more future disaster data to ensure they have a rich trove of information to analyze. Understanding the three data communities in specific geographic, cultural, and demographic contexts will go a long way in helping to build resilience to disaster and conflict.

The hallmarks of each data type in each community may change during these phases – including mobile phone data, social media sentiment, satellite imagery, and mobility/migration patterns – and understanding these patterns of change can help us to anticipate, prevent, and react to events and rebuild after they occur more effectively.

Related Stories

Leave a comment


This will only be used to quickly provide signup information and will not allow us to post to your account or appear on your timeline.