An analysis of the issuance of volcanic alert levels during volcanic crises

Abstract

Volcano Alert Levels (VALs) are used by volcanologists to quickly and simply inform local populations and government authorities of the level of volcanic unrest and eruption likelihood. Most VALs do not explicitly forecast volcanic activity but, in many instances they play an important role in informing decisions: defining exclusion zones and issuing evacuation alerts. We have performed an analysis on VALs (194 eruptions, 60 volcanoes) to assess how well they reflect unrest before eruption and what other variables might control them. We have also looked at VALs in cases where there was an increase in alert level but no eruption, these we term 'Unrest without eruption' (UwE). We have analyzed our results in the context of eruption and volcano type, instrumentation, eruption recurrence, and the population within 30 km.

We found that, 19% of the VALs issued between 1990 and 2013 for events that ended with eruption accurately reflect the hazard before eruption. This increases to ~30% if we only consider eruptions with a VEI ≥ 3. VALs of eruptions from closed-vent volcanoes are more appropriately issued than those from open-vents. These two observations likely reflect the longer and stronger unrest signals associated with large eruptions from closed vents. More appropriate VAL issuance is also found in volcanoes with monitoring networks that are moderately-well equipped (3-4 seismometers, GPS and gas monitoring). There is also a better correlation between VALs and eruptions with higher population density.

We see over time (1990 to 2013) that there was an increase in the proportion of `UwE’ alerts to other alerts, suggesting increasing willingness to use VALs well before an eruption is certain. The number of accurate VALs increases from 19% to 55% if we consider all UwE alerts to be appropriate. This higher `success’ rate for all alerts (with or without eruption) is improving over time, but still not optimal. We suggest that the low global accuracy of the issuance of VALs could be improved by having more monitoring networks equipped to a medium level, but also by using probabilistic hazard management during volcanic crisis.

Introduction

Mitigation of volcano hazards relies on the correct identification of unrest signals (Marzocchi et al. [2012]; Sparks et al. [2012]), and thus good monitoring networks and complete knowledge of the system. This allows scientists to assess the probability of an unrest episode leading to an eruption or of a return to background levels (Newhall and Hoblitt [2002]; Marzocchi and Woo [2009]; Lindsay et al. [2010]; Woo [2011]; Marzocchi et al. [2012]; Sparks et al. [2012]). It also requires an efficient means of communicating the information and interpretation of volcanic unrest to the government agencies for ensuring peoples’ safety (De la Cruz-Reyna and Tilling [2008]; Fearnley et al. [2012]). Volcano Alert Levels (VALs) are such a tool, using a simple color, letter or number code to report the level of unrest. Ideally the VAL should increase progressively before eruption or increase and then decrease where unrest does not proceed to eruption. In many cases, changes in VAL carry strong implications about the exclusion areas around a volcano and about evacuation and safe return of populations. However, decisions to change a VAL at any given volcano are fraught with uncertainty about the volcanic system itself, and also psychological and personal factors of the scientific decision-maker (e.g. Woo [2014]).

Given the importance of VALs for saving lives and property (Auker et al. [2013]), we decided to analyze their issuance worldwide. We have compiled a dataset of VALs and eruptions dates and we investigate what parameters play a role in a more appropriate issuance of VALs. For example: are VALs issued in a timelier manner at volcanoes where there is a denser monitoring network? Are the unrest signals easier to interpret at some types of volcanoes than at others, leading to more appropriate VALs? Do more frequent eruptions at a volcano lead to more training of staff and therefore more appropriate VALs? We investigate how the issuance of VALs varies with the volcanic system (size and type of volcano) as well as with some technical (e.g. monitoring network) and social (e.g. population at risk) factors. Finally we also discuss the role of the scientific decision-maker on the appropriate issuance of VALs.

Past and current alert level schemes

The eruption of Kelut in 1919 led to the creation in Indonesia of a `Volcano Watching Service’, which was at the time part of the Department of Mines (Neumann van Padang [1983]; Suryo and Clarke [1985]). The aim of the new service was to study how populations could be protected from volcanic eruptions and specifically considered volcano type, eruption history and volcanic unrest in order to warn and evacuate the population when necessary (Neumann van Padang [1983]). Observation `Posts’ were established at Merapi from 1934 to warn of imminent volcanic threats (Neumann van Padang [1983]). During the 1980’s there was a growing interest in formalizing eruption warnings in the framework of VALs. Although never formally published, one of the first of these was developed by the Rabaul Volcano Observatory (RVO) in response to the unrest in the caldera during the early 1980’s (Rabaul Volcano Observatory, Volcano Information Bulletin no.2, (1983); UNDRO United Nations, Office of the Disaster Relief Coordinator [1985]). When activity began at Mt St. Helens in 1980 it became apparent that the scheme in place (patterned after one used by the National Weather Service) did not capture rapid changes within the overall eruptive sequence. As an alternative, the USGS issued `factual statements’ -descriptions of current conditions and `extended outlook advisories’ that looked several weeks ahead (Newhall [1984]; Swanson et al. [1985]). There was also a concerted effort to communicate risk to the people who went back to work in the areas decimated by the lateral blast in May 1980 (Newhall [1984]). These events raised the need for comprehensive VALs.

The schemes that are in place today were designed in the late 1980’s and early 1990’s. Their creation was influenced by two significant events:

• 1. The encounter between a Boeing 747 bound for Anchorage International Airport and the ash cloud produced by the eruption of Redoubt (Guffanti and Miller [2013]). Although the crash was averted the event spurred the development of the aviation alert level scheme by the USGS scientists at the Alaskan Volcano Observatory (AVO) (Guffanti and Miller [2013]). The scheme was adopted by the Kamchatkan Volcanic Eruption Response Team (KVERT) in 1993 and was standardized across the United States in 2006 (Fearnley et al. [2012]; Guffanti and Miller [2013]).

• 2. The eruption of Pinatubo in the Philippines in 1991 (Tayag et al. [1996]). Escalating volcanic unrest preceded this eruption, and an alert level scheme was created and used to simplify communication between scientists and the local officials. The five-point scheme that was developed at Pinatubo was then extended to other volcanoes in the Philippines: Mayon, Taal, Kanlaon and Bulusan, though the VAL criteria developed by the Philippine Institute of Volcanology and Seismology (PHIVOLCS) are specific to individual volcanoes to take into account the different physical characteristics.

Further VAL schemes were developed globally through the 1990’s, and some have been simplified and standardized since then (Japan Meteorological Agency [2009]; Fearnley et al. [2012]).

Most VALs are country-specific and further subdivided by the repose time (frequently active/reawakening). We looked at alert level information from 11 different countries. Their schemes can be quite different, but we found that they broadly contain several or all of the following points:

• 1. Indicative Phenomena: A description of the precursory activity currently occurring at the volcano, possibly including a description of seismic, geodetic or gas changes.

• 2. Volcano Status: A quick, one-line description of the overall activity at the volcano, e.g. `Minor eruption in progress’, `Hazardous eruption in progress’ or `Volcano is in typical, background, non-eruptive state’

• 3. Recommended Action: Information on exclusion zones for a given specific activity as well as recommendations for evacuations.

• 4. Time Scale: Some indication of the potential time frame before an eruption, usually given in days, weeks or months.

• 5. Number or Color Scale: Either numbers or colors (or both) to communicate an escalating (or decreasing) volcanic hazard. When colors are used they are usually in the `traffic light’ scheme, where green is equated with normal conditions and red is the most dangerous level. Most numeric schemes link `0’ or `1’ to green and `4’ or `5’ to red, but some countries invert this, with `1’ being the highest level of unrest.

For all the volcanoes that we examined, the final level included the statement that `an eruption was imminent’ or that an `eruption was underway’.

Methodology

Data sources and treatment

Volcanological surveys issue VALs for local and national purposes. We retrieved these VALs from online archives of volcanological and geological surveys, the `Smithsonian/USGS Weekly Volcanic Activity Report’ and the Smithsonian Institution’s monthly `Bulletin of the Global Volcanism Network’, and internal hard-copy archives of some of the surveys in the study. We also drew supporting information from EM_DAT (Emergency Events Database) (CRED Network Website [2008]) and DesInventar (Disaster Information Management System) (UNISDR Website [2011]).

To determine which volcano observatories issue alert levels, we surveyed all of the volcanoes in the GVP database, recording the ones that had any weekly or monthly bulletins. We used reports of both volcano observatories and summarized monthly reports in the Smithsonian archives. We identified volcanoes that had activity after a given observatory began using VALs. Where we were unable to determine when a specific observatory began using alert levels, we took the date of the first record of any VAL at that volcano. This survey encompassed 133 volcanoes including 291 eruptions from 17 countries. However, many data sets were incomplete, so to ensure that the records were consistent with each other we only use cases where: (1) VALs had been established at the volcano before the eruption began, and (2) the records of VAL were continuous. We also excluded volcanoes that had no eruptions in the time frame of modern alert levels (e.g. since 1991) and volcanoes that were in continuous eruption for more than 10 years. After filtering by these criteria, our database includes 60 volcanoes and 194 eruptions from 11 countries: Chile, Colombia, Ecuador, Indonesia, Japan, New Zealand, Peru, Philippines, Russia, United States and Vanuatu. The eruptions occurred between 1 January 1990 and 31 December 2013. We note here that some countries with large numbers of volcanoes, such as Italy and Iceland, are not part of our study because either they don’t publicly issue VALs or else we were not able to retrieve their data. Our complete dataset can be found in the electronic supplement (Additional file 1).

Data classification

We made a timeline of the changes in VALs at each volcano and compared it with the start dates of eruptive episodes from the GVP. We plotted VAL data from 30 days prior to the onset of an eruption through to the end of the eruption. We assumed that any unrest within 30 days prior to eruption is related to the causes of the eruption. Using a longer time window (e.g. 60 days) does not change our results.

We tracked how the VALs changed with time relative to the onset of eruption. Our main tenet is that a VAL is appropriate if it progressively increases before eruption, or if it increases then decreases when an intrusion of magma does not erupt. We identified six possible VAL patterns (Figure 1): Category 1 has been termed `Missed’, where the onset of the eruption was not preceded by an increase in the VAL, and the VAL was not moved (and usually remained low) during the course of the eruption. In Category 2 (`Too late’) the VAL was increased for the first time after the eruption began (days to weeks) but before the eruption ended. Category 3 is for `Premature’ alerts, where the VAL was increased but subsequently decreased to lower levels prior to the onset of eruption. Category 4 is called `Almost’ because the VAL was increased prior to the eruption, but not to the highest level that indicates `increased potential of eruption’ or `high alert’. Often, the VAL was changed from the `Normal’ or `Background’ level to a higher level of unrest, but no further. Category 5 is called `Timely’ because the VAL was increased in a stepwise fashion and, at the point of eruption, was at an appropriately high level. Category 6 `UwE’ (Unrest without Eruption) refers to cases in which the VAL was increased, remained elevated and then was lowered without an eruption occurring within the next 30 days. We consider Categories 4 (`Almost’) and 5 (`Timely’) to be the most appropriate issuance of alert levels in cases where unrest led to an eruption, and Category 6 `UwE’ as appropriate when unrest did not lead to eruption. We have analysed reports from these episodes of unrest and found that the `UwE’ alerts recorded in this study are well correlated with the state of the volcano, and were thus appropriate. However, we have no way of counting the episodes of unrest that did not lead to an eruption that were not represented by an increase in the VAL. We therefore, cannot estimate the percentage of `UwE’ that were “missed” by the VAL systems.

Figure 1

Classification of the issuance of alert levels relative to eruptions. The VAL is plotted against time (number of days). The first 30-day window shows the time before the eruption onset. The red line denotes the onset of the eruption, the plot continues until the end of the eruption (in this example at day 60).

We also note that the eruption start dates in the GVP often refer to the onset of phreatic precursors to magmatic eruptions. VALs, on the other hand, are more likely to track magmatic ascent. To test whether reference to phreatic or magmatic onsets would change our characterization of VALs we did an analysis for larger eruptions, all clearly magmatic (VEI ≥ 3). Using the eruption reports we recorded the dates where magmatic activity can be assumed to have begun. This is defined as either the day that 1) Dome growth was recorded; 2) Pyroclastic flows were observed; 3) Eruption column reached 10 km or greater, or 4) the date of the maximum eruption column height. We then assessed whether there was any change in how the event would have been ranked. We found this effect to be negligible (only 1 case out of 40 would have been classified differently).

Data sources for level of monitoring

We gathered information on the monitoring network at specific volcanoes from the records that are kept by World Organisation of Volcano Observatories (WOVO) and by each observatory. To assess the level of monitoring we used a slightly modified scheme from that developed by Ewert et al. ([2005]), in the development of the National Volcano Early Warning System (NVEWS), where a numerical value was assigned to different degrees of volcano monitoring capacity. We focused on the seismic, deformation, gas and remote sensing capabilities of the network when we assessed the level of monitoring at the volcanoes in our study.

Level 0 denotes `No In-Situ’ monitoring; eruption confirmation is usually by remote sensing. We have eight volcanoes with this level of monitoring. Volcanoes with Level 1 or `Limited’ have no real time eruption data from ground-based sensors, but the volcano is within a regional seismic network. Eruption confirmation is still by remote sensing and there is a baseline inventory of satellite images available: five volcanoes from our sample have Level 1 monitoring. Level 2 network is considered `Basic’, and includes: 1 – 2 seismometers within 10 km of the volcano and any combination of: 3 continuous GPS or tiltmeters in the vicinity, repeated campaigns of gas measurements, and near-real-time images. There are 22 volcanoes that have this level of monitoring. Level 3 volcanoes have `Good’ monitoring, with 3-4 seismometers in the near field and any combination of: 6 seismometers within 20 km of the vent, least 6 continuous GPS in the vicinity and continuous or periodic gas monitoring. There are 21 volcanoes in our sample at this level. Level 4 or `Research Level Monitoring’ volcanoes include 12 - 20 stations within 20 km of the vent, sufficient continuous deformation stations to do detailed source inverse modeling and continuous gas monitoring. There are only four volcanoes in our study with this level of monitoring.

When we were in doubt about the level of instrumentation at a volcano we have awarded the higher ranking among the possible levels. Therefore our assessment can be seen as a measure of the maximum monitoring capability based on the information available. Because of the difficulty in accessing data about the changes in monitoring capability over time, we used the most up to date information. We have not been able to make an assessment of the staffing levels and capabilities at observatories, although we assume that there is loosely an increase in the number of staff and the level of training with an increasingly dense network.

Results

From our sample of 194 eruptions we find that about 80% of VALs fall into the categories `Missed’ (Category 1; 76 eruptions, 39% ) and `Too Late’ (Category 2; 76 eruptions, 39% ), about 14% as `Almost’ (28 eruptions), and only about 5% can be classified as `Timely’ (10 eruptions; 5% ) (Figure 2A). For the quantification of category `UwE’ we used the percentage of the number of `UwE’ from the total number of issued alerts (therefore excluding `Missed’). We found that overall 33% of alerts issued were `UwE’ (Figure 2B).

Figure 2

Overall results from the study. Pie chart (A) shows the relative frequencies of Categories 1 – 5. There is a large difference in the sample sizes (n). N values for categories are,; Cat 1: 75, Cat 2: 76, Cat 3: 5, Cat 4: 28, and Cat 5: 10. Pie chart (B) shows the proportion of `UwE’ alerts as a percentage Categories 2-6, i.e. of total alerts less Category 1 (`Missed’).

If we group the categories of `Timely’ and `Almost’ we find that only 19% of the alert levels were increased by any amount before the onset of eruption. We were surprised by this low percentage. Below we investigate what may be controlling the different categories by analyzing the results according to different parameters. First we look at the volcanic system: volcano type and eruption styles. Later we address the technical and societal factors that may be involved, including; the monitoring capacity, the effect of the number of eruptions within a system and the population at risk.

Volcano and eruption styles

We divided the eruptions according to the VEI (as assigned by the GVP) and found that in general there is an increase in the number of Category 5 (`Timely’) VALs as the eruption size increases (Figure 3). For VEIs ≤ 2 there is a high percentage of `Missed’ and `Too Late’ VALs (42% and 39% , respectively) and of the 154 eruptions, we see only 3% of the VALs in the `Timely’ category for these small eruptions. As the eruption size increases to VEI 3 the VAL’s in the `Missed’ category decrease (26% ), but those in the `Too Late’ increase (41% ). There is also an increase in the percentage of `Timely’ (12% ) and `Almost’ for eruptions of VEI’s < 2 to VEI 3. For VEI ≥4 eruptions the percentages in the `Missed’, `Too Late’ and `Timely’ categories are about the same (33% ), but the sample size (n = 6) is too small for statistical significance.

Figure 3

VAL results for eruptions with different VEI’s. Percentages of data are classified by categories 1 (`Missed’ in red), 2 (`Too Late’ in orange), 3 (`Premature’, in yellow), 4 (`Almost’, in light green) and 5 (`Perfect’, in dark green).

We also looked at the character of the volcano to determine if this would have any effect on the issuance of VALs. For this, we simplified our classification into: (1) Open-vent volcanoes that produce many frequent, small eruptions. They are often continuously degassing and tend to have more effusive eruption products. This continuous release of gas seems to make large eruptions less likely, unless the system seals itself (Newhall [2007]), (2) Closed-vent volcanoes have less frequent eruptions and are commonly dormant for decades, centuries, or longer. About 53% of the volcanoes in our database were classified as open-vent, and produced 76% of the eruptions, whereas the remaining 47% are closed-vent, and produced 24% of all eruptions. We found a higher percentage of `Timely’ and `Almost’ VALs (10% and 19% , respectively) in closed-vent (Figure 4) than in open-vent systems (3% and 12%, respectively). Open-vent systems also show a higher number of `Missed’ events (43% ) than closed-vent systems (22%). Finally the proportion of `UwE’ at open-vent volcanoes is 28% of the total number of alerts, but this proportion increases to 42% at closed-vent volcanoes.

Figure 4

Proportion of VALs issues at Open and Closed Volcanoes. Percentages of data set that are classified as Categories 1 – 5 following the color code for Figure 2. Pie Charts show UwE (Category 6, black) as a % of all alerts except `Missed.’ Data have been separated by volcano type.

Role of monitoring networks and equipment

We found that there is an increase of the `Timely’ and `Almost’ categories with the increase in the capacity of the monitoring network (Figure 5). Volcanoes with monitoring Levels 0 and 1 have a high number of `Missed’ and `Too Late’ VALs (88% and 80% , respectively for 13 total events). For volcanoes with Levels 2 and 3, we find that there is an increase in the `Timely’ and `Almost’ VALs from 14 (Level 2) to 43% (Level 3). The value of the `Almost’ category for monitoring Level 4 still increases to about 50% , although given that we only have 4 volcanoes in this category, it is not clear if the increase is statistically significant. At volcanoes with low monitoring levels, the `UwE’ alerts are a very high percentage of total alerts issued (72% for Level 0 monitoring) and then broadly decrease as the monitoring level increases (33% for Level 4 monitoring).

Figure 5

Effect of the level of monitoring on the issuance of VAL’s. Percentages of data set that are classified as Categories 1 – 5 following the color code for Figure 2. Pie Charts show UwE (Category 6) as a % of all alerts except `Missed’.

Effect of eruption frequency on the issuance of VALs

To test whether an increased knowledge of the system (e.g. a given volcano) increases the reliability of the issuance of VALs, we looked for an increase in the number of `Timely’ and `Almost’ VAL categories with an increase in the number of eruptions at a given volcano. In our sample, 34 volcanoes had more than one eruption over the time period and this filter still includes 161 eruptions. We found a large spread in data and no relationship between the number of eruptions per volcano and the percentage of `Timely’ and `Almost’ VAL categories (Figure 6). However, some pattern emerges if we consider frequency of eruption and level of monitoring. Volcanoes with frequent eruptions (mostly, open vent volcanoes) with the highest percentages of appropriate alert levels are those with monitoring levels 2 and 3 (Figure 7). The lowest percentages of appropriate VALs are for volcanoes that have fewer eruptions (mostly, closed-vent volcanoes), and a moderately high level of monitoring (level 2 or higher).

Figure 6

The effect of multiple eruptions on the issuance of appropriate VALs. Data are separated by Open (23 volcanoes, 135 eruptions) and Closed (10 volcanoes, 26 eruptions) vent volcanoes.

Figure 7

Controls on the issuance of VALs due to changes in monitoring at volcanoes that have had multiple eruptions. Color shaded fields group volcanoes values with similar monitoring capacity. A) For `Closed Vent’ volcanoes: Blue area represents 3 volcanoes, 9 eruptions all have monitoring level 2. Green area represents 5 volcanoes, 15 eruptions 86% of which have a monitoring level of 2. B) For `Open Vent’ volcanoes: Red area represents 4 volcanoes with 48 eruptions, 75% of which have monitoring level 0, blue area represents 11 volcanoes, 53 eruptions 73.5% of which have monitoring level 2, green area represents 6 volcanoes, 16 eruptions 75% of which have level 3(+) monitoring.

Effect of population on VALs

The purpose of VALs is primarily to inform local populations (and, in some cases, air traffic) of the potential for volcanic hazard. Local authorities also use them to inform decision making for evacuations. We checked to see if the size of population living around a given volcano affects the issuance of VALs. We used the population figures from the Smithsonian GVP, which were obtained from the LandScan 2007 data (Oak Ridge National Laboratory [2008]). We considered the population size within 30 km of the volcano, and normalized the data by the number of volcanoes within each population bin size (Figure 8A), to account for variable number of volcanoes per bin. We found that there are more `Timely’ alerts in the bin with the highest population, but in general there are similar proportions (within a few percentage points) for the `Timely’ and `Almost’ categories for all population sizes (Figure 8A), meaning that there is no significant correlation between appropriate VALs and population size. The proportion of `UwE’ varies between about 24 and 48% for all population sizes, without a clear change with population. As an additional measure to test the influence of the population on the decision to change VALs, we compared data from two countries with very different demographics. Country A has a large population living within 30 km of volcanoes, whereas Country B has much lower numbers of people (on average less than a 100) living close to its volcanic centers.

Figure 8

Effect of Population on VALs. A) Distribution of VALs issued for all volcanoes in the study compared to the population living within 30 km of the volcano. B) Data for Country A – high average population within 30 km. C) Data for Country B – low average population within 30 km. All percentages are normalized by the number of volcanoes in each bin. Percentages of data are classified as Categories 1 – 5 following the color code of Figure 2. Pie charts show UwE (Category 6) as a % of all alerts except `Missed’.

For Country A we found that for volcanoes with populations of less than 1 million people about 21% of the VALs fall into the `Timely’ or `Almost’ (Figure 8B), but this increases to 45% for volcanoes with more than 1 million people, so there is an improvement with population size. The highest proportion of `Missed’ events occurs within the population range of 1,000 – 10,000. The `UwE’ show a somewhat bi-modal distribution for Country A with peaks for the lowest (1,000 – 10,000) and the highest (>1,000,000) populations. In contrast, for Country B with sparsely populated volcanoes (less than 2,500 people living within a 30 km radius of the volcanoes) we see that even in the lowest populations (less than 1000) there are higher proportions `Timely’ and `Almost’ VALs (Figure 8C) compared to the highly populated Country A. But there is also an increase in appropriate VAL’s with population size. There is no dominant trend in the `UwE’ alerts that can be attributed to differences in population. The general decrease in `UwE’ alerts for Country B is more likely a factor of the small sample size, rather than a true trend.

Has the issuance of VALs improved over time?

Since 1990 there has been an increase in the number of countries and institutes that have adopted a form of VAL, from one in 1990 to 60 in 2013 (Figure 9). To be able to compare the results from different years we have normalized the data by the number of volcanoes. We find that there is quite a large variety from year to year and not a linear increase of any of the categories with time (Figure 10). Many `Missed’ and `Too Late’ VALs occurred prior to 2000 while from 2004 onwards most years have higher `Timely’ and `Almost’ categories (in general > 20% ). To better test whether the VALs are improving with time, we performed a moving average analysis of the `Timely’ and `Almost’ categories (Figure 10) which shows a complex pattern but a general improvement after 2003.

Figure 9

Distribution of data in study over time. A) Change in cumulative number of volcanoes over time. B) Change in cumulative number of eruptions over time. C) Change in cumulative number of alerts over time. D) Change in cumulative number of `UwE’ over time.

Figure 10

Change in VAL issuance in episodes that ended with eruption and those of `UwE’ over time. Percentages of VALs issued are normalized by the number of volcanoes included in the study year to year. Percentages of data are classified as Categories 1 – 5 following the color code of Figure 2. Blue line is the % of `UwE’ and the Black line is the 3 year running average of the `Timely’ and `Almost’ categories.

We also compared the number of alerts and number of eruptions over time normalized by the number of volcanoes and some trends can be identified (Figure 11). We find that from 1996 to 2004 the normalized number of eruptions is higher than the number of alerts, which reflects that many eruptions occurred without any alert being issued. However, starting from 2005 the ratio of numbers of alerts to eruptions is similar or higher than one, so that on average at least one or more alert was issued per eruption, something that is also reflected in the decrease in `Missed’ events over this time period (Figure 10). Thus, in general we find that with time there is an increasing number of VALs issued per volcano and per eruption that could reflect an increase in the monitoring network or change in the scientific decision making process as discussed below.

Figure 11

Total cumulative number of alerts issued (`Timely’, `Almost’,’Premature’, `Too Late’ and ` UwE’) and total cumulative number of eruptions in the study. Blue line is the ratio of the two data series. All data are normalized by the number of volcanoes included in the study year to year.

Discussion

We discuss how well VALs have reflected the state of the volcanoes, and two factors that may influence these results -- the physical nature of the system, e.g. the size of the eruption or the type of volcano, the level of monitoring, or the number of outcomes (eruptions) from given volcano, and roles of population and scientific decision-making process.

How well do VALs reflect the state of a volcano?

We consider three more specific, component questions:

• 1) What percentage of eruptions was anticipated by appropriate alert levels? This is an eruption-centric evaluation of VALs. We sum Categories 4 (“Almost”) and 5 (“Timely”) and divide by the number of eruptions, getting 19% , a disconcertingly low number. Clearly, much improvement is needed.

• 2) What percentage of unrest was anticipated by appropriate alert levels? Unfortunately, we cannot answer this question, as there are an unknown number of episodes of unrest for which no VALs were issued. In our experience, there are many slight elevations of unrest that are not tagged with VALs so this “missing” number might be significant.

• 3) What percentage of alert levels was appropriate to the status of the volcano? This is an alert-centric evaluation of VALs. We sum percentage of categories 4, 5, and 6, from the total sample of alerts. The result is 55% percent, higher than the result for eruption-centric evaluation but still rather low compared to what we expect of volcano observatories. Both the eruption-centric and this alert-centric `success’ numbers are kept low by the alarmingly high number of `Missed’ and `Too Late’ alerts.

Physical and technical parameters that affect issuance of VALs

One of the clearest trends that we have found is a general increase of `Timely’ and `Almost’ categories with VEI (Figure 3) which could be related to the difference in volcanic precursors and unrest signals for small versus large eruptions. VEI 1’s include small ash puffs and phreatic explosions that may arise from hot gas and rock interacting with ground water (such as the eruption of Mt Mayon in the Philippines in May 2013), which have few or no discernible precursors and thus no basis for raising alert levels. The larger and longer episodes of unrest that are typically associated with VEI 3 or 4 eruptions allow for more progressive increase of VALs. Larger eruptions are better anticipated than smaller ones. One example is Merapi, which had much larger unrest signals and a much larger and explosive eruption in 2010 compared to those in 2006 and other past eruptions (Surono et al. [2012]).

The more appropriate issuance of VALs at closed vents (Figure 4) is probably because magma needs to fracture rock or clear the conduit before it can reach the Earth’s surface. This may also explain why `UwE’ are more prevalent at closed systems than at open systems because the longer repose periods would lead to a greater need for re-establishing pathways within the conduit and may cause multiple episodes of unrest over an extended period prior to eruptions. The signals at these volcanoes may therefore be stronger and a higher number of `UwE’ can be expected due to several intrusions or episodes of magma ascent before an eruption is triggered.

We find that VALs at open vent volcanoes (Figure 7B) are improved by a higher number of eruptions and opportunities for the observers to be `trained’ by the events, and/or by a better level of monitoring that allows for subtle precursors to be recorded. As closed vent volcanoes have fewer eruptions, there is less opportunity for observers to experience volcanic crisis. Instead they need to rely more heavily on monitoring networks and well-constrained eruptive histories (both from detailed stratigraphic analysis and the careful review of historical documents). The lack of experience in closed-vent volcanoes or in volcanoes that reawaken after a long period of dormancy (e.g. Sinabung volcano in 2013) could be mitigated by the use of a worldwide database of volcano monitoring signals (Widiwijayanti et al. [2013]).

Another factor that significantly improves VALs is an increased level of monitoring (Figure 5). This finding could be anticipated as more data available to the volcanologist makes for better understanding of the system. However, the increase in the accuracy from Levels 0 to 1, or Level 1 to 2 is much less than from Level 2 to 3. This suggests that it is necessary to increase the monitoring network to at least 3-4 seismometers, and/or GPS and gas monitoring for an effective improvement of the appropriate issuance of the VALs. Such an increase is usually coupled with an increase in human resources, as more effort is invested in interpretation of the monitoring data. The results for monitoring Level 4 volcanoes are inconclusive because the sample size is only four volcanoes. It is also possible that an observatory with a monitoring level as dense as a Level 4 network would have a large volume of data from differing sources to reconcile before a decision could be made about the potential for eruption at a given time. The increase of the proportion of appropriate VALs and of `UwE’ with time (Figure 10) could be a reflection of an increase of the level of monitoring or understanding of the volcanic systems. Better monitoring enables detection of more subtle unrest signals that may not lead to immediate eruption but which could be related to magma intrusion at depth (e.g. Moran et al. [2011]). Unfortunately it is not possible for us to quantify this relationship because we could not determine how the monitoring level at each volcano changed with time. Another explanation is that the increase in number of `UwE’ reflects a change in the decision-making process whereby there is less reluctance to issue an alert even if the certainty of the eruption is not higher than in the past (see below).

The influence of the decision-maker(s) on the issuance of VALs

So far we have analyzed the role of the physical parameters (volcano and eruption types) and technical factors (e.g. monitoring level) that may lead to more appropriate issuance of VALs. However, a change in VAL is ultimately a scientific decision-making exercise, usually involving a single individual or a group. It is difficult to identify a quantifiable measure to characterize the effect of the decision maker on the reliability of VALs, but it is probably very large and may explain why the parameters we have investigated do not fully explain the different percentages of the categories. Decisions in high-risk situations such a volcanic crisis may be affected by the individual risk aversion of the decision-maker and by personal interest (e.g., losing reputation) (Woo [2014]). To compensate for the potential bias of the scientist-decision-maker, several workers have advocated for a more robust methodology based on probabilistic hazard assessment and cost-benefit analysis (Marzocchi and Woo [2007], [2009]; Marzocchi et al. [2012]; Woo [2014]).

Two factors from our data suggest a significant effect of the decision-making process on the appropriate issuance of VALs. One factor is the increase of `Timely’ and `Almost’ categories and another is the proportion of Alert Levels and eruptions with time (Figures 10 and 11). As noted above, these factors could reflect results from an improvement of the monitoring networks with time, but also include a realization that to improve VAL issuances it is necessary to be less conservative. Our data could indicate that decision are starting to be made from an optimal risk management perspective, rather than waiting for eruption probabilities to become very high (e.g., Woo [2014]). The other indication of influence from the decision making process is the relationship between the VALs, `UwE’, and the population size. The global population dataset shows only an increase of the `Timely’ category with increasing population density, whilst the `Almost’ VALs show no trend. However, we found that two different countries with different demographics both show an increase of appropriate VALs with increasing population suggesting that the decision makers are more risk averse when changing VALs depending on the population at risk. We find that there is a general increase of `Almost’ and `Timely’ VALs with increasing population but that this increase seems to occur at different country specific thresholds. Country B with its low population density shows an increase in `Almost’ and `Timely’ VALs at low population numbers (>100) (Figure 8B). In Country A, which has a higher population density, we see an increase at populations >10,000. This increase with population size suggests that although VAL schemes maybe designed simply to communicate the hazard, they are being implemented with risk to local populations in mind. In a specific example (Indonesia), CVGHM did not raise the VAL to the highest level at Sinabung in 2013 even when phreatomagmatic ash columns were reaching 7 – 10 km as there was little risk to the population on the ground. Instead, they waited until there was a high likelihood of a summit lava dome collapse, so that evacuations would be more commensurate with the risk (John Pallister, personal communication, July 2014).

The trends of `UwE’ alarms in Countries A and B may hint at the complexity of this decision-making process. In Country B (Figure 8C) the decrease in `UwE’ as populations increase may suggest that there is an effort to not issue alerts until there is a high level of certainty about impending eruptions. Country A, though, shows a more complex response (Figure 8B). It could be argued that the increase in `UwE’ alarms as populations increase (after 10,000 people) reflects intolerance to leaving populations at risk. It is interesting to note that the higher levels of `UwE’ seem to be coupled with higher levels of `Timely’ VALs.

Conclusions

The reliability of issuance of VALs depends on the type of volcano, size of the eruption and monitoring network. Large eruptions (VEI 3 or larger) from closed-vent volcanoes with reasonably instrumented monitoring networks (Level 3), and which have a large population living in their flanks are those for which VALs are issued most appropriately. Over time, we see modest improvements in the VALs that are issued, including an increasing number of `UwE’ alerts. Improvement is seen in both eruption-centric and alert-centric evaluations, and suggests improvement in the monitoring networks and also an increasing willingness to issue alerts even if unrest may not lead to eruption.

However, “improved” does not mean ’satisfactory’. Only 19% of the eruptions in our dataset were correctly anticipated by issuance of appropriate VALs. The low overall success in anticipating eruptions may reflect both the complexity of volcanic systems and the complex, multidisciplinary decision-making process of changing VALs during a volcanic crisis (Marzocchi et al. [2012]; Sparks et al. [2012]). We infer other remaining problems in using VAL schemes, such as an insufficient density of monitoring networks and/or reluctance to raise alerts because of concern that they may generate undue public concern, slow tourist arrivals, generate extra communication challenges for the scientists, etc.

Whilst we highlight a need for the improvement of monitoring networks in our findings, a conservative approach to changing alerts, or waiting for a clear cut evidence to become available seemingly leads to VALs being issued after the onset of eruptions and potentially leaving populations at risk. There are far more cases of `Missed’ and `Too Late’ than there are of `Premature’ alerts – so many more that we suggest observatories need not worry too much about issuing `PrematureResub’ alerts. We further suggest that to reduce the number of `Missed’ and `Too Late’ calls there is a need for less conservative decision-making – raising alert levels freely -- so that mitigating action can be taken even when there is still considerable scientific uncertainty. This is, indeed, one of the most important goals of VAL schemes.

We invite countries to perform their own self-evaluation and weigh the cost of a higher number of alerts against the benefit of a higher accuracy in VAL issuances and to decide how to proceed accordingly with their own local populations. We also suggest that with the support of databases such as WOVOdat, the most effective way to increase the appropriate issuance of VALs could be to expand the implementation of the use of probabilistic hazards management and cost-benefit analysis so that decisions can be as balanced as possible.