OR/17/045 Findings from the stakeholder workshop

Duncan, M, Mee, K, Hicks, A, Engwell, S, Robertson, R, Forbes, M, Ferdinand, I, Jordan, C, and Loughlin, S. 2017. Using the 'myVolcano' mobile phone app for citizen science in St. Vincent and the Grenadines: a pilot study. British Geological Survey Open Report, OR/17/045.

The qualitative results of the workshop were analysed using the analysis software NVivo. The semi-quantitative questionnaire results were analysed using Excel. The findings from the workshop are divided into five main themes: (1) types of data and their usefulness; (2) challenges in collecting and visualising observations; (3) suggested improvements to myVolcano; (4) perceived role of myVolcano; and (5) findings from the post-workshop questionnaires.

Types of observations (data) and their usefulness

Much of the content of the focus group discussions related to using the data collected by myVolcano during an event (emergency/disaster), therefore the emphasis was on how quickly that information could be received and displayed in the app. Real-time functionality was therefore at the forefront of the group discussions. Related to this were discussions around data verification, and the 24-hour delay in uploaded observations appearing in myVolcano (shown in Figure 5).

Suggestions to improve this included the ability to update and discuss information (e.g. by adding a commentary) or filtering data (e.g. by only uploading unverified data for high hazard areas).

Perceived useful formats of data extended to different media (e.g. videos and audio recordings). In terms of making and sharing observations, photographs were deemed to be more powerful than written descriptions; however, there was an acknowledgement from Group A that (1) this is dependent on the quality or subject of the photo and (2) that it might be challenging/impossible to take photos of some hazards (e.g. ‘an explosion’ or ‘earthquake rumbling’).

The relative usefulness of two different photographs of flooding was discussed by one group (Figure 7). In the first example (Figure 7a) they stated that they would focus solely on the photo and pay little attention to the accompanying text because the photo was useful as it was. With the second example, they felt that the photo of the damaged bridge (Figure 7b) was most relevant to those responsible for repairing the bridge.

In general, the usefulness of observations differed from one person to another as they were generally considering the usefulness of each photo from their own perspectives i.e. what would be useful to their work, or to their routine (e.g. if it might affect their normal route home from work). Group A commented on the fact that some of the observations might be more useful for scientists (e.g. ‘cloud over the volcano’) than for those in the group. It was agreed that photos and descriptions about the eruption would be most useful for the Soufriere Monitoring Unit and SRC.

Figure 7    Photographs showing flooding and bridge damage in St. Vincent used in the workshop scenario (photo credits: Anna Hicks, BGS © NERC).

Group B also suggested that photographs might be better than descriptions because they show what is happening without interpretation. For example, Group C noted that ‘cloud from the volcano’ is an interpretation and therefore questioned its reliability.

There were also suggestions for incorporating additional information into myVolcano (e.g. information on different hazards and contact details of emergency groups), as well as mechanisms of exchanging information. Suggested improvements included how the uploaded observations should be displayed to provide updates on the current situation (e.g. via push notifications from NEMO) and advice on what to do during an emergency.

For those able to view the additional multi-hazard check list in the prototype Version 3, the ability to input more detailed descriptions about hazards was viewed as a useful functionality, however users might require explanations as to what the different tick boxes (e.g. ‘pyroclastic flows’) mean. Unfortunately, Wi-Fi limitations meant that only one group had the opportunity to look at this additional functionality, limiting feedback.

Challenges in uploading and visualising observations in myVolcano

Participants noted several challenges related to collecting observations using myVolcano in its current design. In terms of uploading, there was a suggested need for more detailed location descriptions (e.g. not just the name of a town) and Group C stated that they found it confusing that the GPS icon was the same as the drop-pin in Google. There were suggestions that Google maps should be used as the base maps in myVolcano (currently provided by ESRI) but there is a limit on daily usage of Google maps, after which there is a charge.

The name ‘myVolcano’ also created some confusion in that it emphasised the application to volcanoes and less so to other hazards, and there were suggestions of renaming the app to something less specific, such as ‘myIsland’. There was also a question as to how information from text messages (e.g. from people without smartphones) could be integrated into myVolcano. Throughout the discussions, there were several suggestions that the app should be ‘all-encompassing’. This was mentioned both in terms of functionality and data (e.g. to be more efficient than in its current state) as well as in terms of potential advice that is provided through the app. For example, Group B agreed that they would be unlikely to visit a website or social media for advice as well as check the app. Instead they would prefer all information to be available in the same place. This was also linked to people’s motivation to use the app.

Improvements to visualisation on the map were also discussed, with groups A and C suggesting the use of different symbols for different hazards and group C suggesting that filtering data could help users to gain a quick understanding of the current situation. The existing icon — a cloud, which represented the location of ash observations from the first version of myVolcano — was considered no longer suitable (Figure 8).

Figure 8    The existing observation icon in myVolcano is represented by a cloud, which was initially to represent locations of ash observations in version 1 of myVolcano. The icon provides no insight into what type of hazard or observation is being presented. (Background mapping provided by ESRI Online).

One group also raised a concern regarding duplication of photographs (e.g. several people providing images of the same scene) and whether this would clog up the map interface. Group B suggested that adding a filter to enable a subset of observations to be viewed based on the time they were taken (e.g. most recent submissions) would enable better interpretation of the information. Furthermore, there were discussions about whether the observations should be displayed as they are received or with a level of interpretation (e.g. by a validator such as SRC or NEMO staff). Suggestions ranged from adding a comment (e.g. by the validator) to help qualify the observation (but without altering its original form), to providing advice (e.g. alerts) from the local authorities and decision makers (e.g. NEMO).

More practically, both the challenges of maintaining Wi-Fi access during the workshop and technical discussions regarding communication systems on island led to concerns about low bandwidth and operating offline. myVolcano does not currently cache data and so map layers and observations were not visible when offline. Furthermore, the limited Wi-Fi access meant that some participants were relying on the web tool rather than mobile app, which emphasises the importance of maintaining a web tool. The workshop questionnaires, however, highlighted that respondents were more likely to use an app over a web tool.

Linked to these challenges is the theme of system redundancy in St. Vincent. Digicel are working on methods for using apps and data when there is no internet and Group C expressed some faith that the authorities/companies would keep communication systems working during an event. There was also the suggestion that filtering the observations could help when using the app during times of low bandwidth (e.g. less content to load). Thus, although challenges were identified, participants also suggested a number of improvements that would refine myVolcano and make it more suitable to their purposes. These are presented in the following section.

Suggested improvements: meeting user needs

Participants identified a number of suggestions and strategies for overcoming the challenges they had identified and how to meet their required needs. These are summarised in Figure 9 (see Appendix 4 - Suggestions for improvements to myVolcano from workshop participants for detailed responses). Many of the suggestions were interrelated and, whilst some were specific to the needs of St. Vincent (e.g. localised place names), it is clear that these would be transferable to other locations.

Many of the suggested improvements related to the subject of communication and whether it should be (1) interactive (between uploaders and recipients), (2) more informative and (3) actionable. Two-way communication, also identified in the 2015 scoping study (Mee and Duncan, 2015), was deemed an essential component of myVolcano that would not only improve preparedness for and response during crises, but also increase motivation of users as they would be more inclined to use the app if they felt that they were receiving action-based information in return. Ideas for improving communication were also highly linked to the verification process, since much of the discussion focused on an event/emergency situation. Means of improving the efficiency of the verification process by, for instance, encouraging a forum-based commentary, filtering data depending on the location (high hazard area) and key words and labelling whether it was verified or not, were all suggested.

Figure 9    Summary of suggested improvements to myVolcano made by participants during the workshop (detailed list of suggestions in Appendix 4).

Perceived role of myVolcano: motivation for using the app

Throughout the discussions, four perceived roles for the app were identified:

1. Education and raising awareness;
2. Communication;
3. Providing information and advice;
4. Decision-making.

Whether the app would enable a two-way exchange of data and information between citizens and scientists and emergency managers/responders emerged throughout the discussions. There was a perceived need for users to receive feedback from official authorities (e.g. scientists, civil protection etc.), whether through awareness raising, communication or actionable advice, and that having a single app that is ‘all-encompassing’ — i.e. an app that tells the user what is happening and what to do, as well as allowing them to upload an observation — would be optimal. In addition, it was felt that myVolcano would need to be more efficient than existing tools (e.g. Facebook) “otherwise people will just continue to do what they already do” (participant in final discussion).

Workshop questionnaires

Of the 27 people who attended the workshop, 24 participants filled in the post-workshop questionnaire with a median completion rate of 87.5% (average 82%). The organisational profile of those who responded to the questionnaire is given in Appendix 5. All participants, with the exception of one who provided no answer, own a smartphone, with at least 83% using an Android platform and only one using iOS (Figure 10).

In terms of existing applications used by the participants for sharing data, Facebook and WhatsApp were the preferred platforms. This reiterates the need for myVolcano to complement, combine and build upon existing applications for sharing data, as users do not want multiple applications for different functions.

Figure 10    Proportion of operating systems used by questionnaire respondents.

Figure 11    Data sharing applications typically used by questionnaire respondents.

Likelihood of downloading and using the app

Overall, the responses were positive. There was a general sense that respondents would download the app (54% very likely; 29% ‘likely’). However, it was slightly less likely that participants would use myVolcano for uploading their own observations (42% very likely; 29% likely) or that they would use the data uploaded into myVolcano (33% very likely; 42% likely). More participants were uncertain as to whether they would upload observations to myVolcano (17%) compared with downloading myVolcano (8%), which may reflect the limited contact participants had with the app (Figure 12). Seventy-one percent of respondents’ answers stated that they were either very likely (29%) or likely (46%) to use myVolcano as a source of other information, which is in line with the numerous workshop discussions around communication and receiving ‘useful’ information.

Figure 12    Results of the Likert scale questions in the participant questionnaire.

Usability of myVolcano and suggested improvements

In terms of using myVolcano, 54% of respondents stated that they were more likely to use the app than the website and another 4% said they would use both, depending on their situation (i.e. if at home or not). Twenty-five percent of respondents did not answer this question whilst one participant mentioned that their choice would depend on their device and connection. In terms of when they would use myVolcano to upload observations, 10 respondents said they would make observations during working hours, seven of whom stated that they would also do this in their spare time (two stated it would only be in their spare time). Other respondents either did not provide answers or were more ambiguous in their responses, with some simply stating that they would use it depending on circumstances e.g. “once there is an event or activity that would alert and assist the public” (i.e. notify the public about what is going on).

Of those that answered the question regarding whether they would use the information in myVolcano for decision-making in their work (6 did not), 11 could apply it to their work, for instance, those working in monitoring and emergency response (SMU and Police) saw how the app could assist them in informing NEMO and engaging with their rescue teams.

Other respondents reflected on how myVolcano might be useful for other agencies, rather than their own. Three were more conditional in their answers, with two emphasising that they would first like to see the information verified before it would be used to aid decision-making. Others were less certain at present, with one stating that “[I] would have to see the full app version then [I] could say” (Respondent W20). Two respondents stated explicitly that they would not use myVolcano for their work, but one of these mentioned that they would use it personally (it was just not relevant to their role as an IT technician).

Suggested improvements

Sixteen of the questionnaire respondents (62.5%) suggested improvements to myVolcano, which generally echoed the group discussions. Examples of improvements included:

• Ensuring information is disseminated to app users;
• Feedback on uploads — from scientists/emergency managers to observers;
• Data filter;
• Change symbology;
• Add layers showing vital infrastructure (e.g. shelters);
• Technology capacity (capacity to run on older machines and slower connections);
• Renaming the app (e.g. to ‘myDisaster’, ‘myIsland’).

Other points included making the app ‘more user-friendly’. Those who did make suggestions for improvements also emphasised their support for myVolcano, describing it as “…a very useful app [but that] the useful recommendations from this workshop need to be incorporated into the app.” (Respondent W11).