OR/17/045 Workshop methodology and approach

From MediaWiki
Revision as of 10:59, 10 May 2018 by Ajhil (talk | contribs) (→‎Limitations)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Duncan, M, Mee, K, Hicks, A, Engwell, S, Robertson, R, Forbes, M, Ferdinand, I, Jordan, C, and Loughlin, S. 2017. Using the 'myVolcano' mobile phone app for citizen science in St. Vincent and the Grenadines: a pilot study. British Geological Survey Open Report, OR/17/045.

The workshop and school activities in St. Vincent had the dual purpose of raising awareness of myVolcano and gathering feedback on all aspects of the app and were designed to target a range of participants. The primary activity was a workshop, which involved focus group discussions with key stakeholders, in particular those deemed most likely to use the data being collected by the app (e.g. scientists, decision makers, disaster managers etc.). A second activity directed at secondary school children, involved presentations and question-and-answer sessions and was designed to explore myVolcano from the point of view of someone providing the observations. Questionnaires were provided to the workshop participants and school students after the activities to gather specific feedback on mobile phone operating platforms and attitudes towards myVolcano. A presentation was also given to staff at UWI SRC in Trinidad and informal discussions around the functionality and potential of myVolcano provided further feedback from the perspective of monitoring scientists. The following sections describe the sampling, design and implementation of activities and discuss the details of the methods and approaches adopted during the workshop and school visits.

App versions

It was uncertain how many attendees would have Apple devices and therefore be able to download the published version of myVolcano (v1.3) on their own devices. Therefore, BGS staff provided 3 iPads and 3 laptops with various versions of myVolcano installed. These included:

  1. The published version 1.3 of myVolcano installed on all 3 iPads and available to participants with iOS devices;
  2. The unreleased version 2.0 of myVolcano, which was installed on 2 iPads and 2 laptops;
  3. The prototype ‘multi-hazard’ version of myVolcano, installed on one of the iPads;
  4. The web-version of myVolcano, which was accessible to all users either via their own devices (smartphones, tablets and laptops) or classroom PCs.

The version of myVolcano that participants had access to was dependent on which group they were in and the quality of Wi-Fi connectivity. For example, the majority of the school participants used the web version of myVolcano.

Workshop design

Attendees and activities

Workshop attendees (24 participants) were identified from previous work on resilience building (e.g. through the Strengthening Resilience in Volcanic Areas (STREVA) project) and by NEMO. A deliberately wide range of participants was targeted across government, monitoring agencies, emergency response, disaster risk management and telecommunications.

The workshop was designed to run for 3 hours to accommodate the availability of participants. It was split into four activities designed to introduce the concept of citizen science, the myVolcano app and enable different levels of participation, including:

  1. An interview-style discussion between workshop convenor A. Hicks and volcanologist R. Robertson on the importance of making observations and citizen science, which was conducted in front of the other attendees;
  2. An introductory presentation about myVolcano to workshop attendees;
  3. Several focus group discussions framed around a multi-hazard scenario on St. Vincent to explore the usability of myVolcano and the usefulness of the data that it can capture;
  4. Feedback and summarisation of the main findings of the day and an opportunity for workshop participants to complete feedback questionnaires.

Multi-hazard scenario

In order to stimulate discussion, a scenario was adapted from previous UWI SRC/NEMO events and was run alongside a focus group discussion (Figure 6). The scenario was designed around an eruption and was broken into four parts: unrest, pre-eruption, ash plume and explosion. Each stage of the eruption was designed to include a variety of sensory information (visual, sounds and smells) and included both related (e.g. volcanic) and non-related (e.g. flooding, storms) hazards (Appendix 1 - Scenario exercise for stakeholder workshop). myVolcano was pre-loaded with a number of text observations and photographs to illustrate each stage in the scenario and to stimulate discussions on what constitutes a useful observation.

Figure 6    Workshop participants take part in a focus group discussion framed around a volcanic eruption scenario, to explore the potential uses of myVolcano (photo credits: Samantha Engwell, BGS © NERC).

A written guide was provided to each focus group, which included questions to facilitate the discussion, whilst allowing for flexibility for different thoughts and themes to emerge. The questions focused upon the following areas:

  • Participants’ current use of citizen science;
  • Perceived users of myVolcano and their requirements;
  • How data is visualised in myVolcano;
  • Challenges/problems associated with making observations and using myVolcano;
  • Other data/information that would be useful in myVolcano;
  • Suggested improvements.

Feedback questionnaires

Questionnaires, completed at the end of the workshop, were selected as a means of capturing additional, specific information about individual participants. These included questions about:

  • Participants’ organisation and role;
  • Participants’ mobile/smart phone platform (e.g. iOS, Android etc.);
  • Feedback on myVolcano;
  • Whether or not participants would download and use the app;
  • When and how they would use it.

The questions included a mixture of free text and Likert (or rating) scale questions and were intended to be concise, owing to the fact that participants had already engaged in three hours of workshop activities. A copy of the workshop questionnaire is included as Appendix 2 - Stakeholder workshop – feedback questionnaires.

Implementation of the workshop

The workshop opened with an interview-style exercise between A. Hicks (BGS) and R. Robertson (UWI SRC), which was conducted in front of the other workshop attendees. The aim was to explore the meanings of the terms ‘scientist’ and ‘citizen scientist’ and discuss some existing citizen science initiatives in the Caribbean. The aim of this exercise was to highlight the importance of making observations and how citizens can make a valuable contribution to this data collection effort. Using local examples and anecdotes, the workshop convenors were able to explore the idea of science in terms that were tangible to participants:

“He knew more about geology and he was an amateur geologist/volcanologist. In a sense he was a naturalist — he knew everything about St. Vincent, he knew about birds, but he wasn’t trained in that field. He was, in my head, a citizen scientist because he acquired knowledge in a particular way and he applied that knowledge and he made conclusions based on that.” (R. Robertson, talking about a citizen of St. Vincent who was a trained vet and an amateur geologist)

Overall, the interview was about engaging and empowering the participants in the room to see their role as potential citizen scientists and users of citizen science data, and that citizen science in the Caribbean is not new:

“So citizen science is… not just observations, there are programs which involve people actually doing measurements and taking information and gathering data, quite apart from just visual observations or qualitative stuff, but actually measuring things and collecting it back.” (R. Robertson)

“I think a lot of what we call citizen science is people who have always been around, observing nature, observing human systems and making conclusions based on it all. A lot of people wouldn’t think of themselves [as a scientist] like that.” (R. Robertson)

The myVolcano app was then presented as a means by which observations can be shared and visualised. The history behind the app was described, and the functions and implementation of myVolcano were presented in detail by S. Engwell (BGS). The workshop then proceeded into the participatory group exercise using the scenario of an eruption to simulate use of the app and stimulate discussions about its functionality. Participants were encouraged to discuss citizen science and the app’s role in the scenario, and were given the opportunity to test myVolcano.

Profile of the discussion groups

Participants were split into three different groups of approximately 6–8 participants per group, based on where they were situated in the room with a facilitator was assigned to each group. A detailed profile of the groups was not possible to capture, but in general they could be categorised as:

  • Group A (Facilitated by A. Hicks): had a wide range of participants spanning the Information Technology Services Division, GIS Planning, Service Commission, Soufriere Monitoring Unit (SMU) and NEMO. Although an iPad was available with the most recent (unreleased) version of myVolcano (Version 2.0) this was not working at the time. Most participants used their phones to interact with the web-based version of the app, whilst some downloaded the published version of myVolcano (Version 1.3) onto their own devices.
  • Group B (Facilitated by S. Engwell): dominated by technicians from Digicel and other communications companies. They had access to the unreleased Version 2.0 of myVolcano on an iPad and focused particularly on the technical aspects of the app, providing information and feedback on the usability and feasibility of using myVolcano in St. Vincent. They generally perceived themselves as providers of information and did not deviate from this role when working through the scenario.
  • Group C (largest group; Facilitated by C. Jordan): had a range of participants but also had a consensus of feedback. They had access to a prototype (unreleased) of a multi-hazard version of myVolcano, which includes more detailed forms for users to complete about different hazards. There was some focused feedback on myVolcano, but the main feedback related to the overall usability of myVolcano during the ‘full emergency lifecycle’.

Limitations

A flexible approach to the delivery and gathering of findings from the workshop was adopted: although activities had been planned and resources generated in advance, the approaches needed to be adaptable in the event of anticipated (e.g. Wi-Fi connectivity) and unanticipated issues. The workshop design was heavily reliant upon internet availability (a finding of the study in itself) and the limited number of devices available to participants during the workshop. In the event of Wi-Fi connection problems, the flexibility of the workshop design was such that it could run without internet. However, this would mean relying heavily on the web version of myVolcano which only contains the most basic functionality (e.g. adding and viewing observations). The Wi-Fi issues encountered during the workshop emphasised these inefficiencies in the web version of myVolcano, intended to be a stand-in service for non-iOS users whilst an Android version of myVolcano was being developed. These issues have since been addressed by completing the development of myVolcano on Android, which was released in April 2017. However, the heavy use of the website version of myVolcano (particularly during the school visits, as outlined in Section 6) has highlighted the need for an equivalent web-based tool which contains the same functionality as any released versions of myVolcano.

One of the earliest constraints identified during the 2015 scoping study was that iOS devices are not abundantly used in St. Vincent, with Android being the preferred platform (Mee and Duncan, 2015[1]). This was confirmed by the post-workshop questionnaires (Workshop questionnaires), which showed a predominance of Android platform usage amongst workshop participants.

School visits

A total of four schools were visited during the pilot study, organised by Idelia Ferdinand of the Ministry of Education, Government of St. Vincent and the Grenadines. Two of the schools were mixed, whilst the other two comprised a boys’ and a girls’ school. All children were of secondary school age (Table 1). As with the workshop, issues with internet access were expected to limit the extent to which activities could be implemented and therefore activities were designed with a degree of flexibility. The purpose of the visits was to raise awareness of myVolcano, whilst also collecting feedback from a target user group. The length of visits was limited to single or double class lengths and were, therefore, designed to accommodate two exercises: (1) questions on the definition of scientists, citizen scientists and observations and (2) an opportunity to input data into myVolcano and provide feedback on their experience. The students were provided with an activity sheet (Appendix 3) to compile their answers, which used both free text and Likert scale questions, similar to those used in the workshop questionnaire.

Ethics and informed consent

Ethical considerations were taken into account when designing the workshop and associated activities to ensure that risks were minimised. The questions asked at the workshop did not extend to personal information beyond the name, organisation and job title and type of mobile phone owned by the participant. In the case of the school visits, some additional personal information was requested, including the age and gender of the participant and the school they attended, as it was considered that this background information might inform the analysis. All of the data collected is stored according to NERC/BGS data procedures and privacy policies, and participants and students have been anonymised for the purpose of this report.

All participants of the workshop and school visits were fully informed of the purpose of the project and that their contributions would feed into (1) informing future app developments, (2) enhancing current understanding of the role of citizen science and (3) key outputs, including this report. Written consent was deemed unnecessary owing to the scope of the project. The school visits were attended by teachers and facilitated by the Ministry of Education, who were aware of the purpose of the visits.

Table 1    Summary of school visits (NB: different schools/groups had access to different versions of the app
i.e. some used the web version whilst some used mobile devices).
School Location Description of activities Number of participants Gender
St. Vincent Girls’ High School Kingstown Almost all students had a smart phone and/or an iPad. Internet did not allow for easy access to the app.
1 hour class: Presentation and both exercises
46 Female
St. Vincent Grammar School Kingstown Almost all students had a smart phone or access to one. Internet allowed some students to access the app. Some downloaded the app, while others used the webpage.
50 min class: Presentation and both exercises
27 Male
Thomas Saunders Secondary School Kingstown Almost all students had access to a smartphone. Access to WiFi was poor
1 hr 10 mins class: Presentation and both exercises
28 Mixed
Mountain View Adventist Academy Richland Park All students had individual access to a school laptop and internet. Most students uploaded information via the webpage, but one student used the published app.
1 hour class: Presentation and both exercises. Finished with STREVA video on St. Vincent (due to available time).
19 Mixed

References

  1. MEE, K, and DUNCAN, M J. 2015. Increasing resilience to natural hazards through crowd-sourcing in St. Vincent and the Grenadines. Nottingham, UK, British Geological Survey, 50pp. (OR/15/032) (Unpublished) www.nora.nerc.ac.uk/511949