Towards more User-Centered Contributory Citizen Science Initiatives: Learning from the u³Green Approach

Hennig, S.,^1* Vogler, R.,¹ Schötz, T., ¹ Strobl, J.¹ and Imanalieva, P.²

¹Paris-Lodron-University Salzburg, Austria

²Central-Asian Institute for Applied Geosciences, Kyrgyzstan

*Corresponding Author

Abstract

In many citizen science projects the use of geospatial technologies plays an important role. This refers in particular to initiatives where the public contributes spatial data. Along with the benefits, such initiatives also face challenges related to the quantity and quality of data provided. A suitable means of overcoming these problems are user-centered applications. However, since developers often lack a sufficient understanding of the requirements of users from the public, their appropriate involvement in application development is crucial. Despite existing approaches, there is still a need to further improve user involvement in these processes to deliver (more) user-centric applications. This paper contributes to this by presenting experiences from the u³Green project. The aim of u³Green is to develop an application tailored for young people to contribute spatial data about urban green that support the implementation of child- and youth-friendly cities. To generate a youth-centered application, a project-specific development process was created, which combines strong participatory design with the prototyping model and selected methods to involve young people in all development tasks. Collaborating with young people in this way confirmed several well-known aspects of designing user-centered data contribution solutions but also revealed new, previously unknown criteria for the implementation of user-centered applications.

Keywords: Child- And Youth-Friendly Cities, Crowdsourcing, Crowd Mapping, Participatory Design, Process Models, Public Participation, Urban Green

1. Introduction and Research Question

Citizen science, i.e. the engagement of citizens in scientific research, not only has a long history, it also can take on different forms of participation, such as contributory, collaborative, and co-created participation (Table 1). Contributory citizen science projects traditionally and currently represent the majority [1][2] and [3]. For contributory citizen science, advances in information and communication technologies (ICT) have opened up new possibilities to engage the public, their knowledge and resources to scientific research, thereby generating new insights and knowledge [2][4] and [5]. Here, the use of ICT also includes geospatial technologies, which takes in the generation, reporting and sharing of data with an explicit geographic reference [6] and [7].

The possibility to contribute spatial data has met with growing interest from both citizens and scientists. For instance, the use of spatial data and related tools is an excitement factor for citizens, and it promotes their learning and acquisition of spatial literacy skills [6] and [8]. Scientists and their projects benefit, for example, from information that would otherwise be unavailable and/or are difficult to obtain.

Table 1: Forms of citizen science projects [1] and [3]

This refers to data that reflects citizens" observations, their local spatial knowledge, e.g. how they perceive, experience, value, and use infrastructure or resources. Additionally, not only can data be gathered at large geographic scales, but it can also support monitoring initiatives that often pose difficulties to traditional data collection approaches. This increases the amount and scale of spatial data available [9] and [10].

But contributory citizen science projects also face challenges. This refers, for instance, to the quantity and quality of (spatial) data contributed by the public as well as to the sufficient involvement of the intended target group [6]. Here, user-centered (spatial) data contribution applications as tools that focus on the public or a specific group of society, their requirements, preferences, knowledge and skills, are a success factor of contributory citizen science projects [11]. However, the development of citizen-centered applications is a challenge. This is underlined by the fact that citizens often consider (spatial) data contribution applications to be (too) complex, (too) unintuitive or (too) unsuitable. One reason for this is that the public is a very diverse audience that - compared to traditional GI users - is less known to scientists in terms of their needs and barriers especially when it comes to using ICT and geospatial technologies. To counter this situation, in addition to sufficient consideration of the users, i.e. the citizens, and their requirements, in application development processes, citizen participation in development processes is of decisive importance [6].

Despite the many possibilities that exist to involve people in the different tasks that build up application development processes (e.g. requirements specification, application design and implementation, and testing), there is still a need to integrate citizens more actively and directly to get to know and understand them [12] and [13]. But what possibilities exist and can be used to support appropriate citizen involvement to develop data contribution applications that in fact meet the needs of users from the general public? How can these aspects be optimally combined? What are the challenges associated with resulting citizen involvement in application development?

Findings from the Sparkling Science 2.0 project u³Green ("Promotion of child- and youth-friendly urban landscapes through participatory research on urban green"; funded by the Austrian BMBWF; https://u3green-zgis.hub.arcgis.com/) contribute to answer these questions. u³Green aims to develop a spatial data collection application tailored for young people allowing them to contribute data on urban green, including associated infrastructure and characteristics. Such data is seen as an important means to support the development of (more) child-and youth-friendly cities, characterized by sufficient natural environments as well child- and youth-friendly urban designs. In u³Green, an application development process has been specially created to fully involve young people in the development process to deliver a truly youth-centered data contribution application.

2. Methods

In u³Green, the design of a project-specific application development process relied on several steps, each using different methods (Figure 1): (i) getting to know the target group of the u³Green data contribution application, i.e. young people, in detail; (ii) identification and evaluation of appropriate possibilities to support user involvement in application development; (iii) selection and combination of the appropriate possibilities regarding application development that enable user involvement; (iv) assessment and optimization of the created application development process. These points are briefly outlined in more detail below.

Knowing and understanding the general characteristics of young people - who are the target group of the u³Green data contribution application - is crucial for designing an application development process that allows for their involvement. This is the basis for carrying out the further work steps, that is identify and evaluate user involvement possibilities, select and combine possibilities that build up the development process for appropriate user involvement, as well as evaluate and optimize the specially created development process.

Literature and Internet research was used to gain knowledge about the young people and identify suitable possibilities for their involvement in the development process. This refers, on the one hand, to literature and Internet resources regarding characteristics of young people (i.e. their levels of skills, capabilities, abilities, and knowledge) and, on the other hand, also to software, requirements and usability engineering as well as empirical social science, participation and education. The selection and combination of possibilities to involve young people in the development process as well as the evaluation and optimization of the created development process was carried out by experts in focus group discussions.

Figure 1: Creation of the u³Green application development process

3. Results

The results relate to (i) an overview of selected possibilities to involve users, i.e. young people in application development processes, (ii) the specially designed u³Green application development process, and (iii) the spatial data contribution solution realized through the u³Green development process including recommendations for the design and implementation of according tools.

3.1 Possibilities of User Involvement in Application Development Processes

Application development processes are usually framed by design approaches and/or development process models, and they include different task-related techniques and methods [14]. Within the context of these aspects, users are involved in the various development activities. Selected approaches, process models and methods are presented below.

3.1.1 Design approaches

Design approaches put the users at the center of a development process. Such approaches, which are also related to the field of human-computer interaction, include, for instance, user-centered design, design thinking and participatory design [15][16] and [17]. User-centered design aims for early and repeated user involvement during the phases of application design and implementation, which allows for iterative refinement of design and implementation, including requirements specification. Depending on the project, user-centered design consists of several methods and tasks to enable users to assess a tool for its usability [18] and [19].

Design thinking is a process for solving problems by prioritizing the users" needs above all else. Here, what is desirable from a human perspective is brought together with what is technologically and economically feasible. Design thinking relies on observing, with empathy, how people interact with their environments. It employs an iterative, hands-on approach to generate innovative solutions. Design thinking uses evidence of how users actually engage with a product, rather than how someone else or an organization thinks that users will engage with a product. It is less of a means to get to a single solution, and more of a way to continuously evolve the thinking and respond to consumer needs [20] and [21]. The overall goal is to identify alternative strategies and solutions that are not instantly apparent with an initial level of understanding [22].

Participatory design aims to designing-with rather than designing-for users. Users actively and directly take part in all activities of a development process [15] and [17]. A distinction is made between strong and weak participatory design. In strong participatory design, users participate throughout the entire development process and in decision-making. In weak participatory design, user input is solicited, but decision-making is undertaken by the experts and developers [15] and [23].

3.1.2 Development process model

Application development is generally based on tasks related to requirements specification, tool design and implementation, and testing. Depending on the process model, the tasks and related activities regarding requirements specification, design, implementation, and testing, are organized into different stages (i.e. phases), which also have different relevance [24] and [25]. Examples of process models as presented in Table 2 are the waterfall model, the incremental model, and the prototyping model [26][27] and [28].

Table 2: Examples of process models

Table 3: Selected methods to involve users in the development process [32] and [33]

Table 4: Levels of user involvement (based on [12] and [34])

3.1.3 User involvement methods

There are many methods to involve users in application development. This refers to methods traditionally applied in the context of requirements specification and testing, and to methods that are of relevance in the context of application design and implementation. Selected methods are presented in Table 3. These methods allow for different types and levels of user involvement in application development (i.e. requirements specification, application design, implementation, and testing), with levels of user involvement ranging from informative and consultative to participatory [12] and [34] (Table 4). Involving users in different types and to different extents in development tasks has several benefits for the development process and the created product. For instance, participatory user involvement in requirements collection with collaborative design and implementation of the according materials (e.g. questionnaires) brings additional results on understanding users" needs and preferences (e.g. use of language, preferences regarding colors) than purely consultative user involvement, i.e. using the questionnaire [14].

3.2 u³Green Application Development Process

The development process specially created in u³Green focusses on properly involving the future users, that is young people, in all application development tasks. The process is characterized by a project-specific combination of an appropriate design approach and process model as well as different methods to involve users (Figure 2).

Figure 2: Schematic and simplified u³Green data collection application development process

The general framework of the u³Green development process is defined by the use of the strong participatory design approach (i.e. user input is not only solicited, but decisions are also made by the users involved in the process). Further, the prototyping model was applied. By creating and discussing various prototypes, requirements were collected and refined, which ultimately served as the basis for the design and implementation of the final u³Green solution. Within this framework, the involvement of the young people (i.e. high school students from 11 school classes from 6 schools) was based on the use of different methods in the various steps and development activities. The gathering of the initial requirements (step 1) relied on information from a literature and Internet review. This made it possible to learn more generally about the relationship between urban green and young people, as well as participation and young people. To close the knowledge gaps identified, an online questionnaire was carried out and the Metaplan method was applied. The information obtained in this way was assessed and ranked using the Q-method. As a result, aspects most important for young people (urban green, participation) were identified and could be considered accordingly in the further steps of the u³Green application development.

The initial requirements (defined in step 1) were used in step 2 to develop prototypes by representatives of the target group. By discussing these prototypes, additional requirements could be raised. The development of the first prototypes was based on the parallel design method, in which several groups of high school students developed their prototype in parallel. Lessons learned from discussion of the prototypes were combined into a single prototype, which was further refined by representatives of the target group using the station talk method. The insights from step 2 allowed the definition of the final requirements (step 3). The final u³Green solution was designed and implemented on this basis (step 4). In step 5, the u³Green solution was tested and optimized. Methods such as cognitive walkthroughs applied in focus groups as well as interviewing users after they had data contributed through the u³Green solution were used.

3.3 u³Green Solution

Through the project-specific involvement of young people in the entire u³Green application development process, insights into the requirements and preferences of young people - regarding urban green and participation - could be gained. They found their way into the design and implementation of the u³Green solution. First of all, this refers to the fact that the data collection application must be supplemented with additional components. Accordingly, the u³Green solution consists of three components: (i) a data contribution component, i.e. online survey, (ii) online presence consisting of a content management system (CMS) project website as well as different social media channels including digital (multimedia) materials, and (iii) personal contact and face-to-face activities. The implementation of the u³Green solution relied on easy-to-handle off-the-shelf tools and (social media) platforms, see Table 5.

Table 5: Used Tools and (social media) platforms

Table 6: Selected aspects of the u³Green solution

For the realization of the spatial data contribution app (component 1) ESRI"s Survey123 for ArcGIS online was utilized, which also offers the ability to view the data that has already been collected before being processed, e.g. in an interactive dashboard (e.g. https://arcg.is/15XqSa). For the implementation of the CMS project website, ESRI"s ArcGIS Hub was used and in terms of the (social media) platforms Flickr, Instagram, Facebook, and TikTok were used (component 2) (see Table 5).

As for Component 3, related events were held not only by the u³Green project team per se, but mainly by the high school students who were part of the project team (Figure 2). In general, the three components - as shown in Table 6 - include numerous elements related to project-related information, instructions for contributing urban green data, addressing motivational factors, and supporting community building.

The u³Green solution, its components and elements are characterized by special aspects regarding structure, design, content and strategies considered. Their design is tailored to the preferences and needs of young people. This relates to usually valid and known aspects (e.g. reduced number of questions, short texts, no registration, no contribution of personal data, avoidance of mandatory questions, provision of additional information, background information, use of videos instead of text; see, e.g. [14] and [35]). Further new and previously unknown aspects were discovered and considered. For instance, one key aspect outlined through the cooperation with young people in the development process refers to the relevance of personal contacts in face-to-face situations to introduce and promote the project (e.g. in the classroom, peer groups, youth-specific events). Here, not adults but young people should be responsible for this task to generate a sense of belonging and thus commitment to the initiative. Beyond that, high school students participating in the project and application development reported that only peers know how to explain the relevance of the initiative in the right way (wording, reasons etc.) and therefore are way more reliable to the target group. Finally, the u³Green solution makes use of social media channels and multimedia, which are relevant for the target group. For instance, Facebook is perceived as outdated by young people, Instagram and TikTok are the "tool of choice". This goes in line with the relevance of short videos (i.e. "shorts") as key information media. Of course, this must be seen as a fluid development, since the popularity change of social media platforms has a very fast pace. Additional aspects are summarized in Table 6.

4. Discussion

4.1 User-Centered Development Process

While participatory design is per se a suitable means to involve users in development activities, the use of process models is further helpful in meeting and supporting the demand for appropriate forms of user involvement in the different tasks [36] and [37]. Using such a model, the design and implementation of products follows a well-known process that is divided into several phases addressing requirements specification, application design and implementation, as well as testing [24][25] and [26]. This has several advantages. For instance, it allows the easy assignment of different (groups of) representatives of the public to the various development stages and tasks. It structures development activities, helps to coordinate and communicate the tasks required to create the product as effectively as possible, and provides those involved in development activities with a comprehensive and understandable picture of the development process [26] and [27]. This enables the users involved in the development process to have a good understanding of all tasks, how they are related to each other, and the role their contribution plays in the whole process [35]. Here, the use of the prototyping model allowed to involve young people actively and directly since they designed and implemented own prototypes reflecting their needs and also to present and discuss them with the experts. In addition, the prototyping model is particularly suitable when the requirements of the future users of an application are relatively unknown at the beginning of the development process or when developers deal with an unknown or new topic [30] and [31]. This also applies to u³Green, where several issues regarding the relationship of young people to urban green and participation are less known. Thus, through the development, discussion and improvement of prototypes by the young people, knowledge and understanding of their requirements and how to cope with them could be gradually improved.

Easy to use and to learn methods, appropriate to their background and skills, are critical to engaging representatives of the public in development processes. This requires a careful selection of the methods to be used throughout the development process. In u³Green this related to the use of the Metaplan and Q method as well as parallel design and station talk. These methods - due to supporting the work in small teams - in particular address the characteristics of young people such as reduced attention span, low patience level, low sophisticated research strategies, and partly having problems to get and keep an overview [35].

4.2 Challenges of User Involvement

Involving users - and particularly citizens and moreover young people - in development activities requires specific efforts. This refers, for example, to composing an inter-/ multidisciplinary project team and the availability of sufficient time and resources. User involvement requires teams that consist of professionals from different disciplines (e.g. domain experts, developers, GI experts), representatives of the public (with different backgrounds, knowledge, experiences, and interests), moderators and mediators with domain knowledge, and particularly skilled and motivated citizens who act as peer-tutors, to support and motivate public users to participate in the various activities [38].

Especially when cooperating with young people, mediators and peers play an important role. While moderators lead workshop sessions and discussions with experts and the youth, mediators (with pedagogical background) and peers facilitate cooperation and communication between the experts and young people. This can improve and accelerate application development, as mediators and peers can help provide more appropriate explanations and ensure that the youth are not overwhelmed with the tasks, the methods and tools used, and the scope of the work. They can motivate the involved young people, help them to express themselves and to build trust in their work [35].

Participation of citizens is cumbersome. High-quality participation requires commitment, time, resources including money, and energy from the responsible persons [39]. This was fully confirmed in the context of u³Green. Not only the putting together of an inter-/ multidisciplinary project team and working in that team as well as the selection of suitable methods, tools and materials is labor intensive, but also the work with young people per se. This refers to the following issues already identified in previous user involvement initiatives (see, e.g. [35]): (re)motivate the participants, provide lots of explanations, face and handle long discussions, expect and face unreliability, be flexible (e.g. dealing with discussions out of company time), plan more meetings and roundups as usual to keep everybody up to date, face impatience, build up trust in young people"s skills, help to carry responsibility, support decision making without contributing own ideas or opinions, help young people to express themselves (reasoning, presentations etc.), expect and deal with the problem of neglected deadlines, and expect and deal with their different knowledge backgrounds.

4.3 Relevance for other Citizen Science Initiatives

Citizen Science initiatives can be grouped into two categories: those based on communities and activities rooted in pre-Internet times, where the entry into online forms or other kinds of submission essentially serves as a process automation of prior workflows. Location in this case is added explicitly. A majority of initiatives originated in the fully digital environments addressed in this paper, with location implicitly given through the use of smartphones with typically always-on positioning. The above outlined design principles apply to the latter. Involving the intended users in all stages of development is considered best practice. This still might face challenges in those cases where users come from different communities, age brackets or other demographics. Across all groups, there is clear evidence that sustained participation needs feedback loops -access to outcomes- and symmetric communication.

This already addresses motivation as a key aspect in facilitating engagement and participation. There is no need to clearly distinguish between topical motivation and playful infotainment, the actual purpose of an initiative and the stimulus still achievable through personal technical devices recording sound, imagery, (default) location plus time, and also semantic encoding of information in text and numbers. It has to be clearly understood how participation instances are stimulated: triggered by ad-hoc observation opportunities, or rather as campaign-style organized and scheduled acquisition.

Overall, insights from u³Green are valid beyond the urban green theme and youth as stakeholders. The same principles are fully applicable in other contexts. While designing-with instead of designing-for users might not be easily achievable in all cases, formats like focus groups or stakeholder engagements will be more easily feasible whenever high levels of intrinsic motivation for a "cause" are given. Further, in any contributory citizen science application, it is necessary to strive to ensure that the created interface will be understood by the user. Therefore, it is very important to learn in detail about the intended target group in order to know their needs and skills.

Here, user-centered contributory citizen science initiatives as developed and used by the u³Green are further of interest for developing countries such as Central Asian countries. This approach helps to unite to solve the problem in order to implement initiatives. For example, even when sorting garbage, problems arise when the population itself does not support the initiative. In these countries, the population is mostly passive when it comes to participation. This is because people are used to the fact that everything should be taken care of by the authorities [40]. Since Central Asia is slowly moving away from the Soviet style of work, retraining people will remain a challenge for a long time. Therefore, their involvement, e.g. the involvement of school students, would help to develop urban activism. But of course, it is necessary to spread and show the involvement of residents in new initiatives to improve urban green areas, for example, in social networks. Also, this user-centered approach will be useful when creating GIS systems and data platforms. At the moment, the acceptance of projects created by developers only and further the usability of the final product as well as the interest of the public remain in question.

Accordingly, ex-post trainings and workshops would be required to teach a new system or application, as well as advertising the project. Also important are the skills and competencies of the target group. For the youth, the use of new technologies is, e.g,, easier than for the older audiences. Therefore, the involvement of the public in the creation of applications is important and has a huge potential to support such initiatives [41].

4.4 Relevance for Education

For education, the implementation of the u³Green solution results in the acquisition of essential digital and spatial skills by the target group. The different tasks and activities regarding the u³Green development refer directly to the EU DigiComp 2.1 framework [42], which provides a basis for dealing with digital tools. In our case, this was supplemented with the DigiComp AT 2.2 model [43]. Thereby, not only the digital orientation (i), the handling of information (ii), the digital communication and collaboration (iii) and the production of digital content (vi) are emphasized, but also problem solving aspects (v) in the digital space. Thus, by working with geomedia and considering the Spatial Citizenship approach [44], important spatial skills are developed. Furthermore, basic skills in the form of spatial awareness [45], as well as technical-methodical competencies [46] and [47] were promoted throughout the different steps of the project. are shown in Table 7. However, promoting the development of spatial skills in young people can be seen as an educational benefit of the project.

5. Conclusion

User-centered applications are a success factor for contributory citizen science projects. However, developers often lack sufficient understanding and knowledge about citizens and their requirements. User involvement in application development processes plays a key role in gaining the missing information and knowledge to deliver user-centered data contribution applications. Despite the manifold involvement practices in application development there is a need to further improve user engagement in development activities. The experiences from the u³Green project address this issue, namely how - through suitable user involvement - data contribution applications can be developed that actually meet the needs of citizens.

Area of skills	Skills	addressed
Basic Skills - Spatial Awareness in GIS [45]	Knowledge of relevant concepts and terms related to GI and cartography (e.g., layer, basemap, etc.)	✓
	Awareness of cartographic layout/design (e.g., symbols, labeling, basemap, etc.)	✓
	Spatial Awareness / Spatial orientation in GIS (navigation, zooming, layer on/off, etc.)	✓
Technical-methodological skills [46]	Production:Contribution of own data and ideas	✓
	Data management: Importing, integrating, modifying and handling data	(✓)
	Prosumption: Change of data selection, labeling, marking and annotation
	Visualization: Appropriate visualization of data and information	✓
	Analysis: Use of existing functions for simple questions, analytical tasks, hypotheses from spatial representations	✓
Reflection and reflectivity	Applying knowledge to spatial representations	✓
	Comparison of information with existing knowledge and other sources	✓
	Quality assurance: Identification of hidden and missing information	✓
	Consideration of alternative meanings of spatial scenarios.	✓
	Awareness of social construction of spatial scenarios based on medium, preconditions, interests.	(✓)
	Hypothesis generation with geo media	✓
	Critical reflection on the power of maps	(✓)
Participation and communication	Social networking: Ability to use decision making and negotiation tools	(✓)
	Expression: Communication of meaning constructions and alternative spatial scenarios with GI	(✓)
	Communication: Sharing ideas and meanings; Use of online and offline communication channels	✓
	Negotiation and participation: Using Web2.0 technology as an option	(✓)

The u³Green results confirm that the combination of a design approach (i.e. strong participatory design) with a process model (i.e. prototyping model) and the use of selected, target-group-centered methods (e.g. Metaplan and Q method, parallel design, station talk) really support and enable the development of citizen-centered (i.e. youth-centered) applications. While the design approach relates to the way of putting the intended target group at the center of the application development process, the process model determines the order and emphasize of the process stages and their tasks. In combination both frame the u³Green development process and allow for the use of different types and levels of user involvement methods regarding requirement specification, application design, implementation and testing. This relates to methods that allow for consultative and participatory user involvement and that take users" knowledge and skills as well as their motivation into account.

Involving users in different types and to different extents in development tasks confirmed already known and revealed unknown aspects regarding user-centered data contribution applications. First of all, this refers to the fact that the data collection application must be supplemented with additional components. Accordingly, the u³Green solution consists of three components: (i) a data contribution component, (ii) an online presence consisting of a content management system (CMS) project website and selected (social media) platforms, including digital (multimedia) materials, and (iii) personal contact as well as face-to-face activities with the u³Green project team. The solution including all components is characterized by special aspects regarding structure, design, content and strategies. These findings are also of relevance for other citizen science projects which per se must address participants" motivation and provide easy to use and understand application interfaces. To enable and facilitate the use of these findings in other projects - in particular with respect to the generation of a project-specific workflow - there is the need to develop further refined guidelines regarding the selection and combination of design approaches, process models and methods. The u³Green insights presented here can serve as a conceptual basis for such future initiatives. It should be noted that this paper is not intended to present the results of the project with regard to data (which will be addressed in the next project phase), but rather to provide insights into the project design, its implementation and the lessons learned so far - based on structured observations, documentations, and evaluations.

Acknowledgments

This research is based on findings from the u³Green project that was funded by the Federal Ministry of Education, Science and Research of the Republic of Austria (BMBWF) through the Sparkling Science 2.0 program.

References

[1] Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J. and Wilderman, C. C., (2009). Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education . A CAISE Inquiry Group Report. [Online]. Available: Department of Education, https://files.eric.ed.gov/fulltext/ED519688.pdf. [Accessed July 24, 2023].

[2] Fischer, E. M., Sippel, S. and Knutti, R., (2021). Increasing Probability of Record-Shattering Climate Extremes. Nature Climate Change . Vol. 11, 689-695. https://doi.org/10.1038/s41558-021-01092-9.

[3] Kermish-Allen, R., Sibuma, B., Switzer, A. and MacPhee, M., (2017). Citizen Science, CIRCL Primer Series. [Online]. Available: Circlcenter, http://circlcenter.org/citizen-science.

[4] Bonney, R., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K. V. and Shirk, J., (2009). Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience, Vol. 59(11), 977-984. https://doi.org/10.1525/bio.2009.59.11.9.

[5] Peter, M., Diekotter, T., Hoffler, T. and Kremer, K., (2021). Biodiversity Citizen Science: Outcomes for the Participating Citizens. People and Nature, Vol. 3, 294-311. https://doi.org/10.1002/pan3.10193.

[6] Hennig, S., (2020). Motivation and its Consideration in Participatory Spatial Data Contribution. The Professional Geographer, Vol. 72(2), 238-252. https://doi.org/10.1080/00330124.2019.1676799.

[7] Pánek, J., (2016). From Mental Maps to Geoparticipation. The Cartographic Journal, Vol. 53(4), 300-307. https://doi.org/10.1080/00087041.2016.1243862.

[8] Hecker, S., Wicke, N., Haklay, M. and Bonn, A., (2019). How Does Policy Conceptualise Citizen Science? A Qualitative Content Analysis of International Policy Documents. Citizen Science Theory and Practice, Vol. 4(1), https://doi.org/10.5334/cstp.230.

[9] Cohn, J. P., (2008). Citizen Sscience: Can Volunteers Do Real Research? BioScience, Vol. 58(3), 192-197.

[10] Herfort, B., Eckle-Elze, M., Porto De Albuquerque, J. and Zipf, A., (2015). Towards Assessing the Quality of Volunteered Geographic Information from OpenStreetMap for Identifying Critical Infrastructures. Conference Proceedings of the ISCRAM 2015 Conference,May 24-27, 2015, Kristiansand, Norway.

[11] Haltofová, B., (2019). Critical Success Factors of Geocrowdsourcing use in E-Government: A Case Study from the Czech Republic. Urban Research & Practice,Vol. 13(4), 435-451. https://doi.org/10.1080/17535069.2019.1586990.

[12] Damodaran, L., (1996). User Involvement in the Systems Design Process-A Practical Guide for Users. Behaviour & Information Technology, Vol. 15(6), 363-377. https://doi.org/10.1080/014492996120049.

[13] Kujala, S., Kauppinen, M., Lehtola, L. and Kojo, T., (2005). The Role of User Involvement in Requirements Quality and Project Success. 13th IEEE, 75-84. https://doi.org/10.1109/RE.2005.72.

[14] Hennig, S., (2022). Contributing to Planning for Sustainability: Advancing User Involvement for User-Centered Geoparticipation Applica- tions. GI_Forum Journal, Vol. 10(1), 3-16. https://doi.org/10.1553/giscience2022_01_s3.

[15] Baek, E. O., Cagiltay, K., Boling, E. and Frick, T., (2007). User-Centered Design and Development. Handbook of Research on Educational Communications and Technology. (Eds.,) Abingdon, New York: Routledge, 2007, 660-668.

[16] Mithun, A. and Yafooz, W., (2018). Extended User Centered Design (UCD) Process in the Aspect of Human Computer Interaction. ICSCEE 2018 , 1-6. https://doi.org/10.1109/ICSCEE.2018.8538388.

[17] Wilkinson, C. R. and De Angeli, A., (2014). Applying User-Centred and Participatory Design Approaches to Commercial Product Development. Design Studies,Vol. 35, 614-631. https://doi.org/10.1016/j.destud.2014.06.001.

[18] Roth, R., Ross, K. and MacEachren, A. M., (2015). User-Centered Design for Interactive Maps: A Case Study in Crime Analysis. ISPRS Int. J. Geo-Inf., Vol. 4(1), 262-301. https://doi.org/10.3390/ijgi4010262.

[19] Usability.gov, (2017). User-Centered Design Basics. [Online]. Available: Usability.gov, https://www.usability.gov. [Accessed July 24, 2023].

[20] Tuttle, G., (2021). What is Design Thinking and Why is it Important? [Online]. Available: Wework Ideas, https://www.wework.com. [Accessed July 24, 2023].

[21] IDEO, U., (2023). What is Design Thinking? [Online]. Available: IDEO U, https://www.ideou.com. [Accessed July 24, 2023].

[22] Dam, R. F. and Siang, T. Y., (2023). What is Design Thinking and Why Is It So Popular? [Online]. Available: Interaction Design Foundation, https://www.interaction-design.org.

[23] Kensing, F. and Blomberg, J., (1998). Participatory Design: Issues and Concerns. Computer Supported Cooperative Work.Vol. 7, 167-185. https://doi.org/10.1023/A:1008689307411.

[24] Sommerville, I., (2012). Software Engineering. München: Pearson Stadium IT. https://engineering.futureuniversity.com/BOOKS%20FOR%20IT/Software-Engineering-9th-Edition-by-Ian-Sommerville.pdf.

[25] Pressman, R. and Maxim, B., (2019). Software Engineering: A Practitioner"s Approach . India: McGraw.

[26] Munassar, N. and Govardhan, A., (2010). A Comparison Between Five Models of Software Engineering. International Journal of Computer Science Issues, Vol. 7(5), 94-101. https://www.ijcsi.org/papers/7-5-94-101.pdf.

[27] Stoica, M., Mircea, M., and Ghilic-Micu, B., (2013). Software development: agile vs. traditional. Informatica Economica,Vol.17(4), 64-76. https://doi.org/10.12948/issn14531305/17.4.2013.06.

[28] Subbarayudu, B., Harshika, S., Amareswar, E., Reddy, R. G. and Reddy, K. K., (2017). Review and Comparison on Software Process Models. IJMET,Vol. 8(8), 967-980. https://iaeme.com/MasterAdmin/Journal_uploads/IJMET/VOLUME_8_ISSUE_8/IJMET_08_08_105.pdf.

[29] West, S. and Pateman, R., (2016). Recruiting and Retaining Participants in Citizen Science: What Can Be Learned from the Volunteering Literature? Citizen Science: Theory and Practice, Vol. 1(2), https://doi.org/10.5334/cstp.8.

[30] Agarwal, B. B., Tayal, S. P. and Gupta, M., (2010). Software Engineering and Testing . Sudbury, Massachusetts: Jones and Bartlett Publishers.

[31] Kumar, S., (2021). What is Prototype Model- Advantages, Disadvantages and When to use it? [Online]. Available: TRY QA, http://tryqa.com.

[32] Schnelle, E., (1979). The Metaplan Method: Communication Tools for Planning and Learning Groups . Metaplan Series No. 7, Hamburg: Quickborn.

[33] ASQ., (2022). What is an Affinity Diagram? K-J Method. [Online]. Available: American Society for Quality, https://asq.org. [Accessed April, 20, 2023].

[34] François, M., Osiurak, F., Fort, A., Crave, P. and Navarro, J., (2017). Automotive HMI Design and Participatory User Involvement: Review and Perspectives. Ergonomics,Vol. 60(4), 541-552. https://doi.org/https://doi.org/10.1080/00140139.2016.11 88218.

[35] Hennig, S. and Vogler, R., (2016). User-Centered Map Applications Through Participatory Design: Experiences Gained During the "YouthMap 5020" Project. The Cartographic Journal,Vol. 53(3), 213-229. https://doi.org/10.1080/00087041.2016.1148217.

[36] Peris, M., Sperling, A., Blinn, N., Nüttgens, M. and Gehrke, N., (2011). Participatory Design of Web 2.0 Applications in some Networks. 24th Bled eConference eFuture, 2011, Bled, Austria.

[37] Sanders, E., (2002). From User-Centered to Participatory Design Approaches. Design and the Social Sciences, J. Frascara, Ed. London: Taylor & Francis, 2002, 1-7.

[38] Fidgeon, T., (2005). User-centered Design (UCD)-6 Methods [Online]. Codeprojekt, https://www.codeproject.com. [Accessed Feb. 18, 2021].

[39] UNECE, (2008). Standards of Public Participation. Recommendations for Good Practice [Online]. UNECE, https://unece.org. [Accessed July 24, 2023].

[40] CABAR Central Asian Bureau for Analytical Reporting. (2021). Urban Activism in Central Asia is in its Infancy [Online]. Available: CABAR, https://cabar.asia/en/urban-activism-in-central-asia-is-in-its-infancy. [Accessed July 24, 2023].

[41] Mamedov, G. and Shatalova, O., (2016). Ponyatiya o sovetskom v Tsentral"noy Azii //Al"manakh Shtaba No.2. Tsentral"noaziatskoye khudozhestvenno-teoreticheskoye izdaniy

[42] Carretero, S., Vuorikari, R. and Punie, Y., (2017). DigComp 2.1: The Digital Competence Framework for Citizens: With Eight Proficiency Levels and Examples of Use [Online]. Available: JRC Science Hub, https://www.zuzendari.net/site_argitalpenak/docs/400_tic/4002017002en_Pub_EU_DigComp_en.pdf. [Accessed Sept. 29, 2023].

[43] Bundesministerium für Digitalisierung und Wirtschaftsstandort. (2021). Digitales Kompetenzmodell für Österreich: DigComp 2.2 AT. [Online]. Available: https://www.bmaw.gv.at/dam/jcr:54bbe103-7164-494e-bb30-cd152d9e9b33/DigComp2.2_V33-barrierefrei.pdf. [Accessed Sept. 29, 2023].

[44] Gryl, I. and Jekel, T., (2012). Re-centring Geoinformation in Secondary Education: Toward a Spatial Citizenship Approach. Cartographica: The International Journal for Geographic Information and Geovisualization , Vol. 47(1), 18-28. https://doi.org/10.3138/carto.47.1.18.

[45] McCall, M. K. and Dunn, C. E., (2012). Geo-Information Tools for Participatory Spatial Planning: Fulfilling the Criteria for "Good"governance?, Geoforum, Vol. 43(1), 81-94. https://doi.org/10.1016/j.geoforum.2011.07.007.

[46] Rinner, C. and Bird, M., (2009). Evaluating Community Engagement through Argumentation Maps-A Public Participation GIS Case Study. Environment and Planning B: Planning and Design , Vol. 36(4), 588-601. https://doi.org/10.1068/b34084.

[47] Lee, J. and Bednarz, R., (2009). Effect of GIS Learning on Spatial Thinking. Journal of Geography in Higher Education, Vol. 33(2), 183-198. https://doi.org/10.1080/03098260802276714.