Summary

The world is undergoing a profound transformation in which systems controlled by software systems are increasingly used to support critical tasks across essential domains (e.g., healthcare, transportation, banking) characterized by high degrees of uncertainty introduced by the complex interactions with their human users, the use of machine learning components, nontrivial interdependencies between their physical elements and software, and rapidly changing environmental conditions. Hence, providing assurance about the safety and performance of such “software-intensive” systems under specified levels of uncertainty is crucial to their adoption.

During the last decade, researchers have made an important effort in supporting the analysis and management of software-intensive systems that operate under uncertainty by devising modeling notations, analysis, and assurance mechanisms that have increasingly started to capture and mitigate the effects of different types of uncertainty [1]. However, these solutions tend to tackle different types of uncertainty in isolation; yet, different uncertainty types are rarely independent and often interact, causing emergent effects that impact the achievement of system goals in subtle and often unpredictable ways [2, 3].

Indeed, these interactions can hinder the assurance and adoption of software-intensive systems. Consider, for instance, an autonomous service robot operating in a healthcare facility. When navigating between two hospital locations, this robot may face uncertainty due to: (i) its limited knowledge of the environment (e.g., presence of people in corridors, remaining energy in the battery -- which has to be estimated based on measured voltage), and (ii) an overly abstract model of the environment that does not represent the geometry of obstacles in detail and can increase the chance of collision and the need for subsequent recovery routines that increase energy consumption. These uncertainty sources, when considered together, can cause the robot to deplete its battery before completing its task, while individual sources of uncertainty would not have caused the same situation. For instance, if the robot has an abstract model that causes a collision, an accurate knowledge of the remaining battery and presence of people can allow re-planning that might still allow it to reach its target location. However, the same situation with uncertainty in the remaining battery, or the presence of humans who delay the progress of the robot through a corridor can lead to the generation of a plan based on unrealistic estimates, and therefore prone to make the robot fail its mission (e.g., due to battery depletion).

Our proposed seminar aims to further the advances made by other relevant Shonan and Dagstuhl seminars that have discussed the engineering of software-intensive systems under uncertainty, but have not explicitly acknowledged and therefore have not explored the pivotal role of the Uncertainty Interaction Problem (UIP) [2, 3] and the need for an explicit management of uncertainty interactions in building safer and more resilient software-intensive systems.

Topics to be discussed in the seminar

UIP concepts and terminology. Concepts related to uncertainty (e.g., nature, category, sources) have been coined and developed in different fields like statistics, economics, and computer science. Even within computer science, there are multiple taxonomies that employ different concepts, categorisations and terminology [1, 4, 5, 6]. Hence, the first topic of the seminar will be disentangling this mishmash of terminology and concepts to  reach a clear definition of uncertainty interaction and uncertainty-related concepts.

State-of-the-art methods for taming uncertainty and their integration. The need to manage different types and sources for uncertainty in SiS has fostered the development of various ad-hoc methods that address the specific issues induced by these uncertainties, often isolated from interactions with other sources of uncertainty, and for individual applications [7, 8, 9, 10]. These ad-hoc methods include: (i) representation of uncertainty and its propagation, (ii) analysis techniques able to provide guarantees about system behavior under prescribed levels of uncertainty, for instance using quantitative verification techniques such as probabilistic model checking [11], and (iii) mitigation of the effects of uncertainty, e.g., through adaptation techniques that are able to anticipate disruptions and mitigate their effects proactively [9, 10]. The discussion of these methods during the seminar are key for understanding their merits, limitations, and for making progress towards a common conceptual framework that allows their integration and exploitation.

Uncertainty interaction classification and patterns. To develop a common conceptual framework for managing uncertainty interactions, there is a need to identify the common categories of uncertainty interaction that affect the quality (e.g., safety, security) of SiS across strategic domains. Hence, the seminar will discuss: (i) identification of common types of uncertainty interaction across different domains and classes of system (e.g., ML-enabled systems, CPS), (ii) how to devise appropriate notations and patterns to represent such types of uncertainty interactions, as well as mitigation actions and strategies for their impact on system properties. This discussion will be informed by the expertise of the participants and driven by the set of preliminary challenges identified in [3].

Planned Outcomes

1. A common understanding and a survey of the research landscape on uncertainty interaction across disciplines. This will include an alignment of the state of the art of uncertainty management in the form of an ontology. We aim to achieve a better understanding of uncertainty-related concepts and their connections to facilitate communication, understanding of common problems and solutions.
2. A research agenda for uncertainty interaction in self-adaptive systems. This will include defining a set of open challenges that will unify and inform the future international research on UIP, and identifying and proposing preliminary lines of research that have the potential to lead to solutions for these challenges.
3. An initial suite of use cases for the evaluation of future UIP solutions. This will involve defining use cases in the application domains represented by industrial experts attending the seminar, including at least automotive, robotics and healthcare.
4. A preliminary repository of patterns for uncertainty interaction management. This repository will include descriptions of uncertainty interaction types, pairing them with blueprints of potential solutions. The contents of the repository will be informed, among other sources, by use cases from item 3.
5. A research roadmap for UIP. This will involve operationalizing the UIP research agenda from item 2 by (i) prioritizing and suggesting a high-level plan for the delivery of the research to address its objectives depending on their benefits, interdependencies and current state of progress; and (ii) proposing ways in which the use cases from item 3 can be leveraged to support this delivery.

Seminar Structure

We envisage a 3.5-day meeting with a structure summarized in Table 1. We will start with an introductory session, which will be followed by two rounds of break-out group work. Each round will end with a plenary in which groups will report about their discussion. The seminar will finish with a session to plan next steps and follow-up activities on Thursday morning. The two break-out rounds will focus on delivering outcomes mentioned in this document. In each round, the participants will be incorporated into different groups of between five and seven members (with a total of 27 participants, we may be looking at 3-4 groups). We will actively manage the partition process to aim at balancing group composition.

Table 1. Seminar structure with outcomes that activities will contribute to.

Bibliography

[1] Sara Mahdavi-Hezavehi, Danny Weyns, Paris Avgeriou, Radu Calinescu, Raffaela Mirandola, Diego Perez-Palacin: Uncertainty in Self-adaptive Systems: A Research Community Perspective. ACM Trans. Auton. Adapt. Syst. 15(4): 10:1-10:36 (2021)

[2] Javier Cámara, Javier Troya, Antonio Vallecillo, Nelly Bencomo, Radu Calinescu, Betty H. C. Cheng, David Garlan, Bradley R. Schmerl: The uncertainty interaction problem in self-adaptive systems. Softw. Syst. Model. 21(4): 1277-1294 (2022)

[3] Javier Cámara, Radu Calinescu, Betty Cheng, David Garlan, Bradley Schmerl, Javier Troya, Antonio Vallecillo. Addressing the Uncertainty Interaction Problem in Software-intensive Systems: Challenges and Desiderata. 25th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MoDELS 2022): 24-30 (2022)

[4] Andres J. Ramirez, Adam C. Jensen, and Betty H. C. Cheng. 2012. A taxonomy of uncertainty for dynamically adaptive systems. In Proc. of SEAMS’12. IEEE Computer Society, 99–108.

[5] Javier Troya, Nathalie Moreno, Manuel F. Bertoa, and Antonio Vallecillo. Uncertainty representation in software models: A survey. Softw. Syst. Model. 20, 4 (2021)

[6] Diego Perez-Palacin and Raffaela Mirandola. 2014. Uncertainties in the modeling

of self-adaptive systems: a taxonomy and an example of availability evaluation. In Proc. of ICPE’14. ACM, 3–14.

[7] Burton and B. Herd. Addressing uncertainty in the safety assurance of machine-learning. Frontiers in Computer Science, 5:31.

[8] X. Fang, R. Calinescu, C. Paterson, and J. Wilson. Presto: predicting system-level disruptions through parametric model checking. In Proceedings of the 17th Symposium on Software Engineering for Adaptive and Self-Managing Systems, pages 91–97, 2022.

[9] Gabriel A. Moreno, Javier Cámara, David Garlan, Bradley R. Schmerl: Proactive self-adaptation under uncertainty: a probabilistic model checking approach. ESEC/SIGSOFT FSE 2015: 1-12

[10] Hielscher, J., Kazhamiakin, R., Metzger, A., Pistore, M. (2008). A Framework for Proactive Self-adaptation of Service-Based Applications Based on Online Testing. In: Mähönen, P., Pohl, K., Priol, T. (eds) Towards a Service-Based Internet. ServiceWave 2008. Lecture Notes in Computer Science, vol 5377. Springer, Berlin, Heidelberg.

[11] Marta Z. Kwiatkowska, Gethin Norman, David Parker: Stochastic Model Checking. SFM 2007: 220-270

[12] Danny Weyns, Radu Calinescu, Raffaela Mirandola, Kenji Tei et al.: Towards a Research Agenda for Understanding and Managing Uncertainty in Self-Adaptive Systems. SIGSOFT Softw. Eng. Notes 48, 4 (October 2023), 20–36.