Resilience Talk 22 -  6 June 2022, 15:00-16:00 BST 

The Missing Links? Broken links and nodes as a way of modelling resilience (or the lack of) in sociotechnical system networks

Professor Neville Stanton, University of Southampton

In this talk I will present resilience from a sociotechnical systems perspective.  This perspective considers the interaction between humans and technology within broader systems constraints.  For the purposes of this talk, I will show how the failure of information to be exchanged, transacted or communicated between agents and tasks can led to incidents.  The approach taken models sociotechnical systems as networks: task networks (to model the interdependencies of taskwork), social networks (to model the relationships between agents) and information networks (to model the information structures) as well as composite networks (which combine the task, social and information networks).  The broken links and broken nodes approaches have been used to identify the potential consequences of structural weaknesses in the networks, with the aim of improving the resilience of those networks to disturbances. Whilst the approaches are very much in development, it is hoped that this talk will inspire those attending to apply the methods in their own research.

Resilience Talk 21 -  25 April 2022, 14:00-15:00 BST 

The Role of Trustworthy Autonomous Systems in Digital Twins of the Natural Environment

Professor Gordon Blair, Lancaster University

Digital twins are increasingly important in many domains including in understanding and managing the natural environment. Digital twins of the natural environment are fuelled by the unprecedented amounts of environmental data now available from a variety of sources from remote sensing to potentially dense deployment of earth-based sensors. Because of this, data science techniques, including AI and machine learning techniques, inevitably have a crucial role to play in making sense of this complex, highly heterogeneous data. This seminar will start by reflecting on the role of data science in digital twins of the natural environment, with particular attention on how resultant data models can work alongside the rich legacy of process models that exist in this domain. We then move on to unpicking the complex two-way relationship between data and process understanding. By focusing on the interactions, we end up with a template for digital twins that incorporate a rich, highly dynamic learning process with the potential to handle the complexities and emergent behaviours of this important area. There is however very little experience of such self-adapting, self-organising structures in the environmental science and hence the seminar will conclude by setting up a discussion about the interplay between digital twins of the natural environment and trustworthy autonomous systems, noting the important additional role in trust in supporting decision making in this critical domain.

Resilience Talk 20 -  11 April 2022, 15:00-16:00 BST 

The Road to Collaboration is Paved with Goals

Amel Bennaceur, Director of Research for the School of Computing and Communications, The Open University, UK

Whether it be robots, services, software, or humans, collaboration enables achieving goals that cannot be achieved individually. This talk explores the relationship between autonomy and collaboration. I will show how goal models can be used to reason about and enable collaboration between autonomous agents by composing their capabilities and coordinating their behaviour. While significant progress has been made for enabling collaboration between autonomous systems, collaboration with humans raises additional challenges, which I will explore in the talk and present our ongoing work on using social identity theory to enable collaboration between autonomous agents and humans.

Resilience Talk 19 -  28 March 2022, 15:00-16:00 BST 

Remote maintenance and robotics challenges for nuclear fusion

Robert Skilton, Head of Research, RACE, UK Atomic Energy Authority

RACE is a UK centre for robotics in challenging environments and is primarily focused on enabling nuclear fusion energy but also involved in transformation of nuclear decommissioning. .  In this talk, Rob will discuss  his work which specialises in robotics, autonomy, and perception for operation and inspection in hazardous environments. 

Resilience Talk 18 -  14 March 2022, 15:00-16:00 GMT

Use of Assertion-based Simulation Testing for Validation of Trustworthiness (safety) and Resilience Properties in Autonomous Vehicles and other systems

Chris Harper, Research Fellow - Robotics Safety and Control, University of the West of England, UK

Validation of trustworthiness properties such as safety is difficult to achieve for autonomous systems, owing to the fact that those properties may need to be established over much larger and possibly even unbounded state spaces than has previously been necessary for manually controlled or basic automatic systems. While formal verification may be feasible of individual behavioural properties, it becomes impractical to co-verify multiple safety (or other) properties together in a manner that provides wide coverage of situated state spaces. Simulation testing and physical tests can exercise a complete system for all its properties together, a methodology we at Bristol Robotics Laboratory (BRL) refer to as “Corroborative V&V”. At BRL we have been investigating the use of simulation as a means to complement formal verification through the checking of assertions about the behaviour of autonomous systems, which can determine whether properties were violated during a given simulation run. We are developing a simulation testbench facility to perform these checks automatically in batch mode from an input test scenario. We review our development of a set of safety assertions for autonomous vehicle simulations derived from the UK Highway Code (UKHC), which we argue can provide evidence of safe driving behaviour in an autonomous vehicle.

Assertion-based validation by simulation could be used to evaluate post-facto resilience properties as well as preventative safety properties. There are few explicit rules in the UKHC regarding resilient behaviour, but it looks possible to develop assertion checks for resilience, and in this presentation we explore how this might be done.

Resilience Talk 17 -  28 February 2022, 15:00-16:00 GMT

Trustworthy Human-AI Partnerships

Gopal Ramchurn, Professor of Artificial Intelligence, University of Southampton, UK

Recent advances in AI, Machine learning and Robotics have significantly enhanced the capabilities of machines. Machine intelligence is now able to support human decision making, augment human capabilities, and, in some cases, take over control from humans and act fully autonomously. Machines are becoming  more tightly embedded into systems alongside humans, interacting and influencing each other in a number of ways. Such human-AI partnerships are a new form of socio-technical system in which the potential synergies between humans and machines are much more fully utilised. Designing, building, and deploying human-AI partnerships present a number of new challenges as we begin to understand their impact on our physical and mental well-being, our personal freedoms, and those of the wider society. In this talk I will focus on the challenges in designing trustworthy human-AI partnerships. I will explore the multiple elements of trust in human-AI partnerships and discuss the associated research challenges.I will also aim to identify the risks associated with human-AI partnerships and therefore determine the associated measures to mitigate these risks. I will conclude by giving a brief overview of the UKRI Trustworthy Autonomous Systems Programme (www.tas.ac.uk), a £33m programme launched in 2020 involving over 20 universities, 100+ industry partners, and over 200 researchers.

Resilience Talk 16 -  17  January 2022, 15:00-16:00 GMT

Towards Self-Aware Artificial Intelligence –  Lessons Learned from Optimal Estimation Theory

Nidhal Bouaynaya, Professor in Electrical and Computer Engineering, Rowan University, USA

Deep neural networks (DNNs) have surpassed human-level accuracy in various fields, holding the promise of emerging technologies, such as self-driving cars and autonomous unmanned aircraft systems, smart cities infrastructure, personalized treatment in medicine, and cybersecurity. However, unlike Humans who have a natural cognitive intuition for probabilities, DNN systems - being inherently deterministic - are unable to evaluate their confidence in the decisions. To truly deserve its name, an artificial intelligence system must be aware of its limitations and have the capacity for insightful introspection.  

This talk will advance Bayesian deep learning methods that are able to quantify their uncertainty in the decision and self-assess their performance, are robust to adversarial attacks, and can even expose an attack from ambient noise. This talk will establish the theoretical and algorithmic foundations of uncertainty or belief propagation through complex deep learning models by adopting powerful frameworks from optimal estimation problems in non-linear and non-Gaussian dynamical systems.  

The challenge in DNNs is the multi-layer stages of non-linearities in deep learning models, which makes propagation of high-dimensional distributions mathematically intractable. Drawing upon powerful statistical frameworks for density propagation in non-linear and non-Gaussian dynamical systems, we introduce Tensor Normal distributions as priors over the network parameters and derive a first-order Taylor series mean-covariance propagation framework. We subsequently extend this first-order approximation to an unscented framework that propagates sigma points through the model layers. The unscented framework is shown to be accurate to at least the second-order approximation of the posterior distribution. We finally learn the entire predictive distribution using Particle Filtering, a powerful class of numerical methods for the solution of optimal estimation problems in non-linear, non-Gaussian systems. The uncertainty in the output decision is given by the propagated covariance of the predictive distribution. Furthermore, we show that the proposed framework performs an automatic logit squeezing, which leads to significantly enhanced robustness against noise and adversarial attacks. Experimental results on benchmark datasets, including MNIST, CIFAR-10, real-world synthetic aperture radar (SAR), and Brain tumor segmentation (BraTS 2015), demonstrate: 1) superior robustness against Gaussian noise and adversarial attacks; 2) self-assessment through predictive confidence that monotonically decreases with increasing levels of ambient noise or attack; and 3) an ability to detect a targeted attack from ambient noise.

Resilience Talk 15 - Monday 22 November 2021, 14:30-15:30 GMT

Verifying Machine Ethics

Louise Dennis, University of Manchester, UK 

Machine ethics is concerned with the challenge of constructing ethical and ethically behaving artificial agents and systems.  One important theme within machine ethics concerns explicitly ethical agents – those which are not ethical simply because they are constrained by their programming or deployment to be so but which use a concept of ethics in some way as part of their operation.  Normally this requires the provision of rules, utilities or priorities by a programmer, knowledge engineer or user.  In this talk I will address the question of how such explicitly ethical programs can be verified.  What kind of properties can we consider and what kind of errors might we find?

Resilience Talk 14 - Monday 8 November 2021, 15:00-16:00 GMT

Assuring the Safety of Resilient Robotics and Autonomous Systems – Current Capability and Evidence Requirements

Dr. Mark Nicholson, Senior Lecturer - Safety Critical Systems, Department of Computer Science, University of York, UK

Robotics and Autonomous Systems (RAS) have unique characteristics and requirements that present a considerable challenge for safety assurance and certification. This webinar explores the current state of the art of the assurance of complex software intensive systems, introducing a set of principles by which assurance and certification is currently judged in a system context. It then briefly looks at the challenges introduced by the use of RAS before considering the emerging responses to these challenges. One emerging theme is that assuring the safety of RAS is a multifaceted undertaking, which requires that assurance evidence from a variety of sources is assembled into a coherent whole. Further the validity of this evidence, and hence the safety assurance inferences made over it, are constantly challenged in operation. Building in appropriate levels of resilience ensures that the evidence and safety assurance inferences are not fragile.

Resilience Talk 13 - Monday 11 October 2021, 15:00-16:00 BST

Human-Centred Visual Learning and Its Applications

Hyung Jin Chang, University of Birmingham, UK 

Humans are the best teachers as well as beneficiaries of AI development. My human-centred visual learning research is to develop vision-based algorithms that aim to make systems usable and useful by focusing on humans, especially their needs and requirements. In particular, understanding human body pose/hand/gaze is fundamental for meaningful interpretation of human action and behaviour. In this talk, I will introduce my recent research on human-centred vision tasks in a high level, including real-time hand pose & gaze tracking, and human body movement retargeting, and so forth. Also, some interesting applications using the human-centred vision methods will be presented.

Resilience Talk 12 - Monday 5 July 2021, 15:00-16:00 BST

Mission Planning for Long-Term Autonomy in Mobile Service Robots

Nick Hawes, Oxford Robotics Institute, University of Oxford UK 

The performance of autonomous robots, i.e. robots that can make their own decisions and choose their own actions, is becoming increasingly impressive, but most of them are still constrained to labs, or controlled environments. In addition to this, these robots are typically only able to do intelligent things for a short period of time, before either crashing (physically or digitally) or running out of things to do. In order to go beyond these limitations, and to deliver the kind of autonomous service robots required by society, we must conquer the challenge of combining artificial intelligence and robotics to develop systems capable of long-term autonomy in everyday environments. The first half of this work will provide a retrospective of our prior work on long-term autonomy, specifically the work of the EU STRANDS project. The second half of the talk will cover more recent work on mission planning under uncertainty for service robots which grew out of this project. 

Resilience Talk 11 - Monday 21 Jun 2021, 15:00-16:00 BST

Resilience through Adaptation - the challenge of change 

Jeff Kramer, Imperial College London, UK 

Change in complex systems is inevitable. Providing rigorous techniques and tools to support adaptation so that it can be performed online, at runtime, is certainly challenging. However the potential resilience rewards could be great.  There is the need for an architecture and runtime support for dynamic software configuration, plan execution and plan synthesis, domain environment modelling and monitoring, and ultimately even potentially performing some elements of requirements engineering at runtime! This talk will present our motivation and vision, describing our work to date and our hopes for the future.

Resilience Talk 10 - Monday 7 June 2021, 15:00-16:00 BST

Responsible Robotics - from (ethical) principles to practice

Alan Winfield , University of the West of England, Bristol UK 

In this talk I outline the work of EPSRC project RoboTIPS. I will define and position Responsible Robotics between Responsible Research and Innovation (RRI) and Robot Ethics. I will introduce the essential practice of Ethical Risk Assessment, then describe our work on the Ethical Black Box and its role in Social Robot Accident Investigation.

Resilience Talk 9 - Monday 24 May 2021, 15:00-16:00 BST

Human Factors / Ergonomics (HF/E) of AI in Healthcare

Mark Sujan, University of Warwick and Human Factors Everywhere, UK 

There is a rapidly growing number of publications, reports and other documents in the area of AI in healthcare. However, the focus in many of these publications is technology-centric, i.e. they address issues around data and performance of algorithms. In practice, AI applications will almost always be part of a wider clinical system, and taking this systems perspective enables us to consider important HF/E aspects that require much more attention.

In this presentation I will discuss such HF/E considerations, which arise from the interaction of people with the AI, such as user workload, trust and satisfaction, and patient safety. I will draw upon examples and insights generated by two demonstrator projects funded by the Assuring Autonomy International Programme: (1) The SAM project, which was concerned with the use of autonomous infusion pumps in intensive care; (2) the ASSIST project, which looks at the use of AI to support ambulance service call handlers in the recognition of cardiac arrest.

Resilience Talk 8 - Monday 10 May 2021, 15:00-16:00 BST

Interdisciplinary Approaches to understanding data and AI ethics from JUST AI: Joining Up Society and Technology for AI

Alison Powell,  Imre Bard and Louise Hickman,  London School of Economics and Political Science

As AI and other data-intensive technologies appear to sink into the background, questions of their ethics extend in all directions. No longer do ethical concerns extend only to questions about the principles which ought to guide the development of AI systems, nor the ethical quality of outcomes generated through the use of AI systems, this now also encompasses the practice of ethics within the contexts where AI works. These fields are changing fast, growing in range, depth and influence in the UK and across the world. 

The JUST AI project has been working to create interdisciplinary strategies for representing and intervening in this field. Some of our core research questions concern how ethics is defined, understood and practiced and what dynamics influence the inclusion, exclusion or marginalization of particular ideas about ethics.

In this seminar members of the JUST AI team will discuss our interdisciplinary work mapping, understanding and intervening in the definitions and relationships defining AI and data ethics in the UK. We will discuss the challenges of using bibliometrics, data visualizations and surveys to understand these dynamic changes as well as share insights gained from our most recent phase of research.

Resilience Talk 7 - Monday 26 April 2021, 16:00-17:00 BST

Challenges and Opportunities in Robotic Space Exploration

John Day, NASA Jet Propulsion Laboratory, USA

Resilience Talk 6 - Monday 12 April 2021, 15:00-16:00 BST

Learning to Communicate in Multi-Agent Systems

Amanda Prorok, University of Cambridge, UK 

Effective communication is key to successful multi-agent coordination. Yet it is far from obvious what, how and when information needs to be shared among agents that aim to solve cooperative tasks. In this talk, I discuss our recent work on using Graph Neural Networks (GNNs) to solve multi-agent coordination problems. In my first case-study, I show how we use GNNs to find a decentralized solution to the multi-agent path finding problem, which is known to be NP-hard. I demonstrate how our  policy is able to achieve near-optimal performance, at a fraction of the real-time computational cost. Secondly, I show how GNN-based reinforcement learning can be leveraged to learn inter-agent communication policies. In this case-study, I demonstrate how non-shared optimization objectives can lead to adversarial communication strategies. Finally, I address the challenge of learning robust communication policies, enabling a multi-agent system to maintain high performance in the presence of anonymous non-cooperative agents that communicate faulty, misleading or manipulative information.

Resilience Talk 5 - Monday 29 March 2021, 15:00-16:00 BST

Towards Operational Cyber Resilience

Kerstin Eder, University of Bristol, UK 

Existing approaches to cyber security in the automotive sector are not fit to deliver the resilience required for safe mass deployment of advanced driving features and smart mobility services. In this seminar I will introduce an innovative multi-directional approach to operational cyber resilience, the CyRES methodology, which aims to enable the delivery of robust and resilient engineering practices in this sector from design, via manufacture to operation. CyRES is based on three principles: increasing the probability of Detection, Understanding and Acting on Cyber Events; increasing the number of Engineered Significant Differences; and invoking a continuum of Proactive Updates. I will motivate and explain these principles on examples, focusing mainly on the first two. CyRES is an exciting opportunity for engineers and computer scientists to re-target widely studied, mature methods, such as those developed by the self-adaptive systems community, for cyber security. The aim of this seminar is to raise awareness of the challenges and to highlight some of the ways for attendees to contribute to realise the CyRES vision.

This seminar covers research conducted jointly with Carsten Maple, WMG, University of Warwick, Peter Davies, Thales, UK, Chris Hankin, Imperial College London, Greg Chance, Trustworthy Systems Lab, University of Bristol and Gregory Ephiphaniou, WMG, University of Warwick as part of the Cyber Resilience in Connected and Autonomous Mobility project ResiCAV. 

Resilience Talk 4 - Monday 15 March 2021, 15:00-16:00 GMT

A journey through research in Informatics, publication, and evaluation

Carlo Ghezzi, Politecnico di Milano, Italy 

The talk is about research in Informatics (aka, Computer Science, Computing, ICT). It presents a few general principles and sheds light on what is specific of the field.

It focuses on two crucial aspects of research: publication -- how research results are disclosed to the world -- and evaluation -- how (and why) research and researchers are evaluated.

The talk is based on the book by the author titled "Being a Researcher-an Informatics Perspective", published in 2020 by Springer.

Resilience Talk 3 - Monday 1 March 2021, 15:00-16:00 GMT

Uncertainty in self-adaptive systems: How far have we come?

Danny Weyns, KU Leuven, Belgium

About a decade ago, the awareness grew that uncertainty is a key challenge for research in self-adaptive software systems. This talk looks at what we have achieved in this period and at the challenges ahead. I start by examining the knowledge we have obtained on handling uncertainty and illustrate that with one characteristic approach. Then, I will zoom in on the perceptions we have today on the notion of uncertainty in self-adaptive systems. Finally, I will peek into the future and present a viewpoint on how we may be able to push the boundaries in the ability of self-adaptive systems to handle uncertainty that originates from unanticipated changes.

Resilience Talk 2 - Monday 15 February 2021, 16:00-17:00 GMT

Learning About the Change: An Adaptive Approach to Active Learning

Mohammad Reza Mousavi, University of Leicester, UK

Active automata learning is a technique to learn behavioural models from black-box systems. Variability and evolution is inherent to much of the modern autonomous systems and hence, new sorts of active learning techniques are needed to learn about variability-intensive and evolving systems. In this talk, we first present the basic principles of active learning and then report on two novel techniques for learning variability-annotated models as well as efficient learning for evolving systems by identifying the commonalities and differences in the learning process. 

Based on joint work with Diego Damasceno (Radboud University Nijmegen, NL) and Adenilso Simao (University of Sao Paolo, BR):   https://www.cs.le.ac.uk/people/mm789/pub/mousavi-ifm-2019.pdf ; 

https://www.cs.le.ac.uk/people/mm789/pub/mousavi-emse-2020.pdf 

Resilience Talk 1 - Monday 01 February 2021, 15:00-16:00 GMT

The scope of autonomy in autonomous systems and ethics of decisions: a user-interacting perspective

Paola Inverardi, University of L'Aquila, Italy 

Autonomous systems make decisions independently or on behalf of the user.  This will happen more and more in the future, with the widespread use of AI technologies in the fabric of the society that impacts on the social, economic and political sphere.  Automatizing services and processes inevitably impacts on the user’s prerogatives and puts at danger their autonomy and privacy. From a societal point of view It is therefore crucial to understand which is the space of autonomy that a system can exercise without compromising laws and human rights.  

In this talk I will try to dig into this challenge by making use of philosophical proposals concerning ethics in the digital societies and software engineering design methodologies. In particular I will cast the problem of designing the scope of decisions of an autonomous system in the setting of architectural design where autonomous systems interact within a society, characterized by collective ethical values, with multiple and diverse users, each of them characterized by her individual moral preferences.