1.1.3 Defining the operating environment

Author: SAM demonstrator project

The designers of artificial intelligence (AI) and machine learning (ML) applications need to scope, bound and articulate clearly the situations for which the application is going to be used, and how it is going to be used. In the case of clinical settings, it is very likely that even autonomous systems will have a significant degree of interaction with people. For example, an autonomous infusion pump will require interaction with the nurse in case of unexpected patient deterioration.  

It is important that the definition of operating scenarios is done based on operational realities (work-as-done) rather than through an abstract view of what should be done in principle (work-as-imagined). Typically, a range of situations needs to be considered, such as routine operational scenarios, exceptional or emergency response scenarios, and maintenance and inspection scenarios. Understanding of the operational scenario includes consideration of what specifically needs to be done by the application and by any users, in what kind of order different activities need to be done, what kinds of information are required to complete an activity, what forms of interactions and communication take place, and what other activities people interacting with the application might be engaged with at the same time.  

Definition of operating scenarios can make use of analysis techniques for understanding and representing clinical work. Examples include Hierarchical Task Analysis (HTA) [1] and Functional Resonance Analysis (FRAM) [2].  

HTA represents human activities based on a theory of goal-directed behaviour, and includes a hierarchy of goals and sub-goals linked by plans, which describe how sub-goals combine to achieve the higher-level goal. Plans can be used to express any kind of algorithm. For example, simple sequential ordering (such as do step 1 to step 3 in order), free ordering (do steps 1, 2, 3 in any order), as well as more complex loops (such as do step 1 and step 2 in order until signal A is active, then do step 3). This representation creates a tree-like structure, where the leaves represent task steps that are considered elementary (e.g. basic manual operations) or where further decomposition is not considered necessary.  

FRAM decomposes the clinical system into functions, to move away from “what a system is” to “what it does”. Each function is examined for its potential performance variability, then interactions between functions are examined. “Functional resonance” is used to describe how outcomes can “emerge” from everyday variability of many functions, to move away from simple notions of “cause and effect”.

References

• [1] Stanton N. Hierarchical task analysis: Developments, applications, and extensions. Appl Ergon. 2006;37.
• [2] Hollnagel E. FRAM, the functional resonance analysis method: modelling complex socio-technical systems: Ashgate Publishing, Ltd.; 2012.