1.1.3 Defining operating scenarios

1.1.3 Practical guidance - automotive

Author: Dr Richard Hawkins, Assuring Autonomy International Programme

In order to identify hazards and hazardous events for an autonomous automotive system requires that the operating scenarios of that system are defined as completely and clearly as possible. It is not possible for an open environment, such as that encountered on public roads, to provide an exhaustive detailed specification of the operating scenarios, since the set of scenarios is too large. For this reason the level of detail with which the scenarios are defined is important. The scenarios should be as simple as possible, to enable coverage, but not so simple that important characteristics (that may impact upon the hazard analysis) are not described. 

Automotive standards such as ISO 26262 [2] or ISO PAS 21448 [3] do not provide any clear guidance on how to define operating scenarios for the vehicle. Guidance has been provided by NHTSA [14] on the nature of behavioural competencies. Behavioural competencies are a way of defining what an automated vehicle is required to do to operate in the traffic conditions that it will regularly encounter, including keeping the vehicle in the lane, obeying traffic laws, following reasonable etiquette, and responding to other vehicles, road users, or commonly encountered hazards. The competencies include, for example:

  • Detect and respond to speed limit changes and speed advisories
  • Detect and respond to existing traffic (merging into existing traffic)
  • Detect and respond to encroaching oncoming vehicles from opposing lane into our lane
  • Perform car following (including stop and go)
  • Detect and respond to stopped vehicles
  • Detect and respond to static obstacles in the path of the vehicle
  • Detect and respond to stop/yield signs
  • Navigate intersections and perform turns
  • Detect and respond to work zones and people directing traffic in unplanned or planned events
  • Make appropriate right-of-way decisions
  • Follow local and state driving laws
  • Yield to pedestrians and bicyclists at intersections and crosswalks
  • Provide safe distance from vehicles, pedestrians, bicyclists on side of the road (with or without bike lanes)

Other examples of similar behavioural competencies have been defined, such as by Waymo [15]. On their own however, such lists of competencies do not provide the detailed descriptions of operational scenarios that are required as part of hazard analysis. Indeed NHTSA itself notes that these are necessary, but by no means sufficient, capabilities for public operation. The operating scenarios must include classes of traffic situations to which the vehicle will be exposed and their accompanying environmental conditions. These traffic situations can be described in terms of scenes and scenarios, defined in [1] as follows:

  • A scene describes a snapshot of the environment including the scenery and dynamic elements, as well as all actors’ and observers’ self-representations, and the relationships among those entities. Scene descriptions will inevitably be incomplete and from one or several observers’ points of view.
  • A scenario describes the temporal development between several scenes in a sequence. Every scenario starts with an initial scene, and the temporal development is characterised by a set of actions and events.

Scenarios should cover not just nominal situations that the system would be expected to be exposed to, but also unexpected, but predictable scenarios such as impaired vision, unexpected objects in the environment, or degraded actuation.
The discussion in [4] identifies the following potential sources of scenarios:

  • Expert knowledge:
    • TSC report “Taxonomy of Scenarios for Automated Driving” [5]
    • NHTSA “Federal Automated Vehicles Policy” [6]
  • Pre-existing scenario repositories (both from R&D projects and from industry) – see discussion below.
  • Pre-existing collision databases such as the German GIDAS database [7], or UK Road Accident and safety statistics data [8]
  • Data recorded from sensor-equipped vehicle fleets e.g. MOVE_UK CAV2 project [9]

Pre-existing scenario repositories

There are a number of efforts internationally to document a curated set of vehicle behaviours and scenarios. The largest is the German PEGASUS project [10] that captures what are considered to be appropriately safe system-level behaviours for the AV. The project has established a standard called OpenSCENARIO [11] for encoding scenarios that they hope will in time become a cross-platform industry-wide standard that allows scenarios to be shared widely across multiple organisations. Traffic Sequence Charts (TSCs) [12] are an extension of the OpenSCENARIO approach that provide a formal semantics for describing the required dynamic behaviour of the vehicle within different sets of situations.

Voyage has created a set of publically accessible scenarios that an AV might encounter while operating on public or private roads [13]. The scenarios are divided into behavioural sections, each containing multiple scenarios (see example in figure 1). The scenarios are divided into one or more steps. Each step has an image, scenario description, and expected result:

  • Image: The initial state of the world. It shows where the Ego vehicle and any actors exist in space, the current state of Ego, and what actions the actors should take next.
  • Scenario Description: The actors’ behaviour in that step.
  • Expected Result: The expected behaviour from Ego.

Figure 1 – Example scenario from [13]

All the scenarios are parameterised with key variables, such as speed of vehicles and distance of Ego, whose values can be changed to create multiple scenarios.

Validation of operating scenarios

It is important from an assurance perspective that evidence is provided as to the validity of the defined set of operating scenarios. The first way to achieve this is through adopting a systematic approach such as described above. This helps to provide an argument of completeness with respect to the operating space. In addition, evidence can be provided based upon review of the scenario specification. A rigorous specification to a defined format makes the scenarios more amenable to review and reduces ambiguity. It is important that review is carried out by a range of experienced stakeholders. Providing simulations of the defined scenarios may help to ensure the reviewers have a correct understanding of the scenarios. The defined scenarios can also be checked against collected field data from vehicles in operation to ensure that all encountered scenarios are captured in the operating scenario specification.

Summary of approach

  • Define scope of operation (ODD) – see Objective 1.1.2
  • Define operating scenarios to give coverage of the ODD
  • Validate defined operating scenarios
  • Check coverage of the ODD provided by the scenarios

 

References

 

Related links

Download this guidance as PDF:

Related links

Download this guidance as PDF: