Automatic Testing Mechanism (ATM)

This project designed and demonstrated a mechanism to automatically compute a safety rating, akin to the NCAP rating, for RAS to help users and regulators to identify and compare the safety of different RAS.

The challenge

As RAS become more commonplace the public require a simple indicator of the product’s safety. This project designed and demonstrated a mechanism to automatically compute a safety rating, akin to NCAP rating, for RAS. This is to help users and regulators to identify and compare the safety of different RAS.

The research

The team’s mechanism exploits the observation that if trajectories that commonly cause catastrophic accidents are similar to trajectories commonly taken by the human when interacting with the RAS, then a slight error on the RAS will likely cause the accident.

In any given scenario (e.g. a pedestrian crossing the road in front of an autonomous car) the team first computed the human behaviours that could cause a collision with the car. If these “kamikaze” behaviours are close to the common behaviours a pedestrian takes to cross the road safely, then even slight changes in the human common behaviours or the car’s behaviour could cause a collision between the car and pedestrian.

The safety indicator is based on the average distance between safe trajectories and their closest “kamikaze” trajectories. The safe trajectory can be provided by a regulatory body, data, or synthetically generated. The work shows the probability that a small alteration changes a safe trajectory into an unsafe one can be upper bounded by a value inversely proportional to the proposed safety indicator.

The results

The team tested a preliminary implementation of the safety indicator on a simple simulation of an autonomous car in avoiding a pedestrian crossing a road. They have done this where the car is a variety of distances from the pedestrian and with 100 “kamikaze” trajectories and one common trajectory.

The image above is an illustration of the approach based on a simulation run in CARLA. The green path indicates a safe path for the human to cross the street from left to right. The red path indicates a kamikaze path corresponding to the green safe path. The distance between multiple such paths is used to measure how safe the car system is.

The initial results indicate that for safer car behaviour, the similarity measure generated by the assessment mechanism is relatively high, though this does vary. The team has a current hypothesis that those variances are caused by the variance on the pedestrian’s understanding of the behaviour and state of the car. They are carrying out further experiments to ascertain this, and are testing the mechanism on more complex scenarios. 

The mechanism can be applied to various types of RAS and various tasks of RAS, as long as:

• the “kamikaze” behaviours can be computed
• common behaviours data are available or can be computed
• it’s possible to compute the similarity between these two sets of behaviours

Project partners

• Australian National University