Using WIZARD of Oz prototyping to explore the safe automated-decision making tools in Air Traffic Control

The WIZARD research project team was supported by AAIP and consisted of members from the NATS R&D department collaborating with researchers from the Department of Computer Science at the University of York. The team was purposefully selected to ensure a diverse set of expertise, knowledge and skills within the project. In addition, both NATS R&D and University of York team members were able to reach out to relevant subject matter experts (SMEs) in each organisation to seek specific input, feedback and guidance at different stages of the project’s maturity.

Different perspectives, fresh results

From the start, we embraced a way of working that emphasised learning from each other, an openness to new ideas, and a desire to try new approaches. Throughout, we aimed to constructively explore different perspectives from the team members as well as the wider network of subject matter experts. We applied agile-based methodologies, including weekly sprints, tracking activities in a way which was visible to all team members, as well as regular reflections and discussions with the wider team. 

Additionally, the tasks were distributed in a way which leveraged each team member’s expertise but also allowed everyone to experience the different aspects of rapid prototyping techniques and how these are applied in practice. We encouraged the use of physical or digital developer diaries to record our experiences in a format which worked best for each of us. When necessary, deep dive exploration sessions and focus days were also organised by the team to dedicate time and effort to explore specific research challenges and opportunities which emerged. 

While we were geographically dispersed we met frequently online and also made sure to hold a small number of face-to-face meetings for the team to get to know each other and visit each other’s sites. To make sure we made the most out of the range of expertise involved in the project, members of the team from both sites were involved in the validation activities that took place in NATS and the Institute for Safe Autonomy. 

A unique journey into collaboration

Through new ways of working, the small, agile, dynamic and diverse WIZARD team of “living wizards”, distilled the following from this project’s unique R&D journey:

1. Rapid prototyping techniques such as Wizard of Oz can be learned, used and applied in air traffic control (ATC)  contexts to develop and bring simple as well as complex concepts to life. These provide a powerful means for sharing, communicating and conducting early validation of new ideas and concepts.
2. With a carefully designed team and a culture which enables new ways of working, it is possible to successfully develop low to higher fidelity affordable prototypes in short amounts of time. A concept can be evolved from scratch, engaging diverse stakeholders (including air traffic controllers and other domain experts) across the business and gathering rich feedback to inform future development efforts and strategy.
3. The rapid prototyping techniques which were tested, coupled with a dynamic way of working, can catalyse new concept development, with potential for this approach to scale and cater for a wide variety of business needs.
   The prototypes developed also made it possible to explore safety considerations with key stakeholders when introducing increased levels of automation within ATC. There is potential for the approach to be used for a preliminary safety “assessment” where it could support early identification of potential key hazards thus informing future safety assurance of automated decision-making ATC tools.

From innovation to impact 

Through this project, we established the feasibility of using Wizard of Oz prototyping to explore human-machine interaction in the context of designing and developing novel automated decision-making concepts within ATC. The approach allowed us to constructively engage with diverse stakeholders much earlier on in the development process. In addition, the project demonstrated the possibility for using off-the-shelf, cost-effective materials and tools to bring complex futuristic concepts to life in record time with limited resources and budget. We believe the successful collaboration between NATS and the University of York was facilitated through the adoption of an agile approach, as well as the diverse and enthusiastic team who were committed to learning, experimenting, and sharing constructive feedback with each other.