The University’s Audiolab, based in the School of Physics, Engineering and Technology, is used to seeing its research make an impact. Now, an established relationship with Google has seen its work in three-dimensional (3D) audio adopted within Google apps and services such as YouTube 360, underpinning virtual reality (VR) sound across the internet.
Lead researcher Dr Gavin Kearney explained: “Our research focuses on spatial audio - the creation of immersive 3D sound fields that give you the impression of hearing sound all around you, either over headphones or using loudspeakers.
“This immersive sound can be used in cinema, VR, games or any application in which sound is a factor. We’re creating a plausible sound field, where the listener really feels that the virtual acoustic environment is a real one.”
Headsets
Although his team’s recent work was specifically aimed at VR headsets, Dr Kearney says that wasn’t their initial goal; it was earlier work on improving soundbar technology that caught the attention of the team at Google.
“Typically, soundbars use multiple loudspeakers with the aim of create a pseudo 3D audio experience through some clever signal processing and some psychoacoustic techniques,” he explains.
“My initial research - the EPSRC-funded ‘SADIE’ project - was intended to improve these soundbar systems, which we mainly achieved through using the soundbars in combination with motion tracking technology.”
As a precursor to this work on soundbars, part of the SADIE project considered how 3D audio could optimally be delivered using headphones, a process which requires tricking the brain into thinking the signals it receives at the ears are the same as those experienced without headphones, giving the perception of 3D sound; a feature known as ‘binaural’ sound.
“The difficulty is everyone’s ears are as unique as their fingerprints,” says Gavin. “So when someone is listening to a binaural recording, instead of hearing the voice as being in front of them, the brain might get confused and interpret the sound as coming from behind.”
The team knew that motion tracking of the head could help with this problem, because it can adapt the signals going to the headphones so that it remains stable in 3D space. And to find out what level of binaural sound quality they could achieve, they combined their knowledge of how sound impacts on the ears from different directions, with their ability to adapt sound fields to accommodate for head movements.
Database
To facilitate this work, the team built a database of binaural acoustic measurements by placing subjects in a sound absorbing anechoic chamber, with microphones in their ears and facing a vertical array of loudspeakers.
Gavin describes the measurement process; “The subjects were seated on a motorised chair - actually an adapted antenna dish rotator - which could be accurately controlled. We’d play test sounds from the arc of loudspeakers, take the measurements, rotate the chair to another desired angle, take a new set of measurements and so on. This way we were able to measure a full sphere of points around the head”. This results in highly accurate set of measurements - called binaural filters - that show how sound behaves at all angles to the ears. These measurements can then be used to synthesize 3-D audio soundfields with a high degree of accuracy.
“The data showed us how sound changes in terms of time and level differences at the ears and how the physical features of the human ear affect tonality of signals reaching the brain - this is a crucial cue in enabling the brain to interpret the front/back position and height of sound sources.”
While this method of measuring sound’s interaction with the human ear is not new, Dr Kearney’s team carried out the measurements and created 3-D audio decoders that were optimised for the sound processes specific to VR; and their work was soon noticed.
Gavin said: “We published the data and it was Google who got in touch. They liked the measurements and had used them in comparison against similar measurements they’d undertaken themselves. After rigorous tests they’d found that the perceived spatial audio quality was much higher with our work.”
Impact
Google adopted the data and its open source policy saw the research impact spread even further.
“Because it’s Google and they put things up as open source - as did we - it means that the entire industry follows suit. So now, if you’re producing content for YouTube 360 you need to be using our binaural filters to ensure you have the right audio quality.”
Gavin’s team are already working on the next challenges, and he believes VR is highly likely to become social media’s next frontier.
“The applications of VR go beyond mere entertainment value, into training, therapy, telecoms and even social media. Facebook for example bought Oculus because they see this as an important space where people will use VR headsets to interact with each other,” he says.
And he adds the team is also looking at augmented reality - the concept of overlaying virtual reality sounds and visuals onto the real world - and considering what possibilities 5G smartphones will bring to immersive sound.
“There’s several ways this could go but the key is the technological capability delivered to the end user,” he says. “We want everybody to have engaging immersive 3D audio experiences, but there are significant challenges in the capture, production and rendering of audio for VR and AR. There’s always something to keep us busy, and of course, it’s great fun.”
The text of this article is licensed under a Creative Commons Licence. You're free to republish it, as long as you link back to this page and credit us.
