“Be quiet, please, this is important!” Teachers often are compelled to shout sentences like this at a noisy classroom, when, for instance, three children in the front row are talking about an upcoming test while books and exercise booklets are being closed in the back – the lesson will soon be over anyway – and other classes outside are already romping across the playground.

With so much commotion around them, most pupils can hardly understand their teacher. Children with limited spatial hearing, however, have a particularly hard time: They have problems reducing or blocking out background noise and are therefore always exposed to the entire sound level of the room.

People who do not have a problem with spatial hearing are constantly sorting and filtering out ambient noise. We know this as the “cocktail party effect”: In loud surroundings, a healthy hearing sense unconsciously amplifies one voice and suppresses others. Thus, we are able to have a conversation at a party despite the noise. We usually attribute hearing problems in loud surroundings to older people, but children surprisingly often have a problem with this filtering, too.

Half of the children with learning disabilities have hearing problems

“This is called a defect in spatial auditory perception. Almost every tenth child suffers from it,” says Karsten Plotz. As a paediatric audiologist, he deals with children's hearing problems – and has done so for more than 25 years. He runs a practice in Oldenburg that specialises in this topic and where he conducts hearing screenings for new-borns or treats hearing disorders in children.

In addition, the 60-year-old teaches and researches as a professor of otolaryngology and paediatric audiology at the Jade University of Applied Sciences. Therefore, he knows the problems that can arise from impaired hearing: “About half of all children with learning difficulties have hearing problems, so there's a strong connection there,” he says. Solving their hearing problems would make everyday school life much easier for these children.

VIWER-S pursues two innovative approaches

That is why Plotz started the VIWER-S project in 2019. Together with Professors Jörg Bitzer (Jade University of Applied Sciences) and Christoph Pörschmann (TH Köln), he wants to help children who suffer from hearing problems summarised under the term “Spatial Processing Disorder”. The VIWER-S team is pursuing two approaches: A technical hearing aid for the classroom and a virtual reality-based (VR) system with which children can improve their spatial perception skills.

Jörg Bitzer is responsible for the first part of the project. Like Plotz, he works at the Jade University of Applied Sciences in Oldenburg where he teaches audio signal processing in the “Hearing technology and audiology” course. For VIWER-S, he is developing a modern hearing aid for everyday school use together with students and doctoral candidates. “The system helps hearing-impaired children to correctly locate sounds,” says the 52-year-old researcher.

There are hearing aids for this purpose already on the market, but they have limitations: They usually consist of a microphone that the teacher wears around the neck, and hearing aids or headphones for the child that are connected to the microphone. The problem is that the teacher’s voice forms as a mono signal “right in the middle of the child's head”, as Bitzer puts it.  Every user of customary headphones knows the effect: The sound from the headphones seems to be formed between the ears. The signal conveys neither information about the speakers’ location in the room nor other influences such as reverberation or background noise. Thus, the child’s hearing, which is already underdeveloped with regard to spatial information, continues to be fed false acoustic information.

New system enables “aurally compensated” acoustic signal for the first time

“In contrast to the single-microphone approach, our system has 15 small microphones like those built into smartphones. They pick up all the signals in the room and forward them to a mini-computer,” says Bitzer. The information is then processed by a software that determines the positions of the sound sources in the room and amplifies those that are important. “After all, it is not only the teachers who convey important information,” explains Bitzer, “it can also be classmates answering a question.” The system transmits their voices to special headphones that deliver “aurally compensated” signals to the children's ears: The sounds seem to be actually coming from the direction in which their sources are located.

The challenge is to do all this in real time, Bitzer explains: “The system is complex. We have to factor in the child's head movement and direction of gaze, locate the other children as well as the teacher, who might even be walking around, put everything in relation to each other, isolate important signals, improve them, and transmit them correctly. And all of this must happen so fast that there is no delay in lip sync.” Otherwise, the sound of the voices would be altered or the child would hear everything as an echo. “And you don't want that at all,” says Bitzer.

Currently, the microphone array resembles a small boomerang that measures about 45 centimetres. The microphones are arranged on the wings. “We designed the prototype ourselves and made it with 3D printing,” says Bitzer. In addition to the Boomerang, the system uses another tracking microphone worn by the teacher. Thus, the system always knows when the teacher is speaking and where he or she is located in the room.

Practising spatial hearing in a cartoon world

 In the second part of the VIWER-S project, Christoph Pörschmann does not need to design and produce any hardware. He uses an established consumer product for his work package: the Oculus Quest. This virtual reality (VR) headset, developed by the Facebook parent company Meta, needs neither console nor PC, as all the necessary hardware is already built in.

Pörschmann and his team are developing scenarios for the Oculus Quest that children can use to train their hearing or that enable the diagnosis of spatial hearing impairment. “We have created a virtual world for children in which we conduct standard audiometric tests,” explains Pörschmann.

The test is called “speech in noise test”: The children put on the VR glasses and suddenly find themselves in a large meadow in the middle of a colourful cartoon world. Trees grow on a nearby riverbank, fluffy clouds float against the blue sky. The children are joined by small friendly robot that calls out sentences to them such as: “The boy throws the ball.” At the same time, the headphones put a noise on the ears. With each additional sentence, the volume changes. Moreover, the direction from which the noise comes is varied. This is no problem for the VR glasses, as they can reproduce any sound three-dimensionally. “Thus, we can playfully determine the limit of a child’s ability to understand the words and filter out the noise,” says Pörschmann. “As we know the default values in healthy hearing people, we can tell after a few minutes whether there is a problem with spatial hearing.”

Training scenarios must be child-friendly and varying

The virtual worlds not only enable Pörschmann and his team to make a diagnosis. If the diagnosis is positive, affected children can also improve their spatial hearing. “Damage to spatial hearing can be reduced with our training,” says the professor who, among other things, teaches auditory virtual environments at the TH Köln. The plan is that children with spatial hearing problems can take the VR glasses home and use the training scenarios to practise ten minutes each day for three months. “We are currently working with game designers to determine what features should be implemented to keep the children motivated for a long time,” says the 52-year-old.

The VIWER-S project was originally planned for four years and was to be completed at the end of 2022, but the Corona virus thwarted the ambitious schedule. Due to the pandemic, neither the virtual glasses nor the microphone boomerang could be tested with hearing-impaired children – no ethics committee would have allowed that. Therefore, the project coordinators have applied for an extension until 2024.

“The hurdles for studies with young children are extremely high. Which is totally justified, of course, as our test persons will only be between six and ten years old,” says paediatric audiologist Plotz. However, he does not believe that the delay puts the whole project at risk: “It will be demanding, for sure, but it is simply our task for the next two years to bring the project to a successful end.”