A tiny handful of life, connected to tubes and cables: babies born far too early and weighing less than 1500 grams spend the first weeks of their lives in a neonatal intensive care unit in an incubator, under constant medical supervision. Their organs are not yet fully developed, so they often have problems with breathing, digestion or metabolism. And they are very susceptible to infections because their immune system is not yet functioning well.

Johanna Gleichauf, a doctoral student at Nuremberg Tech, has set out to alleviate the start in life for such premature babies and to replace some of the medical monitoring systems with contactless versions. Although the technology can certainly save lives, it also causes problems: Tubes can cause pressure marks, adhesive ECG electrodes can damage the sensitive baby skin, and cleaning, changing diapers and dressing the wired babies is complicated and time-consuming.

In order not to go beyond the scope of the project and her doctoral thesis, the young scientist is initially concentrating on three parameters to be determined without contact: heart rate, respiration and body temperature. This is complex enough for a four-year research project, says Gleichauf. In order to achieve the goal, the project team - which, in addition to Gleichauf and Niebler, includes one master and several bachelor students - combines different camera systems and a radar with low transmission power. Sensor fusion is the key for success, explains Gleichauf: “The data from the various systems must be combined to form reliable and accurate information.” The young scientist already has experience with this concept, but on a completely different topic: The medical technology student took a detour to work on her master's thesis in a robotics project with autonomous shunting locomotives (project VAL2020 in cooperation with DB Cargo). However, the principle is the same, she says: Neonatology is just a different application.

Tailor-made algorithms and calculation models are the core of the project

The NeoWatch approach is not focusing on sensor development: The team uses commercially available camera and radar equipment, provided by the company InnoSenT, which is involved in the project as a cooperation partner. Instead, the core of the work are custom-made algorithms and calculation models that combine the camera images and the radar data to form useful information and reliably derive the vital parameters. For example, the system must later be able to infer the baby's body temperature from the surface temperature - despite clothing, movement and other disturbing influences.

Pre-development of the camera algorithms has been largely completed and they can soon be tested in a clinical environment. Among other things, the project team collects a large number of images during these tests; neural networks are used to train the system on the basis of the images to correctly recognize and calculate the required parameters. A total of three camera systems will be used: a thermal imaging camera for body temperature, an RGB colour camera and a so-called Structured Light Camera, which provides a three-dimensional point cloud. “This camera is similar to the Kinect cameras for the X-Box and is very well suited to register breathing movements,” says Gleichauf.

The radar is essential for project success

After focusing on the cameras in the first two years of the project, algorithm development for the radar has only just begun. It is technically demanding, but essential for the success of the project. Johanna Gleichauf explains why: “The radar device can detect distance changes in the micrometer range - we need this resolution to be able to reliably detect heartbeats. And unlike the cameras, the radar is completely independent of lighting conditions”. This is particularly important in the neonatology intensive care unit, as there is little light because premature children are sensitive to all stimuli.

Of course, the radar will also have to be clinically tested later, step by step: first on adults and baby dummies, then on normal babies who are in the clinic's paediatric ward because of jaundice, for example. The system must work completely reliably before it can be used for premature babies – and even then, it will initially be installed in addition to the classic monitoring methods on which doctors and nurses can continue to rely. The demonstrator is not yet an approved medical device. “And premature babies are probably the most sensitive patients there are,” says Gleichauf. “In addition, their parents are in an emotionally difficult situation. That's why our development touches a particularly sensitive area, so we have to be especially careful here.”

This summer, the project is halfway through its four-year funding period, and Johanna Gleichauf is very pleased with the progress made so far: NeoWatch is right on schedule, she says. The Corona crisis has even had a positive effect on her work: “Six months ago, we were somewhat behind our schedule. But we have been able to catch up in the meantime, because we had a lot of time and peace and quiet to develop the algorithms and calculation models, and to push ahead with bureaucratically complicated procedures such as the submission of the ethics proposal”.

There is still plenty of room for further development after the end of the project

Of course, the team already has an exact idea of what the demonstrator should look like: Cameras and radar are mounted in a sensor box, which is attached to the inside of the incubator with magnets. But this is only the first step towards a comprehensive contactless monitoring system. Johanna Gleichauf is already thinking a bit further into the future: “For one thing, it would of course make sense if the system were not tied to the incubator at some point, but if monitoring the children were also possible outside the incubator. For example, when they are lying on their parents' chest,” she says. This so-called “kangarooing” is important for the development of the babies and the bond between parents and child.

The young scientist is also thinking about replacing other monitoring systems - first and foremost, of course, a contactless ECG, because the adhesive electrodes of the classic ECG can cause severe skin damage. In addition, a contactless weight determination was originally planned for NeoWatch, which might also be realized via the camera radar system. However, according to Gleichauf, modern incubators with integrated pressure sensors for weight determination are now available. “That is why we have put this aspect on hold for the time being. But theoretically it could be integrated into our system with comparatively little effort.” In any case, there is still enough room for further developments after the end of the project to make it easier for premature infants to start their lives.

The NeoWatch project is going on until summer 2022 - but Johanna Gleichauf will probably continue to work on this topic afterwards. In any case, she does not want to return to the autonomous shunting locomotives. The master's project was instructive, she says, “but I'm more interested in medical topics.”