Modern medicine produces large amounts of images. Patients are x-rayed and scanned, tissue samples are prepared and stained, but then all these images have to be assessed by trained medical staff - “by hand”, so to speak. In other areas of life, AI-controlled image recognition has long been used in such cases. Why is development in the medical field so slow when the use of this technology is otherwise exploding?

Prof. Dr. Christian Herta, data scientist at the HTW Berlin and head of the deepHEALTH project, can name a whole range of reasons for this. For one thing, there are still practical problems to be solved in medicine, for example with images of histological tissue sections: they have about eighty times the resolution of ordinary photos and are therefore too large to be processed in one piece by neural networks. “Histology images can only be analysed in sections. The algorithm can lose crucial information for the overall diagnosis because it lacks the correlations,” says Herta.

Another problem is the availability of data - because AI needs large amounts of it to learn. In addition, the data must first be annotated laboriously, i.e. annotations must be added to the image content. In cancer diagnostics, for example, this means that doctors have to mark tumour areas on tissue sections (see article image) before the images can be used to train an algorithm. And while millions and millions of ordinary photos of houses, cars or animals are stored in freely accessible databases, medical images are strictly protected for good reason. “Data protection can be a real challenge for AI development, because there is simply less training data in well-protected areas,” says Institute staff member Dr. Christian Krumnow.

Images, protein sequences, sleep data - a case for AI

Nevertheless, the Berlin scientists want to try to give medicine a little more artificial intelligence - considering, of course, the special conditions that the field brings with it. In the deepHEALTH project, image recognition plays an important role because there is a lot of potential for its application: the most common diseases, including cancer, naturally produce the largest amounts of images and thus most of the work for pathologists, oncologists and radiologists.

But AI has other advantages as well. For instance, appropriately trained neural networks are also suitable for speech recognition. And speech, in turn, has clear parallels to biochemistry, where sequences play an important role: “The sequence of bases on a DNA strand or of amino acids in a protein is structurally very similar to the sequence of letters, words and sentences,” explains Herta. That is why the team is also developing algorithms for the recognition and correct assignment of biomolecule sequences, so that the machines can, for example, identify the type of pathogen based only on genome fragments in a sample. The project team has also been working on algorithms for sleep medicine, where complex data is generated, and pattern recognition is required.

Algorithms should support the doctor, not take over his job

The Berlin researchers are also working on a special problem that is not quite as relevant in many fields of application of AI, but very relevant in medicine: algorithms are a kind of black box. “Even the programmers of an algorithm are often unable to say how it came to its decision in detail,” explains Christian Krumnow. Usually, the result is more important: You feed the program with input (“Here you have a stack of pictures”) and get the required output (“Find me all pictures with cats”). However, the concrete way that led to this result is often completely unclear.

When it comes to the automatic identification of cat images, this black box can perhaps be tolerated. But in medicine, correct diagnoses are often a matter of life or death - of course, both doctor and patient will want to know exactly why AI claims to have discovered a tumour in a tissue sample. After all, even algorithms do not work flawlessly. On the contrary: sometimes they are far too sure of their facts, which is called “over-confidence”. Krumnow cites a vivid example of the effect: “If you train a standard algorithm to distinguish dog images from cat images, but then present it with images of cars, it will not recognise the cars, but classify them as dogs or cats. Moreover, it will probably signal that it is very confident in its decision”. A suitable algorithm for medical applications, however, must reliably recognise when an object is different from what it has been trained to find, and must then report that it cannot provide a classification at that point. In such cases, the doctor can intervene and form his own opinion.

Avoiding overconfidence and illuminating the decision-making processes in the “black box”: This is one of the most important project goals of deepHEALTH, because there is still a lot of basic research to be done in this area. In addition, Christian Herta and his colleagues also want to reduce reservations among future users: According to Herta, doctors often react sceptically to programmes that apparently relieve them of work. But the aim is to create a system that supports the work of the doctors instead of taking it off their hands. “The system must therefore not work in a non-transparent way and must not rob the doctor of the motivation to make his own decisions”, he says.

In the midst of a rapid development

The fact that their project is fully in line with the latest trends does not only have advantages for Herta and his colleagues. One of the sub-projects was overtaken by current developments even during the application phase: a US research team published results on one of the research questions, answering it before the deepHEALTH project could even start. “However, we were able to adjust the project plan accordingly,” says Herta. And in the meantime they are seeing success: The first doctoral candidate in the project has already completed his work, and his successor is now concentrating on the interpretability of the algorithms and the question of whether and how an explanatory function can be programmed into the system. Meanwhile, Christian Herta is already formulating visions for the future: At some point, he would like to have a programme with voice output that tells the doctor directly what it has discovered and why it interprets the data as it does.

All in all, with deepHEALTH, the HTW Berlin is very close to one of the most exciting topics of the future. Even though some AI systems for medicine are already being tested at university hospitals, most of them have not yet been commercialised. Sooner or later, however, artificial intelligence will also become the norm in medicine - to support its users and perhaps even to find things here and there that even the trained eye of a doctor has overlooked. If the algorithms can really help doctors and patients, then Christian Herta and his colleagues are satisfied - for the time being.