Melons are the mystery-mongers among fruits: Under their thick, firm skin you may find a juicy and sweet flesh – or just rotten slush. Supermarket customers therefore knock on the skin before buying it to avoid unpleasant surprises. You’d think that producers and traders have more advanced quality control methods at hand, but no: The usual quality control for melons is also based on a manual knocking test, supplemented by occasional random samples.  

Professor Ahmad Osman has had his own experiences with rotten melons and believes that there has to be a better way to ensure their quality. In the ki-UltraHaltbarkeit project, his team of young scientists at the htw saar is working on a handheld device that can scan the fruit quickly and without contact.  

Osman is not actually a food specialist; his expertise is in measurement techniques and artificial intelligence. But this project is noticeably close to his heart, he views it as a contribution against food waste: According to 2019 estimates, 18 million tonnes of food are thrown away each year in Germany. Worldwide, the annual food waste amounts to 1.3 billion tonnes. Half of the food could be saved through better planning and storage. And a good third of all avoidable waste is fruit and vegetables; they are discarded almost ten times as often as meat or fish. “If we were able to reliably determine the ripeness of fruit and vegetables, a lot of waste could be avoided,” Osman believes.

But why melons? Any type of fruit would have been suitable as a test object for the fruit scanner, but the project team chose this particular use case for several reasons. On the one hand, the melon is a comparatively expensive product for which the buyer expects a corresponding quality, says project leader Osman. On the other hand, quality control is more demanding with melons than with tomatoes, for instance, as these show distinct outward signs of decay when rotting. Starting with the more complex problem, the team expects that the solution can later be easily transferred to less complex problems.

Intelligent evaluation and interpretation of measurements

Due to their expertise in measuring techniques, the researchers have an abundance of contactless, non-destructive methods at hand. In a preceding feasibility study, they tried everything from thermography to X-ray and spectroscopy to sound and ultrasound. For the melons, the team has decided on an acoustic method that works with frequencies in the audible range: The fruits are exposed to sound waves, and the reflected sound waves are recorded by a sensor.

However, Osman states that evaluating the measurements is much more important than choosing the best measuring technique anyway. “The artificial intelligence we use to analyse the sensor data accounts for about 80 percent of our solution,” he says. After all, a system that produces only raw data would not be very helpful in everyday use. The data have to be classified and interpreted to get useful information from them.

The output of the device will then be a simple and clear statement about the condition of the tested melon: unripe, ripe, overripe, spoiled. Moreover, the AI will even be able to consider the different preferences in different countries. For instance, a melon that is called ripe in Germany might be deemed just about edible in other regions of the world. Osman believes that “A good AI must also In the first year of the project, the team was mainly concerned with collecting data for the AI: The results of the acoustic measurements on melons were compared with conventional analyses such as determining water and sugar content in order to establish a correlation between the measured values and the actual condition of the fruit. “We have now completed this phase, and the database for training the AI is ready,” Osman reports.

Currently, the team focuses on building the first prototype. Since the hardware requirements for deep learning algorithms are pretty high, the initial design of the device will not be quite as compact as desired – optimising the AI is more important than optimising the device for now. However, the prototype will definitely be portable, says Osman, having roughly the size of a supermarket scale. Later on, it will be scaled down to a handheld device that the user only has to briefly hold close to the fruit to get a result.

Artificial intelligence: Between scepticism and unrealistic expectations

In addition, the team is currently testing their fruit scanner on other objects: It already works with apples and will soon be applied to various other fruits such as strawberries or mangoes. Meat and other complex food will, however, be left for a later stage of development, says Osman. They might require completely different measuring techniques because the molecular spoilage processes can be very different, he explains.

After making sure that the system works reliably in the lab, it will next be tested and optimised together with the project partners. Osman has often observed that potential cooperation partners have many doubts and fears about the use of artificial intelligence and that their reservations have only recently started to dwindle. “There is still a lot of scepticism towards AI in many companies, especially in the top-level management,” he reports.

On the other hand, the expectations are huge: One of the project partners has already expressed a wish for further functions of the scanner, such as a melon seed counter. “Unfortunately, we can't realize that function yet,” says Osman. However, he and his team have plenty of other ideas: From augmented reality to VR glasses, the researchers can imagine many useful enhancements for their fruit scanner.

For now, however, they want to work on more obvious issues: “We are hoping to link our system to the partners' existing merchandise systems,” says Osman. “If that works, retailers can quantitatively track which quality their different suppliers deliver over time.”