The looks of a winner

Delta, 10 Feb 2010

Just by looking at seedlings, experts can predict which plant will bear the most fruit. Now their knowledge has been spelled out and put into an expert vision system for plant selection.

Lifted on top of thin rotating poles, five young tomato plants just a few centimetres tall are pirouetting in front of a camera like tiny ballerinas in front of a jury. A second later, the plants are briskly embraced by robot arms that carry the seedlings into the adjacent room, where they are placed in one of three rows: slow, medium or fast growers. Without delay the robot arms return to the rotating poles to fetch the next batch. It’s the Select-O-Mat at work, the state-of-the-art technology developed by GreenVision at Wageningen University and Research Centre to automatically select and sort tomato seedlings at speeds of up to 15,000 plants per hour. Commercial gardeners want to know the quality of the plants they buy. “Since a good tomato plant gives up to sixty kilograms of fruit, gardeners don’t want to waste their space and efforts on a plant that produces less”, explains dr. Nicole Koenderink, from the Wageningen UR Food & Biobased Research Institute. That’s why traditionally experts always selected the seedlings. Now this can be done by the Select-O-Mat, which works better and faster than the experts, but it’s still not good enough. For one thing, the robot only inspects the leaf surface (big leaves means fast grower), which doesn’t do justice to the expert’s more subtle judgements. So Koenderink has developed a new computer vision system that makes better use of the expert’s knowledge. Last Monday at TU Delft she defended her thesis, entitled ‘A knowledge-intensive approach to computer vision systems’, where Professor Lucas van Vliet from the research group quantitative imaging (faculty of Applied Sciences) served as her PhD supervisor, together with information management specialist professor Jan Top, of VU University in Amsterdam.
“Most computer vision systems are based on black-box models”, says Koenderink. These systems are trained by showing them good and bad specimens, while simultaneously feeding them with the rating. Over the course of time, the neural network behind the system will somehow correlate the visual impression with the rating until it can mimic the judging process. The problem is that no one can really trace what the system is doing in laymen’s terms. “I’ve developed a white-box system instead”, Koenderink says. “A system that embodies the expert knowledge and is transparent in its workings.”
Koenderink worked with the plant-rating experts to find out exactly what they where looking at, and how they judged the features of plug, stem and leaves to arrive at a quality rating for the plant. “It’s a sequence of see, think, act”, she explains.
The system starts with a three-dimensional ‘point cloud’ representing the plant. Next, it should be able to recognise the various parts: thick cylinder is the stonewool plug; thin cylinder is a stem; and curved surface is leaf. A ‘sanity check’ is applied to prevent impossible interpretations. If for example the system ‘sees’ a stem within a leaf, the sanity check decides that it is simply a part of the leaf that somehow must have been folded. Once the various parts of the seedling have been identified, judgement time has come. So, for example, does the candidate plant have less than four leaves; are there dead bits on the leaves; are the leaves misshapen; is the area of the leaves less than 25 percent of the average area or is more than half of the leaf area damaged? If yes, the plant will be rejected as ‘abnormal’. Otherwise, it will be rated as a first, second or third choice plant.
Initially the system made errors in shape recognition, but with software improvements the erroneous interpretations have been reduced. In the existing prototype version it took seven minutes to create the 3-D image and another ten minutes to analyse it. Nonetheless Koenderink and colleagues have cut down the image acquisition time per plant to 0.3 seconds by using an automated imaging system and by streamlining the software. She is confident that the analysis process will be much faster as well in the market-ready machine. She thinks the white-box system is especially suited for judging natural products like flowers, eggs or apples, because of the underlying natural complexity. Koenderink: “Man-made things are quite simple: if they’re deviant, they’re no good.”

Nicole Koenderink Video’s on www.greenvision.wur.nl

Weblink to Delta