Spectral Weed Detection and Precise Spraying

Katholieke Universiteit LEUVEN Spectral Weed Detectionand Precise Spraying Laboratory of AgroMachinery and Processing Els Vrindts, Dimitrios Moshou, Jan ReumersHerman Ramon, Josse De Baerdemaeker Research sponsored by IWT and the Belgian Ministry of Small Trade and Agriculture

Overview • Spectral measurements of crops and weeds • in laboratory • in field • Processing of spectral data with neural networks • Precise spraying

Optical detection of weeds Techniques • red/NIR detectors (vegetation index) • image processing (color, texture, shape) • remote sensing of weed patches • reflection in visible & NIR light different detection possibilities, different scales Requirements for on-line weed detection: • fast & accurate weed detection • synchronized with treatment

Spectral weed detection Factors affecting spectral plant signals • leaf reflection, dependent on species and environment, stress, disease • canopy & measurement geometry • light conditions • detector sensitivity

Laboratory measurements integrating sphere sample computer spectrophotometer Spectral analysis of plant leavesin laboratory Diffuse Reflectance Spectroscopy of Crop and Weed Leaves

Laboratory measurements Diffuse Reflectance of a Leaf

Laboratory measurements Spectral Dataset

Laboratory measurements Reflectance of crop and weed leaves

Laboratory measurements Spectral analysis • stepwise selection of discriminant wavelengths • multivariate discriminant analysis, based on reflectance response at selected wavelengths (dataset a) • assuming multivariate normal distribution • quadratic discriminant rule classes with different covariance structure • testing the discriminant function: classification of spectra from dataset b

Laboratory measurements Spectral response of beet & weeds

Laboratory measurements Spectral response of maize & weeds

Laboratory measurements Spectral response of potato & weeds

Laboratory measurements Classification results

Field measurements Field measurement of crop and weeds Signal path Processingmethod Variation inlight condition Detector sensitivity Measurement geometry

Field measurements Equipment for field measurement spectrograph + 10-bit CCD, digital camera, computer, 12 V battery and transformer on mobile platform

Field measurements Equipment - Spectrograph both spatial and spectral information in images

Field measurements spectral axis spatial axis Image data • maize, sugarbeet, 11 weeds • 2 different days, different light conditions • 755 x 484 pixels

Field measurements Spectral response of sensor

Field measurements Data processing • spectral resolution: 0.71 nm /pixel • plant/soil discrimination with ratio: NIR (745 nm) / red (682 nm) • data reduction by calculating average per 2.1 nm, removing noisy ends • resulting spectra: 484.8 - 814.6 nm range, 2.1 nm step • independent datasets of maize, sugarbeet and weeds

Field measurements Spectral datasets

Field measurements Mean canopy reflections

Field measurements Canonical analysis of Sugarbeet - weeds

Field measurements Canonical analysis of Maize - weeds

Field measurements Discriminant analysis Sugarbeet

Field measurements Discriminant analysis Maize

Field measurements Graphic comparison datasets

Field measurements Discriminant analysis ratiosSugarbeet

Field measurements Discriminant analysis ratiosMaize

Field measurements Results • only spectral info (485-815 nm) • classification based on narrow bands in discriminant functions • good results in similar light and crop conditions • large decrease in performance for other light conditions • using ratios of narrow bands • improvement, but not sufficient

Field measurements Improving results • influence of light conditions • adaption of classification rule • determining light condition and applying appropriate calibration/LUT • spectral inputs that are less affected by environment • measuring irradiance, calculating reflectance • other classification methods

Crop-weed classification Neural network for classification • Comparison of different NN techniques for classification • Self-Organizing Map (SOM) neural network for classification • used in a supervised way for classification • neurons of the SOM are associated with local models • achieves fast convergence and good generalisation.

Crop-weed classification x x x x class Neural lattice (A) Input Layer 1 2 n -1 n Distribution weights Layer second hidden layer …. Pattern Layer first hidden Input Space (V) …. layer O Summation Layer O 1 f ( x ) f ( x ) n 1 2 s1(k) s2(k) s3(k) s4(k) Decision Layer Output Layer Neural network for classification SOM MLP PNN • ADVANTAGES • Learns with reduced • amounts of data • Fast Learning • Visualisation • Retrainable • DISADVANTAGES • Discrete output • ADVANTAGES • Good extrapolation • DISADVANTAGES • Slow Learning • Local minima • Needs a lot of data • ADVANTAGES • Fast Learning • Retrainable • DISADVANTAGES • Needs all training data • during operation • Needs a lot of data

Crop-weed classification Comparison between methods MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self-Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping Moshou et al., 1998, AgEng98, Oslo Moshou et al., 2001, Computers and Electronics in Agriculture 31 (1): 5-16

Crop-weed classification Comparison between methods MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self-Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping

Crop-weed classification Conclusions on LLM SOM technique • The strongest point is the local representation of the data accompanied by a local updating algorithm • Local updating algorithms assure much faster convergence than global updating algorithms (e.g. backpropagation for MLPs) • Because of the topologically preserving character of the SOM, the proposed classification method can deal with missing or noisy data, outperforming “optimal” classifiers (PNN) • The proposed method has been tested and gave superior results compared to a variety statistical and neural classifiers

Precision treatment Precision spraying through controlled dose application Unwanted variations in dose caused by horizontal and vertical boom movements

Validation with ISO 5008 track movement of spray boom tip with and without controller Precision treatment 0.4 0.3 0.2 0.1 Distance (m) 0 -0.1 -0.2 -0.3 0 5 10 15 20 25 30 Time (s) Active horizontal stabilisation of spray boom

Precision treatment reduction fixing between plates electric motor g frame connected to tractor rol q boom cable ultrasonic sensors Vertical stabilisation of spray boom Slow-active system for slopes Resulting boom movement

Precision treatment On-line selective weed treatment Indoor test of on-line weed detection and treatment

Precision treatment Indoor test of on-line weed detection and treatment • Sensor: Spectral line camera • Classification: Probabilistic neural network • Program in Labview with c-code • Image acquisition frequence: 10 images/sec, travel speed: 30cm/sec, segmentation with NDVI ( > 0.3) • Off-line training of NN, On-line classification • Decision to spray: > 20 weed pixels and > 35% of vegetation is weed • Spray boom with PWM nozzles and controller, provided by Teejet Technologies

Precision treatment Indoor test of on-line weed detection and treatment Color image and spectral image

Precision treatment Indoor test - Results • Comparison of nozzle activation with weed positions

Precision treatment Indoor test - Results • separate weed classes (4) did not improve crop-weed classification • Correct detectionof nearly all weeds • Only 6 % redundant spraying of crop • Up to 70 % reduction of herbicide use Experimental set up camera nozzle weed

Spectral Weed Detection and Precise Spraying