Spectral Weed Detection and Precise Spraying

1. Katholieke Universiteit LEUVEN Spectral Weed Detectionand Precise Spraying Laboratory of AgroMachinery and Processing Els Vrindts, Dimitrios Moshou, Jan ReumersHerman Ramon, Josse De Baerdemaeker Research sponsored by IWT and the Belgian Ministry of Small Trade and Agriculture

2. Overview • Spectral measurements of crops and weeds • in laboratory • in field • Processing of spectral data with neural networks • Precise spraying

3. Optical detection of weeds Techniques • red/NIR detectors (vegetation index) • image processing (color, texture, shape) • remote sensing of weed patches • reflection in visible & NIR light different detection possibilities, different scales Requirements for on-line weed detection: • fast & accurate weed detection • synchronized with treatment

4. Spectral weed detection Factors affecting spectral plant signals • leaf reflection, dependent on species and environment, stress, disease • canopy & measurement geometry • light conditions • detector sensitivity

5. Laboratory measurements integrating sphere sample computer spectrophotometer Spectral analysis of plant leavesin laboratory Diffuse Reflectance Spectroscopy of Crop and Weed Leaves

6. Laboratory measurements Diffuse Reflectance of a Leaf

7. Laboratory measurements Spectral Dataset

8. Laboratory measurements Reflectance of crop and weed leaves

9. Laboratory measurements Spectral analysis • stepwise selection of discriminant wavelengths • multivariate discriminant analysis, based on reflectance response at selected wavelengths (dataset a) • assuming multivariate normal distribution • quadratic discriminant rule classes with different covariance structure • testing the discriminant function: classification of spectra from dataset b

10. Laboratory measurements Spectral response of beet & weeds

11. Laboratory measurements Spectral response of maize & weeds

12. Laboratory measurements Spectral response of potato & weeds

13. Laboratory measurements Classification results

14. Field measurements Field measurement of crop and weeds Signal path Processingmethod Variation inlight condition Detector sensitivity Measurement geometry

15. Field measurements Equipment for field measurement spectrograph + 10-bit CCD, digital camera, computer, 12 V battery and transformer on mobile platform

16. Field measurements Equipment - Spectrograph both spatial and spectral information in images

17. Field measurements spectral axis spatial axis Image data • maize, sugarbeet, 11 weeds • 2 different days, different light conditions • 755 x 484 pixels

18. Field measurements Spectral response of sensor

19. Field measurements Data processing • spectral resolution: 0.71 nm /pixel • plant/soil discrimination with ratio: NIR (745 nm) / red (682 nm) • data reduction by calculating average per 2.1 nm, removing noisy ends • resulting spectra: 484.8 - 814.6 nm range, 2.1 nm step • independent datasets of maize, sugarbeet and weeds

20. Field measurements Spectral datasets

21. Field measurements Mean canopy reflections

22. Field measurements Canonical analysis of Sugarbeet - weeds

23. Field measurements Canonical analysis of Maize - weeds

24. Field measurements Discriminant analysis Sugarbeet

25. Field measurements Discriminant analysis Maize

26. Field measurements Graphic comparison datasets

27. Field measurements Graphic comparison datasets

28. Field measurements Graphic comparison datasets

29. Field measurements Discriminant analysis ratiosSugarbeet

30. Field measurements Discriminant analysis ratiosMaize

31. Field measurements Results • only spectral info (485-815 nm) • classification based on narrow bands in discriminant functions • good results in similar light and crop conditions • large decrease in performance for other light conditions • using ratios of narrow bands • improvement, but not sufficient

32. Field measurements Improving results • influence of light conditions • adaption of classification rule • determining light condition and applying appropriate calibration/LUT • spectral inputs that are less affected by environment • measuring irradiance, calculating reflectance • other classification methods

33. Crop-weed classification Neural network for classification • Comparison of different NN techniques for classification • Self-Organizing Map (SOM) neural network for classification • used in a supervised way for classification • neurons of the SOM are associated with local models • achieves fast convergence and good generalisation.

34. Crop-weed classification x x x x class Neural lattice (A) Input Layer 1 2 n -1 n Distribution weights Layer second hidden layer …. Pattern Layer first hidden Input Space (V) …. layer O Summation Layer O 1 f ( x ) f ( x ) n 1 2 s1(k) s2(k) s3(k) s4(k) Decision Layer Output Layer Neural network for classification SOM MLP PNN • ADVANTAGES • Learns with reduced • amounts of data • Fast Learning • Visualisation • Retrainable • DISADVANTAGES • Discrete output • ADVANTAGES • Good extrapolation • DISADVANTAGES • Slow Learning • Local minima • Needs a lot of data • ADVANTAGES • Fast Learning • Retrainable • DISADVANTAGES • Needs all training data • during operation • Needs a lot of data

35. Crop-weed classification Comparison between methods MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self-Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping Moshou et al., 1998, AgEng98, Oslo Moshou et al., 2001, Computers and Electronics in Agriculture 31 (1): 5-16

36. Crop-weed classification Comparison between methods MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self-Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping

37. Crop-weed classification Comparison between methods MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self-Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping

38. Crop-weed classification Conclusions on LLM SOM technique • The strongest point is the local representation of the data accompanied by a local updating algorithm • Local updating algorithms assure much faster convergence than global updating algorithms (e.g. backpropagation for MLPs) • Because of the topologically preserving character of the SOM, the proposed classification method can deal with missing or noisy data, outperforming “optimal” classifiers (PNN) • The proposed method has been tested and gave superior results compared to a variety statistical and neural classifiers

39. Precision treatment Precision spraying through controlled dose application Unwanted variations in dose caused by horizontal and vertical boom movements

40. Validation with ISO 5008 track movement of spray boom tip with and without controller Precision treatment 0.4 0.3 0.2 0.1 Distance (m) 0 -0.1 -0.2 -0.3 0 5 10 15 20 25 30 Time (s) Active horizontal stabilisation of spray boom

41. Precision treatment reduction fixing between plates electric motor g frame connected to tractor rol q boom cable ultrasonic sensors Vertical stabilisation of spray boom Slow-active system for slopes Resulting boom movement

42. Precision treatment On-line selective weed treatment Indoor test of on-line weed detection and treatment

43. Precision treatment Indoor test of on-line weed detection and treatment • Sensor: Spectral line camera • Classification: Probabilistic neural network • Program in Labview with c-code • Image acquisition frequence: 10 images/sec, travel speed: 30cm/sec, segmentation with NDVI ( > 0.3) • Off-line training of NN, On-line classification • Decision to spray: > 20 weed pixels and > 35% of vegetation is weed • Spray boom with PWM nozzles and controller, provided by Teejet Technologies

44. Precision treatment Indoor test of on-line weed detection and treatment Color image and spectral image

45. Precision treatment Indoor test - Results • Comparison of nozzle activation with weed positions

46. Precision treatment Indoor test - Results • separate weed classes (4) did not improve crop-weed classification • Correct detectionof nearly all weeds • Only 6 % redundant spraying of crop • Up to 70 % reduction of herbicide use Experimental set up camera nozzle weed