Progress report for GNE20-234
The major aim of this research is to develop an automated precision spraying system for pest management in the orchards considering the different tree canopy structures and orchard terrain conditions of Pennsylvania. The research project will be focused on the following objectives:
- Development of LiDAR-guided sensing technology for assessing tree canopy density of different tree architectures
A LiDAR-guided sensing system will be developed to accurately measure the canopy density of an individual tree. An individual tree will be divided into six sub-sections due to having six nozzles on each side of the sprayer unit. Then the canopy density of each section will be calculated. An Inertial Measurement System (IMU) will be used to record the posture of the LiDAR system to perform necessary adjustments of the canopy information as well as tree orientation. A Global Navigation Satellite System (GNSS) system will be used to track the geo-reference of the trees with canopy density and terrain information. The combination of the tree canopy and terrain information will be analyzed to provide guidance for the control of the precision sprayer. Real-time tests will be conducted to evaluate the system performance before integrating it into the spraying unit.
- Automation of nozzles and air-blast sprayer fan for target spraying
An automatic system will be developed aiming to conduct target spraying using adjusted tree canopy density information and real-time weather data. Both the nozzle and air-blast fan speed will be controlled automatically during spraying. The nozzles will be controlled by using the tree canopy information and the air-blast sprayer fan speed will be controlled by using both tree canopy density, and real-time weather information (e.g., temperature, humidity, wind speed, and direction, etc.). Fan speed control is important because it is responsible for pulls air in and redistributes it upwards into the tree canopy for reaching the spray droplet on target.
- Performance evaluation of the developed precision spraying system for the efficacy of field control of disease and pest
The developed precision spraying system will be tested in the field to evaluate its efficacy for the apple scab and codling moth control for the apple trees. The robustness of pest control by this newly developed spraying system will be compared with conventional methods, as well as the working efficiency and overall chemical usage.
The purpose of this project is to maintain the sustainability of the tree fruit industry by reducing the excessive use of pesticides in the tree-fruit orchards. This proposed study is focused on adjusting the spray volume based on canopy density of the trees aims to develop an automated precision spraying system for apple orchards. Upon completion of this project, a significant amount of pesticides could be saved which leads to reduce risk of environmental pollutions and public health as well as lower cost of production in the orchards.
Conventional sprayers waste a large number of agrochemicals between trees and into open territories above short trees or around trunks underneath canopies. As a result, a significant amount of pesticide is misused which increases production cost. On the other hand, pesticide misuse not only wastes the farmer's money but also represents a serious public concern about contamination of the environment in general and the impact of agriculture on food products specifically. In response to the critical needs of reducing chemical usages and enhance the apple industry's profitability, precise control of pesticide application is needed. Therefore, it is very important to develop a variable-rate spraying system to apply agrochemicals as needed to the target trees for reducing off-target deposition of the spray droplets.
Objective #1: Development of LiDAR-Guided Sensing Technology for Assessing Tree Canopy Density of Different Tree Architectures
Objective #1a: Measurement of Tree Canopy Density for Different Tree Architectures
- Sensor System Integration and Data Acquisition
Tress canopy data acquisition is the first step for the canopy density measurement of the trees. A point cloud data acquisition system was developed by integrating a VLP-16 LiDAR scanner, an interface box for data transmission and power conversion, and a laptop computer. Trees in a row were scanned from both sides to cover the entire canopy in each orchard (Fig. 1).
Figure 1. Experimental setup of the LiDAR-guided system (a) utility vehicle run at the center of the row (orchard site 2) (b) a consecutive five trees were scanned
2. Tree Canopy Determination
In the test orchards, there are trellis wires and poles in the tree rows. To calculate canopy foliage density, the support poles and trellis wires needed to be segmented and removed. The tree trunk is another parameter responsible for increasing the number of points and was also removed. To detect the tree canopy points, algorithms were developed to segment the tree trunk, trellis wires and support poles from the acquired point cloud data (Fig. 2).
Figure 2. Tree trunk, trellis wire and support pole detection (TP: trunk points; PP: support pole points; TWP: trellis wire points; CP: canopy points)
3. Canopy Foliage Point and Leaf Counting
To identify canopy density, the canopy foliage points were counted from four sub-sections (Fig. 3). Determining the edge of the tree canopy is crucial to sub-divide the individual tree sections. The edge of both sides was detected. To calculate the 3D-based canopy density, the 3D grid was divided into four sub-sections according to the locations of three trellis wires.
Figure 3. Labelled canopy points in four sections of an apple tree (section 1 (bottom) to section 4 (top))
4. Canopy Density Measurement and Map
Upon removing all unnecessary points (e.g., tree trunk, trellis wires, and support pole points), the remaining points were used to calculate the canopy density of the trees. Each point cloud image from individual trees was divided into small grid areas with equal sizes of 50 cm2. The number of canopy points from each grid area was counted to generate canopy density maps.
Objective #1b: Correction of LiDAR-Sensed Tree Canopy Points in Uneven Terrain Conditions
- Concept of Canopy Corrections
In the rough terrain condition, the sensing and spraying positions are being changed (Fig. 4). Inaccurate sensed canopy data calculates erroneous tree canopy density, which directly contributes to the off-target deposition of the spray droplets during spraying operations. To correct the sensed canopy position, a simulation was performed according to Euler's theorem. The simulation was performed to derive the correction equations for position change in x, y, and z-axis directions aimed to correct the tree canopy position. The simulations were performed using MATLAB® software. Different input parameters, including angle changes of the roll, pitch, and yaw, were used to evaluate the performance of the developed model.
Figure 4. Sensing and spraying position variations in slope change
- Sensor System Integration
An integrated sensor system was developed, comprised of a VLP-16 LiDAR scanner (Velodyne LiDAR, San Jose, CA, USA), an Inertial Navigation System-Global Navigation Satellite System (INS-GNSS) (Inertial Labs, Paeonian Springs, VA, USA), and two laptop computers (Fig. 5).
Figure 5. Integrated hardware system for canopy position corrections
- Sensor Fusion Algorithm for Canopy Point Corrections
A sensor fusion algorithm was developed to perform the canopy position correction. The algorithm begins with loading the pre-processed LiDAR and INS-GNSS data. The LiDAR data includes the canopy point clouds, and the INS-GNSS data contains the roll, pitch, and yaw, and georeferenced location of the tree. For each orchard site, the reference tree point cloud frame was extracted and considered as faulty/incorrect tree canopy data. The corresponding timestamp of the reference tree point cloud acquisition was obtained. After obtaining the timestamp of the reference tree, the roll, pitch, and yaw, and the georeferenced location of the tree were extracted.
Objective 1 (a):
- Canopy Identification in Sections
The point cloud data of five consequent trees in each orchard site were processed using the developed algorithms (Fig. 6 & Fig. 7). These points were marked in different colors to represent four sections of tree canopy divided by the trellis wires.
Figure 6. Point cloud data and canopy identification in sections (orchard site 1) (a) LiDAR point cloud data after ground vegetation removal from left (tree no. 1) to the right (tree no. 5) (b) segmented tree canopy points (without tree trunk, trellis wire, and support pole)
Figure 7. Point cloud data and canopy identification in sections (orchard site 2) (a) LiDAR point cloud data after ground vegetation removal from left (tree no. 5) to the right (tree no. 1) (b) segmented tree canopy points (without tree trunk, trellis wire, and support pole)
- Canopy Density Prediction Model
These relationships between automatic point counts and manual leaf counts from two orchard sites were capable to predict number of leaves in each section of the tree. Linear regression models were developed to predict the numbers of leaves from the LiDAR point cloud data. Analysis of LiDAR point counts vs manually counted leaves suggests a fairly high correlation of 0.95 (R2 = 89.3%) reported in orchard site 2 compared to a lower correlation of 0.82 (R2 = 67.16%) calculated for orchard site 1 (Fig. 8 & Fig. 9).
Figure 8. Correlation between automatic point counts and manual leaf counts in orchard site 1
Figure 9. Correlation between automatic point counts and manual leaf counts in orchard site 2
- Canopy Density Map
The canopy density map generated in this study can provide information about where the tree has high canopy density or low canopy density (Fig. 10). Given the information about tree canopy density, adjustment of the nozzle flow rates is possible to guide the precision sprayer during the spraying operation.
Figure 10. Canopy foliage density map (a) tree canopy points without the trunk, trellis wires, and support pole points divided into sections (b) density map considering the number of canopy points (per m2) (c) canopy density map considering the number of leaves (per grid area)
Tree canopy density measurement: Tree canopy density maps were generated from the acquired points and the estimated number of leaves. Average processing time of a canopy density map was 2 seconds. A canopy density map based on the acquired points was created with small grids of size 0.0567 m × 0.0884 m (width × height of one grid). The color bar (at the right side) shows the number of canopy points per m2 area. Another canopy foliage density map was generated based on the number of leaves where a grid size of 0.04 m × 0.0125 m was created to visualize the map (Fig. c). The color bar used in the map provides information about the number of leaves per 0.005 m2 (i.e., area of one grid) area. Using the canopy density map, the number of leaves in each section of the tree can be easily calculated. Canopy foliage density is an important parameter for site-specific pest management. The canopy density measurement system developed in this study helps locate positions where the tree may have a higher chance of being infected. The canopy density information would also be an indicator for controlling the flow rate of spray nozzles placed at different locations.
Objective 1 (b):
- Simulation Results for Changes in Roll, Pitch, and Yaw
Figure 11 shows the simulation results for the canopy point corrections with respect to the 20° change of both the roll and pitch angles. Twenty canopy points from an apple tree and their corresponding corrections are illustrated. In the simulation, the yaw angle was not considered because when the sprayer is travelling straight through the orchard row, the angle change around the z-axis (yaw angle) is negligible. The possibility of yaw angle change is relatively low compared to the roll and pitch angle. The change in roll or pitch angle could influence the magnitude and orientation of the canopy point position movement/correction.
Figure 11. Changes of canopy point position with roll and pitch
- Tree Canopy Point Correction:
After point cloud data was acquired, each individual tree canopy could be corrected according to the sensor orientation at the tree location and at the location when the sprayer nozzles (pretended) pass the tree. Figure 12 shows an example of a tree canopy correction. Blue and red colors represent the tree canopy points before and after correction, respectively.
Figure 12. Canopy point corrections in longitudinal slope; tree canopy point correction about (a) 2.44 m (change of roll (1.29°), pitch (4.68°), and yaw (0.02°)) (b) 4.88 m (change of roll (3.36°), pitch (8.77°), and yaw (0.39°)), and (c) 7.32 m (change of roll (4.83°), pitch (13.46°), and yaw (0.57°)) from the reference tree
The change of position is dependent on the geometric components in the model equation. The correction is needed when there is a slope change for accurate tree canopy characteristics measurements, aim to reduce the off-target spray deposition. In the future, the integration of the state-of-art of this study will be merged with the canopy density algorithms for accurate tree canopy foliage density measurement. The nozzle flow rate of the precision spraying system will be regulated for applying variable-rate pesticides to the apple tree canopies.
Education & Outreach Activities and Participation Summary
Two articles (one research and one review article) were submitted and now under-review in two reputed journals. Two conference abstracts were submitted recently at 2021 American Society of Agricultural and Biological Engineers (ASABE) Annual International Meeting. One upcoming oral presentation based on this project's research results will be given at 2021 Mid-Atlantic Fruit and Vegetable Convention in February 2021.