Development of an Automated Precision Spraying System for Sustainable Pest Management in Apple Orchards

Progress report for GNE20-234

Project Type: Graduate Student
Funds awarded in 2020: $15,000.00
Projected End Date: 10/31/2022
Grant Recipient: The Pennsylvania State University
Region: Northeast
State: Pennsylvania
Graduate Student:
Faculty Advisor:
Long He
Pennsylvania State University
Expand All

Project Information

Project Objectives:

The major aim of this research is to develop an automated precision spraying system for pest management in the orchards considering the different tree canopy structures and orchard terrain conditions of Pennsylvania. The research project will be focused on the following objectives:

  1. Development of LiDAR-guided sensing technology for assessing tree canopy density of different tree architectures

A LiDAR-guided sensing system will be developed to accurately measure the canopy density of an individual tree. An individual tree will be divided into six sub-sections due to having six nozzles on each side of the sprayer unit. Then the canopy density of each section will be calculated. An Inertial Measurement System (IMU) will be used to record the posture of the LiDAR system to perform necessary adjustments of the canopy information as well as tree orientation. A Global Navigation Satellite System (GNSS) system will be used to track the geo-reference of the trees with canopy density and terrain information. The combination of the tree canopy and terrain information will be analyzed to provide guidance for the control of the precision sprayer. Real-time tests will be conducted to evaluate the system performance before integrating it into the spraying unit.

  1. Automation of nozzles and air-blast sprayer fan for target spraying

An automatic system will be developed aiming to conduct target spraying using adjusted tree canopy density information and real-time weather data. Both the nozzle and air-blast fan speed will be controlled automatically during spraying. The nozzles will be controlled by using the tree canopy information and the air-blast sprayer fan speed will be controlled by using both tree canopy density, and real-time weather information (e.g., temperature, humidity, wind speed, and direction, etc.). Fan speed control is important because it is responsible for pulls air in and redistributes it upwards into the tree canopy for reaching the spray droplet on target.

  1. Performance evaluation of the developed precision spraying system for the efficacy of field control of disease and pest

The developed precision spraying system will be tested in the field to evaluate its efficacy for the apple scab and codling moth control for the apple trees. The robustness of pest control by this newly developed spraying system will be compared with conventional methods, as well as the working efficiency and overall chemical usage.

Introduction:

The purpose of this project is to maintain the sustainability of the tree fruit industry by reducing the excessive use of pesticides in the tree-fruit orchards. This proposed study is focused on adjusting the spray volume based on canopy density of the trees aims to develop an automated precision spraying system for apple orchards. Upon completion of this project, a significant amount of pesticides could be saved which leads to reduce risk of environmental pollutions and public health as well as lower cost of production in the orchards.

Conventional sprayers waste a large number of agrochemicals between trees and into open territories above short trees or around trunks underneath canopies. As a result, a significant amount of pesticide is misused which increases production cost. On the other hand, pesticide misuse not only wastes the farmer's money but also represents a serious public concern about contamination of the environment in general and the impact of agriculture on food products specifically. In response to the critical needs of reducing chemical usages and enhance the apple industry's profitability, precise control of pesticide application is needed. Therefore, it is very important to develop a variable-rate spraying system to apply agrochemicals as needed to the target trees for reducing off-target deposition of the spray droplets.

Cooperators

Click linked name(s) to expand/collapse or show everyone's info
  • Dr. Paul Heinemann (Researcher)
  • Dr. Daeun Choi (Researcher)
  • Azlan Zahid (Researcher)
  • Dr. Grzegorz Krawczyk (Researcher)
  • Dr. Heping Zhu (Researcher)

Research

Materials and methods:

Objective #1: Development of a LiDAR-Guided Sensing Technology for Assessing Tree Canopy Density of Different Tree Architectures

Objective #1a: Measurement of Tree Canopy Density for Different Tree Architectures

1) Sensor System Integration and Data Acquisition

Tree canopy data acquisition is the first step for the canopy density measurement of the trees. A point cloud data acquisition system was developed by integrating a VLP-16 LiDAR scanner, an interface box for data transmission and power conversion, and a laptop computer. Trees in a row were scanned from both sides to cover the entire canopy in each orchard (Fig. 1).

Figure 1. Experimental setup of the LiDAR-guided system (a) utility vehicle run at the center of the row (orchard site 2) (b) a consecutive five trees were scanned

2) Tree Canopy Determination

In the test orchards, there are trellis wires and poles in the tree rows. To calculate canopy foliage density, the support poles and trellis wires needed to be segmented and removed. The tree trunk is another parameter responsible for increasing the number of points and was also removed. To detect the tree canopy points, algorithms were developed to segment the tree trunk, trellis wires and support poles from the acquired point cloud data (Fig. 2).

Figure 2. Tree trunk, trellis wire and support pole detection (TP: trunk points; PP: support pole points; TWP: trellis wire points; CP: canopy points)

3) Canopy Foliage Point and Leaf Counting

To identify canopy density, the canopy foliage points were counted from four sub-sections (Fig. 3). Determining the edge of the tree canopy is crucial to sub-divide the individual tree sections. The edge of both sides was detected. To calculate the 3D-based canopy density, the 3D grid was divided into four sub-sections according to the locations of three trellis wires.

Figure 3. Labelled canopy points in four sections of an apple tree (section 1 (bottom) to section 4 (top))

4) Canopy Density Measurement and Map

Upon removing all unnecessary points (e.g., tree trunk, trellis wires, and support pole points), the remaining points were used to calculate the canopy density of the trees. Each point cloud image from individual trees was divided into small grid areas with equal sizes of 50 cm2. The number of canopy points from each grid area was counted to generate canopy density maps.

Objective #1b: Correction of LiDAR-Sensed Tree Canopy Points in Uneven Terrain Conditions

1) Concept of Canopy Corrections

In the rough terrain condition, the sensing and spraying positions are being changed (Fig. 4). Inaccurate sensed canopy data calculates erroneous tree canopy density, which directly contributes to the off-target deposition of the spray droplets during spraying operations. To correct the sensed canopy position, a simulation was performed according to Euler's theorem. The simulation was performed to derive the correction equations for position change in x, y, and z-axis directions aimed to correct the tree canopy position. The simulations were performed using MATLAB® software. Different input parameters, including angle changes of the roll, pitch, and yaw, were used to evaluate the performance of the developed model.

Figure 4. Sensing and spraying position variations in slope change

2) Sensor System Integration

An integrated sensor system was developed, comprised of a VLP-16 LiDAR scanner (Velodyne LiDAR, San Jose, CA, USA), an Inertial Navigation System-Global Navigation Satellite System (INS-GNSS) (Inertial Labs, Paeonian Springs, VA, USA), and two laptop computers (Fig. 5).

Figure 5. Integrated hardware system for canopy position corrections

3) Sensor Fusion Algorithm for Canopy Point Corrections

A sensor fusion algorithm was developed to perform the canopy position correction. The algorithm begins with loading the pre-processed LiDAR and INS-GNSS data. The LiDAR data includes the canopy point clouds, and the INS-GNSS data contains the roll, pitch, and yaw, and georeferenced location of the tree. For each orchard site, the reference tree point cloud frame was extracted and considered as faulty/incorrect tree canopy data. The corresponding timestamp of the reference tree point cloud acquisition was obtained. After obtaining the timestamp of the reference tree, the roll, pitch, and yaw, and the georeferenced location of the tree were extracted.

Objective #2: Automation of Nozzles and Air-blast Sprayer Fan for Target Spraying

1) Integration of the Airflow Control System

An automatic airflow control system was developed by integrating a LiDAR range sensor, a micro-controller, a stepper motor, and an iris damper into an existing intelligent sprayer (Fig. 6a). The three-point hitch-mounted airblast intelligent sprayer (Smartguide Systems Inc., Indianapolis, IN, USA) was used to conduct the targeted spray operation based on tree canopy density information. The iris damper was designed and retrofitted to the sprayer air inlet for airflow control (Fig. 6b).

 

Figure 6. Automatic airflow control system mounted on the LiDAR-guided orchard sprayer: (a) hardware integration, (b) design of iris damper

2) Airflow Control Model Development

To determine the canopy density variability of the trees, the number of canopy points of individual trees were calculated based on data collected from the LiDAR scanner. The airflow was measured and spray operations were performed with different iris damper air inlet openings to determine the effective airflow required for trees of a particular density. The spray operation was conducted five times on each tree with the five damper openings. The spray coverage on each water sensitive paper (WSP) was assessed to determine appropriate damper opening for the particular density trees (Fig. 7). The recorded canopy points of trees and their effective damper openings were used to develop a mathematical model to predict the required damper opening and airflow for any sized trees. The relationship between the stepper motor steps and the openings of the iris damper was also established.

Figure 7. Experimental setup for spray deposition test with water-sensitive papers placed (marked as orange rectangle) at twelve different locations (L1 to L12)

3) Automatic Airflow Inlet Control

An algorithm was developed to adjust the damper opening based on tree canopy points to provide appropriate airflow for adequate spray deposition on the tree. The algorithm captured and then averaged the canopy point densities from ten trees in a row (Fig. 8). The average canopy density was used to calculate the required damper opening and motor steps using the two models. A stepper motor control program was uploaded to the Arduino to receive steps and rotation information to control the damper openings for automatic airflow control during the spraying operation. Field tests were conducted on calm days (low wind conditions) to evaluate the performance of the automatic airflow control system.

Figure 8. Canopy points scanned from ten consecutive trees of tested orchard site (Gala). Different colors represent Z value (height) of the point cloud

Objective #3: Performance Evaluation of the Developed Precision Spraying System for the Efficacy of Field Control of Disease and Pest

1) Field Evaluation of Precision Spraying System

The precision sprayer system was tested to evaluate its potential for chemical usage (Fig. 9). The sprayer system included an onboard tablet computer that was programmed only to open the solenoids to spray material if there were trees sensed by the laser-scanning sensor. The program calculated canopy foliage density for each corresponding nozzle and then measured the duty cycle for the pulse-width modulation (PWM) signal that manipulated the spray output. The sprayer also had the capability to switch to conventional sprayer mode in order to spray like a regular airblast sprayer.

Figure 9. Precision Sprayer is in Operation at an Apple Orchard

Experimental Design: The efficacy of the precision sprayer treatment was compared with the conventional constant-rate airblast sprayer in two experimental blocks (Gala and Rouser orchard) with cultivars: Gala. The treatments (precision sprayer and conventional airblast sprayer) were arranged in a randomized complete block design for both blocks. Sprays were applied approximately every two weeks on data trees up to harvest. The sprayer tested a total of 7 times in the apple growing season in 2021. The chemical saving was calculated each time based on the difference between conventional and precision spray applications.

Research results and discussion:

Objective 1 (a):

1) Canopy Identification in Sections

The point cloud data of five consequent trees in each orchard site were processed using the developed algorithms (Fig. 10 & Fig.11). These points were marked in different colors to represent four sections of tree canopy divided by the trellis wires.

Figure 10. Point cloud data and canopy identification in sections (orchard site 1) (a) LiDAR point cloud data after ground vegetation removal from left (tree no. 1) to the right (tree no. 5) (b) segmented tree canopy points (without tree trunk, trellis wire, and support pole)

Figure 11. Point cloud data and canopy identification in sections (orchard site 2) (a) LiDAR point cloud data after ground vegetation removal from left (tree no. 5) to the right (tree no. 1) (b) segmented tree canopy points (without tree trunk, trellis wire, and support pole)

2) Canopy Density Prediction Model

These relationships between automatic point counts and manual leaf counts from two orchard sites were capable to predict number of leaves in each section of the tree. Linear regression models were developed to predict the numbers of leaves from the LiDAR point cloud data. Analysis of LiDAR point counts vs manually counted leaves suggests a fairly high correlation of 0.95 (R2 = 89.3%) reported in orchard site 2 compared to a lower correlation of 0.82 (R2 = 67.16%) calculated for orchard site 1 (Fig. 12 & Fig. 13).

Figure 12. Correlation between automatic point counts and manual leaf counts in orchard site 1

Figure 13. Correlation between automatic point counts and manual leaf counts in orchard site 2

3) Canopy Density Map

The canopy density map generated in this study can provide information about where the tree has high canopy density or low canopy density (Fig. 14). Given the information about tree canopy density, adjustment of the nozzle flow rates is possible to guide the precision sprayer during the spraying operation.

Figure 14. Canopy foliage density map (a) tree canopy points without the trunk, trellis wires, and support pole points divided into sections (b) density map considering the number of canopy points (per m2) (c) canopy density map considering the number of leaves (per grid area)

Discussion: Tree canopy density measurement: Tree canopy density maps were generated from the acquired points and the estimated number of leaves. Average processing time of a canopy density map was 2 seconds. A canopy density map based on the acquired points was created with small grids of size 0.0567 m × 0.0884 m (width × height of one grid). The color bar (at the right side) shows the number of canopy points per m2 area. Another canopy foliage density map was generated based on the number of leaves where a grid size of 0.04 m × 0.0125 m was created to visualize the map (Fig. c). The color bar used in the map provides information about the number of leaves per 0.005 m2 (i.e., area of one grid) area. Using the canopy density map, the number of leaves in each section of the tree can be easily calculated. Canopy foliage density is an important parameter for site-specific pest management. The canopy density measurement system developed in this study helps locate positions where the tree may have a higher chance of being infected. The canopy density information would also be an indicator for controlling the flow rate of spray nozzles placed at different locations.

Objective 1 (b):

1) Simulation Results for Changes in Roll, Pitch, and Yaw

Figure 15 shows the simulation results for the canopy point corrections with respect to the 20° change of both the roll and pitch angles. Twenty canopy points from an apple tree and their corresponding corrections are illustrated. In the simulation, the yaw angle was not considered because when the sprayer is travelling straight through the orchard row, the angle change around the z-axis (yaw angle) is negligible. The possibility of yaw angle change is relatively low compared to the roll and pitch angle. The change in roll or pitch angle could influence the magnitude and orientation of the canopy point position movement/correction.

Figure 15. Changes of canopy point position with roll and pitch

2) Tree Canopy Point Correction:

After point cloud data was acquired, each individual tree canopy could be corrected according to the sensor orientation at the tree location and at the location when the sprayer nozzles (pretended) pass the tree. Figure 16 shows an example of a tree canopy correction. Blue and red colors represent the tree canopy points before and after correction, respectively.

Figure 16. Canopy point corrections in longitudinal slope; tree canopy point correction about (a) 2.44 m (change of roll (1.29°), pitch (4.68°), and yaw (0.02°)) (b) 4.88 m (change of roll (3.36°), pitch (8.77°), and yaw (0.39°)), and (c) 7.32 m (change of roll (4.83°), pitch (13.46°), and yaw (0.57°)) from the reference tree

Discussion: The change of position is dependent on the geometric components in the model equation. The correction is needed when there is a slope change for accurate tree canopy characteristics measurements, aim to reduce the off-target spray deposition. In the future, the integration of the state-of-art of this study will be merged with the canopy density algorithms for accurate tree canopy foliage density measurement. The nozzle flow rate of the precision spraying system will be regulated for applying variable-rate pesticides to the apple tree canopies.

Objective 2:

1) Air Penetrations at Various Density Trees

Air penetration through canopies was higher in the lower density trees compared to the medium and higher density trees. The recorded airflows from the damper openings showed good correlations with the number of tree canopy points. The R2 values of the “airflow-canopy relationship” were 89.5%, 81.2%, 83.8%, 86.9%, and 87.6% for openings 1, 2, 3, 4, and 5, respectively (Fig. 17).

Figure 17. Relationship between tree canopy points and airflows from five damper openings

2) Spray Deposition on Various Density Trees

Spray deposits at the ground, top, middle, bottom, backside, and next row were compared separately among five damper openings. Variations of the spray deposits were observed due to the change of the airflows (from different damper openings). A particular damper opening is considered good if it showed higher deposition on tree canopy sections (top, middle, bottom, and backside) and lower deposition at off-target sections (ground and next row), would be chosen for those types of trees. As shown in Figure 18, spray depositions were significantly affected by the damper openings.

Figure 18. Spray depositions at six different sections for GoldRush orchard (medium density). Groups identified by the same letter did not differ statistically

3) Airflow Control Model

An airflow control model was developed based on the evaluation of spray depositions using different damper openings (Fig. 19). The model shows strong prediction ability with an R2 of 97.5%. It offers automatic calculation of the required damper opening in response to the number of tree canopy points received from the LiDAR sensor.

Figure 19. Relationship between tree canopy points density and required damper opening

4) Field Validation of Developed System

Comparison results from theoretical and experimental airflow measurements reported a Mean Absolute Error (MAE) of 2.27 and 1.42 km·h-1 and a Root Mean Square Error (RMSE) of 2.41 and 1.6 km·h-1 for medium and high density orchards, respectively. The targeted trees were sprayed using the damper opening calculated from the average tree canopy points in an individual row. The amounts of spray deposition at six different sections were significantly different in orchard sites (Fig. 20).

Figure 20. Spray deposition on five tested trees in the GoldRush site (medium density). Letters represent homogeneous groups obtained by the Duncan post-hoc test (p ≤ 0.05). Groups identified by the same letter did not differ statistically

Discussion: The major airflow attributes from airblast sprayers are their ability to agitate canopies and transmit spray droplets to the target tree sections. The requirement of airflow is different among trees, primarily dependent upon tree canopy densities. High-density trees require relatively stronger airflow than medium-density, whereas relatively less airflow is required for low-density trees. However, identifying the airflow needed for particular canopy densities or orchard sites is challenging. Therefore, these results provided a potential approach for current precision airblast sprayers to match the airflow requirements of different canopy densities to reduce off-target spray depositions. Although this study was conducted with three types of tree canopy densities, the system is not limited to these. The damper opening control model can calculate the required opening for any canopy density. However, the damper opening has maximum and minimum limits, and cannot go beyond that limit when the calculated opening is out of the range. It should be noted that the airflow control system was developed with a specific configuration of the sensor, motor, micro-controller, sprayer ground speed, fan speed, nozzle type, and flow rate; hence it is very important to maintain the same configuration when integrating with the precision airblast sprayers.

Objective 3:

1) Chemical savings with the intelligent sprayer

The chemical saving was ranged from 30.6 to 53.6% (Table 1). At the beginning of the season, the chemical saving was higher than at the end of the season. This was due to the canopy density being lower in the early season than at the end of the season.

Table 1. Field evaluation results of the precision spraying system 

Discussion: Conventional orchard spraying systems waste a huge amount of agrochemical every year, which can cause serious human health hazards and environmental contamination. Excessive use of agrochemicals is equally dangerous for the ecosystem and affects their short-term or long-term changes in the normal functioning of living creatures. In this study, we evaluate the potential of the precision spraying technology to reduce the excessive agrochemical application, which is considered the tree canopy variabilities for spray decision.

Participation Summary

Education & Outreach Activities and Participation Summary

4 Journal articles
5 Webinars / talks / presentations

Participation Summary:

Education/outreach description:

Three articles (two research and one review article) were published in two reputed journals. One research article is now under-review with one peer review journal. Five conference talks were given at American Society of Agricultural and Biological Engineers (ASABE) Annual International Meeting, Northeast Agricultural & Biological Engineering Conference (NABEC) and Mid-Atlantic Fruit and Vegetable Convention. One upcoming oral presentation based on this project's research results will be given at Mid-Atlantic Fruit and Vegetable Convention in February 2022. One abstract was submitted to 2022 ASABE Annual International Meeting.

Information Products

Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the U.S. Department of Agriculture or SARE.