Development of a Robotic Pruning System for Sustainable Apple Production

Progress report for GNE19-225

Project Type: Graduate Student
Funds awarded in 2019: $15,000.00
Projected End Date: 07/31/2021
Grant Recipient: The Pennsylvania State University
Region: Northeast
State: Pennsylvania
Graduate Student:
Faculty Advisor:
Long He
Pennsylvania State University
Expand All

Project Information

Project Objectives:

The primary goal of this project is the development of a robotic manipulator system for apple tree pruning. The project will be focused on the following objectives:

1. Development of pruning end-effector: The end effector will be developed by considering the maneuvering capabilities to cut the branches of all orientations and minimum spatial requirements to freely move in a limited workspace. The field test for pruning force measurement will be conducted which is essential for the selection of the end effector components. The design validation and workspace analysis of the end effector will also be performed.

2. Development of an integrated robotic manipulator for apple tree pruning: A three rotational (3R) degrees of freedom (DoF) electric shear cutter end effector will be developed. A cartesian manipulator (3P) will be developed. The end-effector will be integrated with cartesian system to develop a 6 DoF (3R3P) robotic pruning system. The simulations will be performed for the robot workspace analysis, kinematic dexterity, manipulability, and velocity ellipsoids. A microcontroller operated electronic controller will be developed for validation. Field tests will be performed to validate the design parameters and simulation results.

3. Collison-free path planning framework for branch accessibility: A simulation environment will be developed for automatic path planning of the robotic pruner. A series of simulations of the integrated systems (6R and 3R3P) in a virtual environment will be performed to determine the spatial requirements and branch accessibility. Obstacle avoidance algorithms will be developed to establish a collision-free trajectory for the end effector to reach the target position. A sensing system will be developed for 3D reconstruction of the real-world tree for the validation of the simulated collision-free paths.

 

 

Introduction:

The long-term goal of this project is to reduce the dependence of human labor for apple production, thus, to increase the competitiveness and sustainability of the United States tree fruit industry. The work is the first step towards developing a robotic pruning system for apple trees. The outcomes of this project will provide guideline information for a fully automated pruning system. 

Tree fruit industry is an important sector for U.S agriculture. Apple is an important constituent of the U.S. tree fruit industry, contributing approximately $2.75 billion to the economy, however, the production operations are still primarily dependent on human labor. In the U.S., agriculture is dependent on migrant workers but, in the past few years, the average number of hired farm workers has declined. At farms, the labor force is required almost throughout the year to perform various orchard operations such as pruning, and harvesting. These manually performed operations are labor-intensive and have high associated costs. The labor account for about 56% of the total variable costs for producing apples. Meanwhile, the limited availability of farm labor is becoming an issue for the growers, especially for time-sensitive operations. The tree fruit industry will be facing a crisis due to the issues of labor availability and associated high costs. It would be very important to have an alternative method/solution for the pruning task to reduce human labor dependency. Automation/robotics is the potential solution, but majority of the research on tree fruits robotics is focused on harvesting and thinning operations, and pruning has attracted less attention from the researchers due to the complexity of the operation.

Pruning is an important operation to maintain the fruit quality and also improves pest and disease control. Traditionally, pruning of apple trees is performed manually; but the operation is labor-intensive, costly, and the pruning decision varies from person-to-person, based on the skills and experience of the individual worker. Pruning account for about 20% of the total labor cost for producing apples. Studies reported that it requires about 30 to 35 hours of skilled labor per acre for pruning apple trees. Robotic pruning could address the issues of labor shortages and high associated costs and make uniform pruning decisions. This proposed study focuses on the investigation of some key components/technologies for automated pruning system for apple trees. The successful completion of this project will help to develop an automated pruning system for apple trees.

Cooperators

Click linked name(s) to expand
  • James Schupp (Researcher)
  • Deaun Choi (Researcher)
  • Paul Heinemann (Researcher)
  • Md Sultan Mahmud (Researcher)

Research

Materials and methods:

Objective #1:  Development of an End-effector for Robotic Pruning

1. Pruning Force Measurement

The amount of force required for cutting branches is an important parameter for the pruning end-effector design. The force measurement was performed using a traditional manual pruner and thin force sensor Phidgets-1131 (Phidgets Inc., Calgary, Canada) capable of detecting the forces exerted by the hand (Figure 1). The sensor was attached to the arm of a manual pruner and positioned to coincide with the point of contact of an operator’s finger with the shear handles to investigate the amount of force applied during pruning. The cutting torque was calculated using the measured force and the length of the pruner arm. In the test, 75 cuts were made in the ten branches with different diameters. The maximum force for each cut was recorded, as well as the corresponding branch diameter.

Figure 1. Manual pruner equipped with a force sensor for pruning force measurement

2. Pneumatic End-effector Development

A concept design of a robotic pruning end-effector were carried out in SolidWorks environment (Figure 2). Considering the needed maneuverability of the end-effector, the concept design includes two rotational joints, and a cutter which consists of a cutting blade and an anvil. The cutting mechanism was operated using the pneumatic system. The simulation was also conducted in Matlab software for workspace analysis of the end-effector, as well as the capability of aligning with the branches at different orientation.

Figure 2. A two DoF robotic pruning end-effector. 1, 2- Motor, 3, 6- Gearbox, 4- Cutter, 5- Pneumatic cylinder

3. Experiment of the Integrated Pneumatic End-effector

The field test for the validation of simulation results and performance assessment of the end-effector was conducted on tall spindle ‘Fuji’ apple trees at Penn State Fruit Research and Extension Center (FREC, Biglerville, Pennsylvania). Five ‘Fuji’ trees were selected randomly, and 15 to 20 branches of different orientations were selected from each tree for tests. The pressure of the pneumatic cylinder was set at 0.8-0.825 MPa. In test one, the pneumatic cylinder with a bore size of 2.2 cm was used, and 100 cuts were tested as multiple cuts were applied to one branch at different locations and diameter. In test two, the pneumatic cylinder with a bore size of 2.54 cm was used to provide a larger cutting force. A similar test was conducted on randomly selected trees and 120 cuts were tested including multiple cuts to one branch at different diameter and orientations on selected branches.

Figure 3. The developed robotic end-effector integrated to the three directional linear actuators

 

 

Objective #2: Development of An Integrated Manipulator System

1. Three Degrees of Freedom Pruning End-effector

The end-effector was designed with an intrinsic three-rotational (3R) degrees of freedom (DoF) configuration (Figure 4). A computer aided design (CAD) software, named SolidWorks, was used a design tool in this work. One of the primary objectives of the work was to minimize the spatial requirements of the robot during maneuvering in the tree canopy. The design criterions including manipulability, mechanical strength, and spatial requirements were satisfied using the Motion Simulation Toolbox in SolidWorks. An electric shear blade cutter was developed and attached to the last joint of the end-effector. To avoid joint collisions, the position/placement of the joints and shear cutter were selected based on the motion simulation results.

Figure 4. Concept design of the end-effector. (components: 1. Motor for yaw rotation; 2. Motor for pitch rotation; 3. Motor for roll rotation; 4. Self-locking worm gearbox; 5. Shear cutter)

2. Integrated Pruning Manipulator

A three-prismatic (3P) DoF cartesian manipulator was designed in SolidWorks for the integration of the 3P DoF end-effector. A rigid linear arm was designed to integrate both systems to make a complete 6 DoF robotic pruning system (Figure 4). SolidWorks motion simulation Toolbox was used to validate the design for joint collision and assess the performance of the system.

Figure 5. Integrated pruning system with an end-effector attached to a cartesian manipulator; the components include: 1) x-axis rails; 2) y-axis rail; 3) z-axis linear actuator; 4) axis limit switches; 5) linear arm; 6) pruning end-effector

3. Experiment of the Integrated Pruning System

The field tests were conducted on trellised ‘Fuji’/ Bud. 9 apple trees trained to a fruiting wall architecture at Penn State’s Fruit Research and Extension Center, Biglerville, Pennsylvania (Figure 6). About 100 cuts were applied on branches at a wide array of orientation ranges. A set of ten apple trees were selected randomly from the same orchard block. For each tree, about ten branches of different positions and orientations were selected. The coordinates of the cut points and the orientation of branches were estimated manually from the fixed manipulator base and entered through the GUI. Some cuts were applied approximately 10 to 20 mm from the tree trunk to evaluate the end-effector cutter capability to prune the branches close to the trunk. The branch diameter at the cut point and joint positions were recorded to validate the design parameters of the integrated pruning system.

Figure 6. Experimental setup of the integrated pruning system in the apple orchard

 

 

Objective #3: Investigation of Branch Accessibility with a Robotic Pruner

The spatial requirements of a robotic arm could be estimated based on the location of the branch to establish the trajectory of the robotic arm. Canopy characteristics such as branch density and branch dimension could possibly affect the path of the robotic arm to reach the object. The study includes the following tasks: 1) Establishing simulation environment in Matlab; 2) Collision-free trajectory generation for reaching the targeted pruning points.

1. Simulation Environment

The manipulator and a virtual tree were imported in Matlab to develop the simulation environment. The robotic manipulator used in the study was a 6R DoF industrial robotic arm (UR-5, Universal Robots, Odense, Denmark). A virtual tree with several primary branches also established in the environment. The base of the manipulator was set as origin i.e. x= 0 mm, y=0 mm, and z= 0 mm. Based on the allowable workspace of the manipulator and the physical parameters of virtual tree model, the distance between the manipulator base and tree trunk was set at 400 mm (Figure 7). The orientation of the virtual tree was set to make most branches accessible to the manipulator. Then a simulation of branch accessibility with the pruning robot was carried out to find a collision-free path for reaching to the targeted branch. 

Figure 7. Simulation environment includes virtual tree and robotic pruner

2. Simulation Approach

An obstacle avoidance algorithm using Rapidly-exploring Random Tree (RRT) was implemented for a collision-free path to reach the target pruning points. The RRT algorithm performs two checks: manipulator body collision and end-effector path collision. Target branch coordinates were added and if the algorithm found no collision, the specific path nodes are added to the final solution, and the process continues until connected nodes reach the target location. Furthermore, a RRT path smoothing, and optimization algorithms were also used to reduce path length and calculate the optimized path. The RRT path smoothing aims to reduce the path length by removing unnecessary path nodes. For path optimization, a non-linear optimization algorithm was implemented with initial and boundary conditions. The minimum avoidance distance from the obstacles were set to 60 mm. The simulation was performed for reaching different target pruning points in the virtual tree. The coordinates of target pruning points on each branch were marked 50 mm away from the tree trunk. The simulation was conducted at the perpendicular cutter pose at the target branch, i.e., the cutter opening was perpendicular to the branch axis (Figure 8). The data were recorded for path finding success, path length, and computational time for all path planning method to compare the performance.

Figure 8. Cutter approach posture at the target branch

 

 

Research results and discussion:

Objective #1:  Development of an End-effector for Robotic Pruning

1. Pruning Force Measurement

The relationship between pruning force/torque and branch diameter followed a rational 2×2 curve fit with an R2 of 0.9334 (Figure 9). The results are helpful for selecting and optimizing end-effector components such as pneumatic cylinder sizes for cutters, pressure, orientation motor torque, and mounting frames.

Figure 9. The required pruning torque for different diameter branches</strong>

2. Simulation of Robotic End-effector

These cutter orientation lines depict that the end-effector was capable of aligning the cutter to a wide range of possible orientations in a 3D workspace. The apple tree branches have a wide range of possible orientations, leading to little available space for maneuvering of the end-effector. This simulation shows that the proposed end-effector can be aligned to all possible orientations while utilizing a small workspace for maneuvering within the canopy (Figure 10).

Figure 10. Simulation results of the end-effector workspace and cutter orientation during pruning

3. Field Tests of the End-effector

The rotation capability of the end-effector along two perpendicular directions (Motor 1 and Motor 2) gave the ability to cut the branches at a wide range of orientation in a given 3D space. For any given target and branch orientation, the designed end-effector was successful in producing smooth and split-free cut branches up to 8 mm and 12 mm diameter for test one and test two respectively. The performance of the end-effector was found to be unsatisfactory to cut branches with a diameter greater than ~12 mm as it required two or three strokes of the pneumatic cylinder. The torque produced by the cutter was insufficient to produce large enough force to cut these primary branches.

A more powerful cutter is required to cut the large diameter branches i.e. 12 mm and above. For that purpose, the system may need to be equipped with a large pneumatic cylinder or a modified electric pruner considering the space utilization. The replacement of the pneumatic system with an electrical pruner will expect to improve the cutter efficiency. It was also observed that while cutting, placement of branches closer to the cutter pivot makes it easier to cut compared to when branches are placed close to the tip of the pruner blade. This was an important observation for developing an automatic trajectory and target positioning system.

 

 

Objective #2: Development of An Integrated Manipulator System

1. Simulation for Kinematic Dexterity Analysis

Computer software Matlab was used to perform a series of simulation studies. From the simulation of robot workspace analysis, it was observed that the designed robotic system has a spherical workspace. The green, blue and red lines show the 3D frame of the cutter (Figure 11). The simulation suggests that the end-effector could attain wide orientation of the cutter tool, and can reach and cut almost every branch available within the workspace of the robot.

Figure 11. Reachable workspace for the integrated end-effector with (a) Reachable points, (b) Cutter face planes, (c) Cutter tool frames

The manipulability index was determined to be independent of the rotation of the first and last joint of the end-effector (Figure 12). The undesirable joint configurations exist in all robotic manipulator, which occurs when two axis of rotation (two joints) becomes parallel to each other, resultantly the robot cannot differentiate between those two joints. In case of developed robot, these simulations suggest that the developed system has only two undesirable configurations, which is the least possible scenario for robots. Also, the places or region where these configurations could occur in our robot was vertical pointing up or down, and this is very unlikely to cut the branches by pointing the cutter up or down. The unexpected behavior of the robot could also be avoided using an electronic control system. For field tests, a microcontroller operated electronic control system was developed to control the movement of the cutter to accurately reach the target point.

Figure 12. Manipulability index of the integrated end-effector. (a) Each joint variable, (b) Combined variation in joint variables

2. Field Tests of the Integrated Manipulator 

An Arduino-based control system was also developed along utilizing a Matlab graphical user interface (GUI). A series of field tests were conducted on ‘Fuji’/ Bud. 9 apple trees with trellis-trained architecture. The coordinates of the pruning points were entered into the controller of the robot, and the robot moves to reach the target point. The field tests validated the simulation results for manipulability and dexterity. The developed cutter was able to reach the branches of wide orientation and cut up to 25 mm diameter branches. With this cutting capability, the developed system is suitable for pruning any modern apple tree architecture.

During the test, it was observed that when the target point was close to the trunk and only the perpendicular cutting posture was considered, the cutter could collide with the trunk. And when the cutter plane and branch axis were not perpendicular, the effective cutter opening for the branches to enter was greatly reduced, which increased the chance of missing the target branch. This is very important observation for automatic trajectory planning. Multiple poses of the cutter at target branch could be selected if the ideal approach is not feasible due to the complex workspace of apple tree canopy

 

 

Objective #3: Investigation of Branch Accessibility with a Robotic Pruner

The RRT algorithm was successful in finding a collision-free path i.e., red line path, for defined pruning points within the virtual tree environment (Figure 13). The RRT path may include some retention nodes which increases the path length. The smoothing method successfully reduced the RRT path lengths i.e., green line path for all target branches by removing the redundant nodes in the original path. The variation coefficient of path length for optimization method was low compared to RRT with smoothing, which confirms the stability and repeatability of the method. However, the smallest path length was achieved using the RRT with smoothing method.

The RRT smoothing was successful in reducing the time for trajectory generation and also streamlined the joint angles, velocities, and accelerations. The results suggest that the robotic arm can be used for pruning apple trees. However, the developed RRT algorithm was relatively slow in path finding. A modified RRT is suggested with optimization algorithms to stabilize the path generation and improve the obstacle avoidance efficiency. The orientation of the end-effector was also monitored considering the orientation of the tool frame during the simulation. The results show that the end-effector cutter was successfully aligned perpendicular to branch orientation before making the cut. The selection of cutter orientation is not only based on the orientation of the target branches but also on the orientation and position of the nearby obstacles.

Figure 13. Simulation of the trajectory for the pruning robot to reach a targeted branch

In the future, lab tests will be conducted to validate the results using a UR5 robotic manipulator integrated with a modified shear pruner end-effector. A vision system will be integrated to create a 3D obstacle model of the real-world apple tree branches and the collision-free path will be generated. These future studies will lay the foundation for the development of a robotic pruning system for the economical and sustainable apple production system. Furthermore, the obstacle model was simplified for parameters such as diameter, shape, position, orientation, and position of branches. The algorithm works well for the adopted obstacle model but for a more complex real world tree that includes multiple secondary branches at wide orientation ranges, the algorithm may need some adjustment based on the physical parameters of the apple tree canopy as well as environmental parameters including wind speed and terrain.

 

Participation Summary

Education & Outreach Activities and Participation Summary

2 Journal articles
2 Webinars / talks / presentations

Participation Summary

Education/outreach description:

Peer-Reviewed Articles

1. Zahid, A., Mahmud, M., He, L., Choi, D., Heinemann, P., and Schupp, J. 2020. Development of an integrated 3R end-effector with a cartesian manipulator for pruning apple trees. Computers and Electronics in Agriculture, 179, 105837

2. Zahid, A., He, L., Zeng, L., Choi, D., Schupp, J., and Heinemann, P. 2020. Development of a robotic end-effector for apple tree pruning. Transactions of the ASABE, 63(4): 847-856

Conference Papers and Presentations

1. Zahid, A., He, L., Choi, D., Schupp, J., and Heinemann, P. 2020. Collision free path planning of a robotic manipulator for pruning apple trees. In 2020 ASABE Annual International Virtual Meeting, Paper No. 2000439 (doi:10.13031/aim.202000439)

2. Zahid, A., Mahmud, M., He, L., Choi, D., Heinemann, P., and Schupp, J. 2020. Development of 3R end-effector for a cartesian manipulator for pruning apple trees. In 2020 Northeast Agricultural and Biological Engineering Conference (NABEC) Virtual Meeting

Posters

Penn State College of Engineering Research Symposium. 2020. 

Mid-Atlantic Fruit and Vegetable Convention. 2020. 

Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the U.S. Department of Agriculture or SARE.