Final Report for GNE11-024
Project Information
There is a small range in optimal fiber content (NDF) for lactating dairy cows, making quality-related harvest management decisions critical. Misestimating composition by 20% can result in late harvests by five or more days, potentially leading to NDF at harvest greater than 50 g kg-1 past target levels. This represents critical potential nutritive and economic losses for dairy farms. Accurate prediction equations exist for estimating NDF content of mixed alfalfa-grass stands in spring, and estimating the optimal harvest date. The weak link is estimating the proportion grass in a stand. The aim of this project was to improve the timing and nutritive value of spring forage harvests for dairy operations by reducing uncertainty in stand composition. In spring 2011, 2012, and 2013 we acquired samples (n=946) of alfalfa-grass stands in farmers’ fields and experimental plots, and determined alfalfa and grass dry matter proportions for each sample. In 2011, one camera was used to capture a digital image of the sampling area. Multiple cameras including an iPhone 4 were used in 2012 and 2013. A digital image analysis program was developed to estimate alfalfa-grass proportions. Multiple image analysis techniques were tested with varying unsatisfactory results, which led to identification of local binary patterns (LBP). Original images were cropped into 64 x 64 pixel tiles. Tiles in a subset of images were then classified as predominately grass, alfalfa, or non-classifiable, and transformed for pattern differentiation using LBPs. LBP differences between 2,000 known classified tiles and testing tiles were used to estimate individual tile class (alfalfa or grass). The ratio of LBP-predicted grass tiles to total tiles in each original image determined its stand composition estimate. LBP-generated estimates significantly predicted actual stand composition (p less than 0.0001, r2=0.64, RMSE=0.106) for pooled 2011 and 2012 images. This LBP-generated estimate was supplemented by additional covariates (e.g., alfalfa maximum height, grass canopy height, and lighting) for statistical model development. In some cases, models improved with additional covariates. Preliminary model testing demonstrated potential for accurate results under a range of conditions. The next phase is to continue model development, testing, and validation with the full dataset. When testing and validation are complete, the only required inputs to an Internet program accessible by computer or smart phone include an image of the stand and agronomic measures. Farmers and consultants could use this technology to prioritize the order of harvest of alfalfa-grass fields to maximize chances of obtaining desired forage NDF content for lactating dairy cow diets.
Introduction:
The purpose of this project was to improve the timing and nutritive value of spring forage harvests for dairy operations in the Northeast. Forage can be defined as a crop that can meet the effective fiber needs of a cow when fed as the primary component in the diet. Much of the cropland in the Northeast is better suited to perennial forage production than to row crop production due to climatic factors and the potential economic gains of investing land in dairy production activities. Forages accumulate large quantities of nutrients each spring, which reduces nutrient leaching and runoff potential. Thus, forage production can play an important role in reducing the environmental impact of agriculture (Cherney and Cherney, 2006).
The timing of spring forage harvests in the Northeast is critical to ensure high quality forage for dairy cattle production. Spring forage quality and timing can be predicted based on NDF concentration when forages are the principal source of fiber in dairy rations since target NDF depends on the class of livestock being fed (Parsons et al., 2006a). Other forage quality parameters, such as protein and fiber digestibility, are important for ration balancing, but they are not as useful for harvest date targets. The target NDF at harvest is approximately 500 g kg-1 dry matter (DM) for pure grass stands for silage and 400 g kg-1 DM for alfalfa (Cherney et al., 2006). Optimal timing ensures high quality first cutting spring forages, and sets the stage for high quality subsequent harvests throughout the growing season.
Equations such as the predictive equation for alfalfa quality have been designed, evaluated in multiple states, and widely used by farmers and extensionists to assess the quality and harvest timing of pure stands of alfalfa (Hintz and Albrecht, 1991; Sulc et al., 1997). Models based on growing degree days, weather-related factors, and plant morphology have also been developed and widely used for alfalfa quality prediction (Fick et al., 1994). Recent work with farmers in New York State has produced simple equations for the prediction of nutritive quality and harvest timing for pure stands of alfalfa, various grass species (e.g., timothy, reed canarygrass, orchardgrass, and tall fescue), and mixed stands of alfalfa-grass (Parsons et al., 2012; Parsons et al., 2006b). Stands have also been tested for dry matter loss and quality changes at variable stubble height, another important management factor (Parsons et al., 2012; Parsons et al., 2009). These equations have proven useful over a range of conditions and years.
Alfalfa is sown with a perennial grass companion on over 85% of the alfalfa acreage in New York State. Required inputs for mixed stand equations include alfalfa maximum height, stand composition (grass fraction in the sward), and targeted harvest NDF concentration. Stand composition is a critical parameter in the equation. It is very difficult to accurately predict by visual observation as demonstrated by the relationship between actual grass fraction and Parsons’ visual estimation of grass fraction for nearly 600 samples in 2004 (y = 0.22 + 0.69x, r2=0.43, RMSE= 0.147) (unpublished). Difficulty in estimating stand composition was also reported by extension educators at the Cornell Field Crop Advisors’ Retreat in April 2011 as the principle problem limiting the utility of mixed-stand equations for Northeast farmers. Mis-estimating composition by just 20% can result in late harvests by five or more days, potentially leading to NDF at harvest greater than 50 g kg-1 past target levels. This represents critical potential nutritive and economic losses for dairy farms. By improving stand composition estimates for use with mixed stand equations and making this information available on the farmer-centered http://www.forages.org website, farmers could improve the quality and timing of spring forage harvests.
An innovative interdisciplinary approach that combines forage science and computer science has been applied to develop a useful user-friendly application to assess mixed-stand composition from digital images and generate optimal timing estimates for spring forage harvests. This could contribute to improving the sustainability of quality forage production on Northeast dairies by reducing uncertainty in spring forage harvests. It could also improve productivity on Northeast dairies by reducing purchased forage costs and ultimately increasing dairy farm net incomes. Dairy farms and their employees could improve their quality of life due to the distribution of additional profits, thereby also yielding a positive impact on local economies.
Literature Cited:
Cherney, J.H., D.J.R. Cherney, and D. Parsons. 2006. Grass Silage Management Issues. p. 37-49. In Proceedings from “Silage for Dairy Farms: Growing, Harvesting, Storing, and Feeding”. NRAES-181. 23-25 Jan., 2006. Harrisburg, PA. Natural Resource, Agriculture, and Engineering Service, Ithaca, NY.
Cherney, D.J.R., and J.H. Cherney. 2006. Split application of nitrogen on temperate perennial grasses in the Northeast USA. Forage Grazingl. doi:10.1094/FG-2006-1211-01-RS.
Fick, G.W., P.W. Wilkens, and J.H. Cherney. 1994. Modeling forage quality changes in the growing crop. p. 757–795. In G.C. Fahey, Jr. et al. (ed.) Forage quality, evaluation, and utilization. ASA, CSSA, and SSSA, Madison, WI.
Hintz, R.W., and K.A. Albrecht. 1991. Prediction of alfalfa chemical composition from maturity and plant morphology. Crop Sci. 31: 1561-1565.
Parsons, D., K.C. McRoberts, J.H. Cherney, D.J.R. Cherney, S.C. Bosworth, and F.R. Jimenez-Serrano. 2012. Preharvest neutral detergent fiber concentration of temperate perennial grasses as influenced by stubble height. Crop Sci. 52: 923-931.
Parsons, D., J.H. Cherney, and P.R. Peterson. 2009. Pre-harvest fiber concentration of alfalfa as influenced by stubble height. Agron. J. 101:769-774.
Parsons, D., J.H. Cherney, and H.G. Gauch, Jr. 2006a. Estimation of spring forage quality for alfalfa in New York State. Online. Forage and Grazingl. doi:10.1094/FG-2006-0323-01-RS.
Parsons, D., J.H. Cherney, and H.G. Gauch, Jr. 2006b. Estimation of Preharvest Fiber Content of Mixed Alfalfa–Grass Stands in New York. Agron. J. 98:1081-1089.
Sulc, R.M., K.A. Albrecht, J.H. Cherney, M.H. Hall, S.C. Mueller, and S.B. Orloff. 1997. Field testing a rapid method for estimating alfalfa quality. Agron. J. 89:952–957.
Ultimate Objective:
Improve the timing and quality of spring forage harvests for Northeast dairy farms by reducing uncertainty in the estimation of alfalfa-grass stand composition.
Proximate Objectives:
1) Acquire digital images from representative samples of mixed stands of alfalfa-grass in Northeast farmers’ fields.
The objective was achieved. Sample sets were gathered in spring 2011, 2012, and 2013 from farmers’ fields and one experimental plot. The combined dataset consists of 946 total mixed stand samples from six species of grass. One digital image was taken for each sample in 2011. Multiple cameras including an iPhone 4 were used to acquire images of each sample in 2012 and 2013.
2) Determine known stand composition values for each sample.
This objective was achieved. The 946 representative samples of mixed stands in farmers’ fields and the experimental plot were harvested, separated into alfalfa and grass fractions, and dried to determine known stand composition values.
3) Create an alfalfa-grass recognition system that can filter and normalize a mixed stand image, evaluate the image, and return the percentage grass and alfalfa in the stand.
This objective is still in progress. Progress was delayed by challenges in developing an image analysis system that generated accurate stand composition estimates. Multiple methods were developed and tested with unsatisfactory results. We have now designed a promising alfalfa-grass recognition system consisting of complex image filters, feature extraction algorithms, local binary pattern analysis, and multivariate statistics. The system has been tested on the 2011 and 2012 datasets. It will be further refined and tested on the full dataset to finalize the recognition system, which is targeted for completion in early 2014.
4) Generate a free web service on http://www.forages.org that will allow farmers to upload images acquired from their fields and receive stand composition results and predictions of optimal forage quality and harvest timing in a rapid manner.
This objective will be undertaken following completion of the alfalfa-grass recognition system in early 2014. It is targeted for completion and field testing in time for the spring forage harvest in 2014.
Cooperators
Research
A) Sampling and Image Acquisition
Certified Crop Advisors assisted in identification of farmers’ fields containing mixed stands with different proportions of alfalfa and grass in Tompkins, Cayuga, Cortland, and Wyoming Counties of New York State. A small experimental plot was also established at Cornell University for 2013 sampling. Sampling locations represented six grass species in the mix (Table 1), which were selected based on frequent establishment in mixed stands with alfalfa in Northeast fields. Grass species sampled in mixed stands with alfalfa (Medicago sativa L.) included: timothy (Phleum pratense L.), orchardgrass (Dactylis glomerata L.), reed canarygrass (Phalaris arundinacea L.), quackgrass (Elytrigia repens (L.) Desv. Ex Nevski), smooth bromegrass (Bromus inermis L.), and tall fescue (Festuca arundinacea Schreb.). In each field, representative samples were selected and delineated using a white round hoop (66 cm diameter). It was then placed on the alfalfa and grass canopy. A digital image was acquired at five megapixel resolution using digital cameras and an iPhone 4 (Table 1). A single camera was used in 2011 and multiple cameras were used in 2012 and 2013. Camera LCD screens were mounted with a small level to ensure consistent image orientation in relation to the grass canopy. In each sample, alfalfa maximum height, grass maximum height, and grass canopy height were measured using a meter stick.
Following image acquisition, a 10 cm high quadrat was inserted under the hoop to define the cutting height and the hoop was lowered to ground level. The forage within each hoop (sample) was harvested at 10 cm above ground level by cutting at the height of the quadrat using battery-operated grass clippers. Harvested forage from each sample was labeled, transported back to the laboratory, and manually separated into alfalfa and grass fractions. Any plant species other than alfalfa and grass were removed from the sample. Samples were dried to stable weight at 60 degrees C in a forced air oven. Samples were weighed directly after removal from the oven to determine weight of alfalfa and grass fractions from each sample on a 60 degrees C dry matter basis. Dry matter weights were used to determine known contribution of grass (or alfalfa) to total dry matter, which associated with each sample and its corresponding digital images.
This multi-year, multi-grass species, and multi-camera dataset has been used to calibrate and test the alfalfa-grass recognition program. The large set of images taken with multiple cameras, of multiple grass species, and in variable field conditions, lighting, stand compositions, and years has ensured the development of a system that is accurate under different conditions and years.
B) Development of Alfalfa-Grass Recognition System
Multiple programming steps have been required to develop the alfalfa-grass recognition system (Figure 1) including (1) Image filtration, (2) Feature extraction, (3) Classification System, and (4) Generation of the stand composition estimate. First, the system user (farmer or extensionist) uploads an image to the web interface. The image is fed through a recognition controller that serves as the control system for the image processing modules described below. The image filtration module enhances important features in the image and normalizes it to a format that permits image processing by the feature extraction module. The feature extraction module uses a mathematical algorithm to extract important features in the image such as the hoop defining the processing area (only necessary in program development) and geometric attributes of alfalfa and grass leaves. This image is also subdivided into many smaller blocks (tiles) to improve the accuracy of the classification system. The classification system then further processes the filtered, extracted images using local binary patterns (Ojala et al., 2002). Finally, the results module uses information from the classification system to generate the stand composition estimate for each image. In addition to stand composition, the system will produce an estimate of current NDF level and days to harvest using existing equations (Parsons et al., 2006). Interaction between the user and the system is mitigated by the recognition controller via the web interface. For example, the user could choose to input stand composition rather than using the system generated estimate. Further details on system development are in the subsequent sections.
i) Image Filtering and Feature Extraction
Following image upload, filtration is necessary to transform the image to a format that can be processed by the classification system. This step also ensures a practical design for Northeast farmers since the system must be capable of processing images taken under different environmental conditions using different cameras and resolutions. The development and testing of algorithms for this step were executed in 2011 and 2012. Each image must be processed through a series of digital image filters to normalize the image and extract key features. For example, a series of algorithms were defined to identify the edges of the white hoop in the image and extract the inner area of the hoop (Figure 2). In system development, it was important to use a hoop to define the sampling area so that the image area in the hoop corresponded to the exact area harvested. However, hoop use is cumbersome and will not be necessary for farmers. Our automated hoop extraction program had an 80% success rate for 2011 and 2012 images. In the remaining images, hoops were extracted manually using the GNU Image Manipulation System (GIMP). The resulting ellipse from each original image was then converted to the gray scale with an emphasis on green pixels (Figure 2). This method was effective in capturing key information about grass composition in the image while minimizing sources of non-composition-related variation arising from lighting and color.
ii) Classification System
Multiple image analysis approaches were tested from 2011 to 2013 during development of the classification system with unsatisfactory results (Table 2). For example, image processing techniques were developed and combined with artificial intelligence approaches to generate stand composition estimates. Early on, the application showing the most potential for success was a tile extraction method with fast Fourier transformation (Polder et al., 2007). This was later combined with support vector machine processing using the LIBSVM open source package (Chang and Lin, 2011). However, estimates generated by the fast Fourier – support vector machine system for single grass species in the mix associated poorly with actual stand composition without additional inputs such as alfalfa maximum height and grass canopy height. Furthermore, the fast Fourier – support vector machine estimate was inaccurate under diverse conditions of the full 2011 and 2012 datasets. This method was abandoned in spring 2013. The reasons for discarding naive Bayes classifier artificial intelligence in 2012 were similar, but the method was less effective. Simpler techniques such as color separation and blob detection were not feasible for this application due to the complexity of mixed stand images. For example, blob detection was infeasible due to broken lines at the pixel resolution level. Thus, blobs (leaves) could not be defined as distinct entities for quantification during image processing. Some techniques were promising with single cameras under similar lighting and field conditions. However, relationships quickly degraded under diverse conditions.
Our current approach, the tile extraction method (Polder et al., 2007) with local binary pattern (LBP) (Figure 3) (Ojala et al., 2002), was identified in May 2013 and has been tested on 2011 and 2012 samples. The use of this method consisted of the following steps:
(1) 64 x 64 pixel chunks (tiles) were cropped from original images.
(2) Tiles in a subset of timothy-alfalfa images (n=94) from 2011 were classified as predominately grass {1}, alfalfa {0}, or unclassifiable. This step was quite time consuming because each original image contained between 300 to 600 tiles. Simple software was constructed and used to prevent user error in development of the tile classification database. This step was initially completed for testing the tile method with fast Fourier transform (Table 2) on timothy-alfalfa mixes, which is why only one grass species mix has been classified.
(3) LBP was determined for each classified tile (Figure 3) using SciPy Tools open source package (Jones et al., 2001).
(4) 2,000 classified tiles (50% grass, 50% alfalfa) were randomly selected for use as a baseline in each LBP run.
(5) The LBP for each tile in each testing image (300 to 600 tiles per image) was determined and compared to LBPs for all 2,000 randomly selected, classified baseline tiles. The lower cumulative LBP difference between each test tile and alfalfa baseline tiles and each test tile and grass baseline tiles determined the tile’s binary prediction (alfalfa or grass).
(6) Total LBP-predicted grass tiles/total tiles in each image determined its LBP-generated stand composition estimate. Multiple LBP runs with different baseline samples have been completed to date and tested on 2011 and 2012 samples. LBP estimates from a single run were used in statistical model development and evaluation (Section 5, Results and Discussion/Milestones).
iii) Stand Composition Estimate
At this stage in the alfalfa-grass recognition system, the stand composition estimate for the image would be returned to the user via the web interface along with a current NDF estimate and a harvest timing projection to achieve desired NDF at harvest, another key user input.
C) Web Service Development: After successful generation of the digital image stand composition software system, the software will be developed into a farmer-friendly application for inclusion as a free web service on http://www.forages.org. The software will incorporate calculators that employ equations to determine optimal forage quality and harvest timing (Parsons et al., 2006). Farmers will be able to simply acquire digital images from representative samples of mixed stands using a camera or smart phone, upload images acquired from their fields to the web application as well as the alfalfa maximum height parameter, and receive stand composition estimates and harvest timing predictions to achieve optimal forage quality from mixed stands at a given stubble height. This step will be completed in early 2014.
Statistical model development was completed using one image from each 2011 and 2012 sample. The LBP estimate from a single camera was randomly selected to represent each sample in the 2012 set. Pure alfalfa and pure grass samples were excluded from model development and testing. Model development was undertaken using a small set of candidate variables to predict actual grass fraction (GFRAC) (Table 3). The full pooled 2011 and 2012 dataset was used in model development, and was also subdivided for development and testing. The pooled 2011 and 2012 samples were randomly split in equal parts and developed models were tested on the set not used in development. 2011 models were also tested on the 2012 dataset and vice versa. Model construction and evaluation were completed in JMP Pro 10.0.0 using the standard least squares procedure. Models were evaluated based on coefficient of determination (R2 or r2), root mean square error (RMSE), slope, and intercept. Future model evaluation and selection will also consider the Schwarz Bayesian Criterion and the distribution of mean squared deviation components as described by Gauch et al. (2003).
The 2013 sample set was recently acquired (May, 2013), and has not yet been processed or tested. The following activities will be completed during the remainder of 2013 and early 2014 to finalize the project.
1) Apply automated hoop extraction to 2013 samples. Extract hoops manually for samples that do not process correctly with the automated system.
2) Undertake stratified classification (by grass species, camera, and possibly sampling year) of 10 to 20% of all images. This will permit a more representative random selection of baseline classified tiles for local binary pattern determination of stand composition. Estimates should improve since the current system uses only a subset of 2011 timothy-alfalfa samples (n=94) as the baseline for local binary pattern estimate generation. The time intensive classification step requires approximately 15 minutes per original image using the classifier program, and will be completed by the project team.
3) Continue local binary pattern development and testing using the updated classified sample set.
4) Continue statistical model development and testing using the full multi-year sample set to ensure system robustness across years, field conditions, grass species in the mix, and image acquisition devices.
5) Finalize local binary pattern and statistical model selection procedure as part of manuscript preparation for submission to a scientific journal.
6) Finalize alfalfa-grass recognition system and web interface in time for 2014 harvests.
7) Test the system on experimental mixed stands at Cornell University in spring 2014.
8) Disseminate extension materials on system use and conduct training sessions (plans in Section 6, Impact of Results/Outcomes).
Literature Cited:
Chang, C.-C. and C.-J. Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on
Intelligent Systems and Technology, 2:27:1--27:27.
Gauch, H.G., J.T.G. Hwang, and G.W. Fick. 2003. Model evaluation by comparison of model-based predictions and measured values. Agron. J. 95:1442–1446.
Jones, E., T. Oliphant, P. Peterson, and others. 2001. SciPy: Open source scientific tools for Python. Scikit-image toolkit version 0.7.2.win32-py2.7. http://www.scipy.org/.
McRoberts, K.C., J.H. Cherney, B.M. Benson, and D.J.C. Cherney. 2012a. Image recognition to predict alfalfa-grass stand composition. Oral presentation if abstract at 2012 North American Alfalfa Improvement, Trifolium, and Grass Breeders Conference, Ithaca, NY.
McRoberts, K.C., J.H. Cherney, B.M. Benson, and D.J.C. Cherney. 2012b. Reducing farmer uncertainty in spring forage harvests: Digital image analysis and artificial intelligence to predict alfalfa-grass stand composition. Poster presentation of abstract at American Society of Agronomy Joint International Annual Meetings, Cincinnati, OH.
Ojala, T., M. Pietikainen, and T. Maenpaa. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Analysis and Machine Intelligence. IEEE Transactions on on Pattern Analysis and Machine Intelligence. 24: 971-987.
Parsons, D., Cherney, J. H., and Gauch, H. G., Jr. 2006. Estimation of Preharvest Fiber Content of Mixed Alfalfa–Grass Stands in New York. Agronomy Journal 98:1081-1089.
Polder, G., F.K. van Evert, A. Lamaker, A. de Jong, G. van der Heijden, L.A.P. Lotz, T. van der Zalm, and C. Kempenaar. 2007. Weed detection using textural image analysis. In 6th Biennial Conference of the European Federation of IT in Agriculture (EFITA). Glasgow, UK.
- Table 1 Sampling summary by grass species from three spring harvests.
- Figure 1 Alfalfa-Grass Recognition System
- Figure 2 Hoop extraction and gray scale conversion
- Table 2 Image analysis approaches tested
- Figure 3 Tile extraction and local binary pattern
- Table 3 Variables for statistical model development
Preliminary model development and testing was completed using 2011 and 2012 datasets. Model development with only the LBP estimate yielded a strong relationship between actual and predicted values (Table 4). Developed models can be generally classified into four categories, ordered by decreasing parsimony: 1) Bivariate model with only the LBP estimate, 2) LBP estimate and agronomic measurements (e.g., AMAX and GPY), 3) LBP estimate, agronomic measurements, and grass species, and, 4) Full models with all significant covariates and interactions.
In general, the greatest gain in multivariate model performance was achieved with the addition of grass species to the model. Model calibration improved (according to higher R2 and lower RMSE of calibration) when the LBP estimate was incorporated into multivariate statistical models, and as models became more complex. The range in R2 is 0.62 to 0.86 while RMSE of calibration ranged from 0.0754 to 0.119 kg kg-1 DM. The magnitude of improvement in R2 and RMSE for more complex models may not offset the potential advantages of more parsimonious model options, which are attractive for user system simplicity (limited inputs) and because simpler models are often more accurate for prediction (Gauch, 2002). A graphical example of the bivariate relationship between actual and predicted values for a candidate multivariate model (Table 4, Equation 30) based on the full 2011 and 2012 dataset is provided (Figure 4).
The benefits of model simplicity are evident in the performance of regression models developed for prediction of GFRAC in paired datasets not used in model development (Table 5). For example, in paired data from different sampling years (2011 or 2012), the strongest, most complex calibration models ranked lowest for estimation of stand composition in the year not used in model development. Multivariate models developed with 2012 data were worse for prediction of 2011 GFRAC than the LBP estimate alone. This result was reinforced by slopes and intercepts that were significantly different than one and zero, respectively, for all models except the simple LBP bivariate model (Equation 7). 2011 models performed better for prediction of 2012 GFRAC with the exception of the most complex model. However, gains in predictive accuracy were minimal and potentially insufficient to justify use of a model more complex than the LBP estimate alone. All slopes and intercepts did not differ from one and zero, respectively, when 2011 models were used to predict 2012 GFRAC.
Models constructed from a random split of pooled 2011 and 2012 data were more accurate (Table 5). RMSE decreased and r2 increased with rising model complexity (more covariates). Results from testing Split 1 models on Split 2 data and Split 2 models on Split 1 data were similar. All models – except the most complex Split 2 model – generated slope and intercept estimates that were not significantly different than one and zero, respectively. Similar to testing paired sampling years, the use of the simplest model is attractive since the magnitude of predictive gain in r2 and RMSE is not extremely high. Furthermore, the best overall prediction equation based on r2 and RMSE criteria is the most complex Split 2 model (Equation 24). However, in this preliminary testing example, this model had significantly biased slope and intercept.
The alfalfa-grass recognition system should be accurate regardless of camera used in image acquisition. As a preliminary indicator of camera impact on LBP estimate, we examined pairwise correlations among cameras, and with the actual grass fraction for the 2012 dataset. All pairwise LBP correlations were strong and positive (r greater than 0.94) (Table 6). Furthermore, as an example of multivariate model performance for different cameras and grass species, we assessed Equation 30 (from Table 4) predictions for different cameras and grass species (Table 7). Coefficient of determination for all cameras and grass species was strong and in a consistent range. RMSE was also similar among cameras and among grass species. Intercepts did not differ from zero except for the Olympus camera. Regression slopes did not differ from one with the exception of the Olympus camera, timothy, and orchardgrass mixes. However, for slopes and intercept that differed significantly from one and zero, the magnitude of the differences was not high. Based on these preliminary results, LBP methods are promising for the estimation of stand composition. Early evidence suggests levels of predictive accuracy and precision that could yield useful results for use by Northeast farmers.
Preliminary model development and testing based on the 2011 and 2012 datasets demonstrates the assortment of challenges inherent in development and selection of prediction models that are useful under a range of variable conditions resulting from different sampling seasons, lighting, cameras, grass species, stand composition, and stand maturity. Further model improvement is anticipated with refinement of the LBP stand classification system, and the addition of the 2013 dataset in model development and testing. Future model development and testing iterations will also incorporate additional model evaluation parameters such as Mean Squared Deviation components (Gauch et al., 2003) and Schwarz Bayesian Criterion to better inform model selection. However, at this preliminary stage the LBP estimate alone more accurately estimates GFRAC than results achieved previously by visual researcher estimation.
Literature Cited:
Gauch, H.G., J.T.G. Hwang, and G.W. Fick. 2003. Model evaluation by comparison of model-based predictions and measured values. Agron. J. 95:1442–1446.
Gauch, H.G. 2002. Scientific method in practice. Cambridge Univ. Press, Cambridge.
This likelihood of long-term success from this project is high because the equations targeted by this project for improvement are already widely used by farmers and extensionists to support spring forage harvest decisions. Furthermore, over 85% of samples were collected in Northeast farmers' fields. These farmers were informed about the purpose of the project and agreed to participate, thus potentially creating conditions necessary to accelerate acceptance, adoption, and horizontal dissemination of this technology. Concurrent with completion of the alfalfa-grass recognition system in spring 2014, research findings for technology use will be documented and disseminated in various extension formats including: 1) A reader-friendly research report, 2) What’s Cropping Up? article, and 3) At least one agronomy fact sheet on estimating the composition of alfalfa-grass stands.
We will distribute findings during extension meetings and post results on http://www.forages.org to reach the widest possible audience across the region. Presentations and training sessions will also be provided at the annual Northeast Certified Crop Advisor training sessions, the annual Field Crop Advisors Retreat, and the annual New York State Field Crop Dealer Meetings.
When complete, the web system will be freely available for farmer and extensionist use on the http://www.forages.org website to accurately estimate mixed stand composition and help reduce uncertainty in spring forage harvests. Despite challenges in developing high performance image processing and artificial intelligence methods in 2012, we are optimistic that our current tile extraction and local binary pattern approach will result in a useful application for reducing farmer uncertainty in the timing of spring forage harvests to ensure high quality, thus improving agricultural sustainability for Northeast dairy farms. It will also improve productivity on Northeast dairies by reducing purchased forage costs and ultimately increasing dairy farm net incomes. Dairy farms and their employees could improve their quality of life due to the distribution of additional profits, thereby also yielding a positive net impact on local economies. Furthermore, we will convey project results to the scientific community in a manuscript focused on the development, evaluation and selection of the alfalfa-grass recognition system and associated predictive equations. Additional scientific publications in computer science and software development journals will be considered.
Outreach activities completed to date are listed in Section 8, Publications and Outreach.
Education & Outreach Activities and Participation Summary
Participation Summary:
Multiple outreach activities and conference presentations were completed in 2012.
1) An update on alfalfa-grass recognition system development was presented at the 2012 New York Field Crop Advisors Retreat in Ithaca, NY. The presentation was entitled, “Reducing farmer uncertainty in spring forage harvests: Image recognition to predict alfalfa-grass stand composition and cover crop biomass.” Follow-up presentations and training will be delivered at future retreats. Notably, participants in the 2011 retreat identified difficulty in estimating stand composition as the principle problem limiting the utility of mixed-stand equations for Northeast farmers, which led to the development of this project.
2) A progress report was presented at the annual Northeast Certified Crop Advisor training sessions to help increase awareness about the usefulness of this upcoming technology. Once technology from this project is available, it will be incorporated in training sessions at future Northeast Certified Crop Advisor sessions.
3) Method development and results of the naive Bayes classifier artificial intelligence approach were presented at the 2012 Joint North American Alfalfa Improvement, Trifolium and Grass Breeders Conference. Citation:
-McRoberts, K.C., J.H. Cherney, B.M. Benson, and D.J.C. Cherney. 2012. Image recognition to predict alfalfa-grass stand composition. Oral presentation of abstract at 2012 North American Alfalfa Improvement, Trifolium, and Grass Breeders Conference, Ithaca, NY.
4) Method development and results of the tile extraction and support vector machine approach were presented at the American Society of Agronomy Joint International Annual Meetings. Citation:
-McRoberts, K.C., J.H. Cherney, B.M. Benson, and D.J.C. Cherney. 2012. Reducing farmer uncertainty in spring forage harvests: Digital image analysis and artificial intelligence to predict alfalfa-grass stand composition. Poster presentation of abstract at American Society of Agronomy Joint International Annual Meetings, Cincinnati, OH.
Project Outcomes
Misestimating alfalfa-grass stand composition by just 20% can result in late harvests by five or more days, potentially leading to NDF at harvest greater than 50 g kg-1 past target levels. This represents critical potential nutritive and economic losses for dairy farms. The Cornell Forage Valuation Program (FORVAL) could be used to estimate the economic impact of mistimed spring harvests due to inaccurate stand composition estimates (Wilkens and Fick, 1988). This analysis has not yet been completed. It would support quantification of the nutritional and economic benefits of using the forthcoming web system to improve spring harvest timing and nutritive value.
Literature Cited:
Wilkens, P.W., and G.W. Fick. 1988. FORVAL: A computer program using chemical analyses and market data to price hay. J. Agron. Educ. 17:122-177.
Farmer Adoption
The potential for farmer acceptance and adoption of technology developed during this project is discussed in Section 6, Impact of Results/Outcomes.
Areas needing additional study
1) We anticipate initial release of the web system on http://www.forages.org in time for the spring 2014 harvest. Development of smartphone apps for iPhone and Android platforms will also be undertaken to help maximize impact via timely delivery of field results for users with smartphone technology.
2) If possible, collection of a small final 2014 dataset at Cornell University experimental plots will be used to assess the accuracy of the web system for prediction of stand composition.
3) Feedback from farmer, extensionist and consultant use of the web application and impact in spring 2014 will be used to improve the application in subsequent years. We will consider applying an ex post web-based survey following future spring harvests to evaluate participant reaction to application use and usefulness of results. The application could be improved based on the results of this assessment. Web system usage data will also provide information about actual use by Northeast farmers, and possibly by farmers and extensionists in other regions.
4) The potential economic impact of this project and related research by our research group at Cornell University has not yet been evaluated (example in Section 7, Economic Analysis). This is the primary objective of a proposed future project entitled “Management of alfalfa-grass forage to maximize economic and environmental benefits.”
5) Image analysis techniques that could be more effective for prediction of stand composition than local binary patterns could be tested for potential to further improve results. One example is local multiple patterns (Zhu and Wang, 2012). Wavelet transformation could also be considered.
Literature Cited:
Zhu, C, and R. Wang. 2012. Local multiple patterns based multiresolution gray-scale and rotation invariant texture classification. Information Sciences. 187: 93–108.