Sustainability indicators as management tools to guide farmers, scientists, policy makers and the general public

Final Report for LS05-178

Project Type: Research and Education
Funds awarded in 2005: $250,000.00
Projected End Date: 12/31/2009
Region: Southern
State: North Carolina
Principal Investigator:
Jon Brandt
North Carolina State University
Expand All

Project Information


At the Center for Environmental Farming System (CEFS) at North Carolina State University we have a unique production systems experiment where a wide range of parameters was collected since 1998. By using these data as a starting point, we tried to develop an approach to relate these data back to the guiding concept of agricultural sustainability.
A comprehensive list of economic, social, and ecological sustainability attributes has been identified. A survey using conjoint analysis methodology and direct attribute ranking was designed and conducted to estimate the relative impact of individual attributes on the overall perception of sustainability by farmers, scientists, and other stakeholders for aggregate sustainability assessment.
An indicator system, including indicators of soil quality, pesticide and fertilizer use, and tillage practices, was developed based on environmental data collected at CEFS to generate some insights into environmental impact of organic, integrated crop-animal, conventional best management practice farming, and plantation forestry. A graphical tool was also developed to assess the overall environmental impact associated with production systems evaluated at CEFS.
Measurements taken during the first 9 years of the experiment revealed significant differences among systems in terms of their environmental impact. At this stage of the experiment, our finding identified the BMP-conventional tillage system as high-risk system mostly because of extensive tillage. The certified organic system was also extensively tilled and was characterized by phosphorous buildup in soil, but performed well based on other assessment factors. Therefore, organic farming may become an alternative to conventional farming in North Carolina in terms of environmental impact reduction given the revision of its nutrient management guidelines to prevent phosphorus buildup in soil. Our data also suggest that organic farming would not significantly compromising crop yields compared to traditionally managed systems. Finally, the BMP-no tillage and the crop-animal integrated system were characterized as low-risk mainly because of reduced tillage.

Project Objectives:

To derive a manageable set of indicators to evaluate the environmental, biological, and financial aspects of sustainability of organic, BMP, conventional, and integrated farming at the system and enterprise (crop/rotation, animal activity) level.
To process data on selected sustainability indicators and collect additional data on specific sustainability indicators for which insufficient information is available from existing projects at CEFS.
To develop a tool for bringing together various indicators of sustainability in a single picture that shows at a glance where progress is made.


The SARE program has now been in place for sixteen years. Excellent research results have been obtained from both component studies and also long-term systems projects. Generally what is reported in these experiments is either the impact of the particular practice on economics (generally the ‘bottom line’ or net income per acre) or some combination of biological parameters, for example, weed densities and yield. Our experiments have become more sophisticated, and our understanding of biological processes more complete, but still there has been little attempt to relate what we are learning back to the guiding concept of ’sustainability’. Of course, this is difficult and understanding the complexity is arduous, but still, it is time to begin moving in this direction.

At the Center for Environmental Farming Systems (CEFS) we have a unique systems experiment where we have collected a wide range of parameters over the last several. Using these data as a starting point, we now seek to develop a method to relate our data to long-term sustainability. We believe that the method can be designed to be transferable so that collectively (farmers and researchers across the Southern Region) we can begin evaluating farming systems on their economic, social, agro-ecological and environmental sustainability. The proposed project will enable an integrated evaluation of all the data collected at CEFS and is a logical step toward the assessment of sustainability of the various agricultural systems under study. Our project could serve as a guide for other regional efforts aimed at monitoring the sustainability of farming systems.

To characterize agricultural systems as sustainable, the concept of sustainability has to be made operational and appropriate methods need to be designed for its long-term measurements. The most widely used definitions of sustainable development (Brundland, 1987; Munro and Holdgate, 1991) are broad and many more definitions have been constructed, all different in subtle ways (Van Calker et al., 2005). An overview of definitions of sustainability specifically with respect to agriculture can be found in e.g., Francis and Youngberg (1990), Bell and Morse (1999), Hansen (1996) or at and The variety of definitions is associated with specific economic, social or ecological concerns (Douglass, 1984) and historical and ideological roots (Kidd, 1992). The meaning of sustainability also varies according to spatial and time scales (Fresco and Kroonenberg, 1992). Different spatial scales can be distinguished (field, farm, village, region and so on) and societal views of sustainability change over time. Finally, how people from different disciplinary backgrounds view sustainability can be quite different (Lowrance et al., 1986). Particularly, there is a difference between traditional economists and ecologists as to the scope of substitution, of human capital in particular, for increasingly scarce natural resources and environmental services (Jaeger, 1995).

A common emergent theme from the various definitions of sustainability in agriculture is that it embodies ecological, economic and social dimensions (Allen et al., 1991). Within each dimension of sustainability, one or more attributes are identified and these are measured by means of indicators (Pannell and Glenn, 2000). For example, economic sustainability might include net income, but also other financial indicators that express: resistance, renewability, regeneration, and resilience (Ikerd, 2004). Knowing which parameters allow us to define the ecological sustainability of systems in the same way is also critically important. Sands and Podmore (2000) divide ecological sustainability in internal and external ecological sustainability. Internal ecological (or agro-ecological) sustainability refers to the agroecosystem itself, i.e. to the maintenance or improvement of the inherent capacity of the soil and water resources that support agricultural production. External ecological (or environmental) sustainability connects agricultural production with its impacts beyond the physical boundaries of the farm and its management. Similarly, internal social sustainability relates to working conditions of the farm operator and his employees and external social sustainability has to do with the societal concerns about the impact of agriculture on the well-being of people and animals.

Although sustainability is an important concern on several spatial scales, it is particularly relevant at farm level (Hansen and Jones, 1996). The farm level is regarded as the most important starting point because economic, ecological and social attributed come together at this level (de Koeijer et al., 1999). Systems that are sustainable for one farm at one point in time may not be sustainable for another farm at another point in time (Ikerd, 1993, p. 31). On the farm and field levels the actual production decisions are made. Environmental effects can only be assessed in the context of the crop rotation scheme, nutrient management and other practices. They are related to the specific biophysical conditions, such as type of soil and groundwater table (Wossink et al., 1992).

Indicators should respond over time to changes in farming practice, the outcomes of indicators should be easy to determine and integrative, and the format should be designed with farmers in mind. Indicators should help farmers make changes in farming systems, else it has no value to a farmer - either economic, social or environmental (Pannell, 2003). Sustainability indicators need to capture the complexities of the ecosystem yet remain simple enough to be easily and routinely monitored. Ideally a holistic appraisal of sustainability should integrate the ecological, economic and social dimension.

In the literature, several methods have been promoted for identifying sustainability in agriculture at the farm and field level (e.g., Rasul and Thapa, 2004; van der Werf and Petit, 2002; Liebig et al., 2001; Sands and Podmore, 2000). Most of these approaches however, cover only part of the three dimensions of sustainability and do not focus on the basic production processes and farming activities. Another limitation is that these approaches address only plant production whereas the farming systems investigated at CEFS include animal production as well. Evaluation of the sustainability of such a system requires the assessment for the combination of two types of production processes −animal production and plant production (Van Calker et al., 2004).

Indicators for annual stock-taking at the level of the agricultural production processes would help in finding the right balance between production economics and ecological goals where the production decisions are made (Halberg, 1999). Indicators need to be integrated with the development of improved management system and farmers (and other stakeholders) as a collective are key to this process (Freebairn and King, 2003).

Evaluation of sustainability seems particularly relevant with respect to organic farming because little comprehensive information is available on its performance compared with traditional and Best Management Practice (BMP) farming systems (Rigby and Caceres, 2001). Although organic agriculture is generally considered a form of sustainable agriculture (Ikerd, 1993) it can also have negative environmental effects. These include the leaching of nitrates from field under legumes and the volatilization of ammonia from livestock wastes, for example.

The procedure of assessment of sustainability implies four steps (e.g., Bell and Morse, 1999; von Wiren-Lehr, 2001; de Boer and Cornelissen, 2002): (1) description of the problem, (2) identification and definition of relevant economic, social, and ecological attributes, (3) selection and quantification of suitable sustainability indicators, and (4) combination/aggregation of indicator information into an overall contribution to sustainable development and the development of management advice.

The purpose of our project is to develop a methodology for steps 2, 3 and 4 with respect to sustainability of farming systems in the Southeast. We will use the outreach contacts and data at CEFS for the implementation of the method. CEFS has a strong working relationship with other state and federal agencies, non-governmental organizations, farmers, and citizens. A comprehensive list of attributes of economic, social and ecological sustainability will first be identified and then ranked by using the perceptions of different stakeholder and experts. Data collected at CEFS will be used to quantify the selected indicators. Next, graphical and numerical comparisons will be employed for an overall assessment and evaluation that can be adapted to natural conditions and socio-institutional constraints.


Click linked name(s) to expand/collapse or show everyone's info
  • Cavell Brownie
  • J. Paul Mueller
  • John O'Sullivan
  • Olha Sydorovych


Materials and methods:

Objective 1: to derive a manageable set of indicators
Based on literature research and expert opinion, we put together an extensive list of potential attributes that could be used to define sustainability in agriculture. A series of individual and group discussions were organized with experts in a variety of sustainability areas representing research institutions, governmental agencies, non-governmental environmental and farmer organizations. Expert selection process took into account their competence evaluated though their professional activities and publications to ensure the diversity of expertise and opinions. The experts were asked to select attributes of four general sustainability components: economic, internal social, external social, and ecological (following Van Calker et al., 2005).

We employed conjoint analysis (CA), a stated preference survey technique, to select those attributes that are the most important for agricultural sustainability based on the perceptions of stakeholders and to estimate their relative weights in the overall sustainability measure, which is a crucial input in the aggregation process. In addition, we investigated the differences in the perceptions of sustainability between farmers and scientists. These two groups were selected because they are more likely to be familiar with the concept of agricultural sustainability and may also have very distinct opinions about it. To help evaluate the performance of CA as a potential method of elicitation of relative weights in sustainability assessment studies, we used additional attribute ranking procedure for comparison purposes.

Next, an indicator system was developed based on environmental data collected at CEFS to quantify selected sustainability attributes and to generate some insights into sustainability of organic, integrated crop-animal, conventional best management practice farming, and plantation forestry.
We used data collected from experimental plots of a long-term systems study, the Farming Systems Research Unit (FSRU), at CEFS. FSRU is an 81-hectare long-term field experiment with five systems and three replications. Systems are conventional best management practices including conventional tillage (BMP/CT) and no-tillage BMP/NT), a certified organic system (COS), an integrated crop-animal system (CAS), a successional (abandoned agronomic field) system (SUS), and a plantation forestry system (PFS). Experimental plots vary in size from 1.2 to 3.8 hectares, depending on the landscape and the size of the block replicate. Soils have been intensively mapped and digitized using GPS and GIS, and sampling is done on the “diagnostic” soil type of each block to minimize soil type variability. Mueller et al. (2002) provide a full description of FSRU (Figure 1).

The two BMP systems predominate among farmers in the area representing a standard for comparison. They are characterized by management practices commonly used by regional growers, such as annual crops and short rotations. Crops are regularly scouted for pests, and pesticides are applied only when economically justified. CAS represents a 15-year rotation of crops and livestock, mainly dairy steers. COS employs unique experimental approaches to nutrient management and pest control and provides critical information that helps growers transition into organic production. Appropriate sivicultural practices are used in PFS under the production of black walnut (Juglans nigra L.). SUS represents an abandoned agronomic field allowed to succeed naturally since 1998. It serves as a control for the evaluation of farming effects.

In selection of indicators we focused on system environmental impact risks and also determined yield index values to assess system productivity. The environmental impact risk assessment was based on soil quality status, amount of pesticides and fertilizers applied, and tillage intensity.

Objective 2: to process data on selected sustainability indicators
Because system crop rotations differed (Table 1), a normalization procedure was done to compare yield data. In years where at least two systems had the same crop, we divided each yield measurement by the highest crop yield observed in the same year. The calculated ratios, ranging from 0 to 1, were used as productivity index values and allowed for system comparisons.

The Soil Management Assessment Framework (SMAF) developed by Andrews et al. (2004) was used to assess soil quality. SMAF uses a system of decision rules to identify a set of soil properties as soil quality indicators. It allows selection of specific indicators based on the primary management goal for each specific site, and makes adjustments for climate, crop rotation, tillage practices, assessment purpose, and inherent soil properties. Soil properties are transformed into unitless scores based on site-specific algorithmic relationships to soil functions. Finally, scores are averaged to produce a single index of soil quality.
Soil physical, chemical and biological properties measured in fall of 2007 were used in SMAF. Properties were available water holding capacity (AWHC), soil bulk density (Db), aggregate stability (AS), soil pH, soil phosphorus (soil P), total organic carbon (TOC), microbial biomass carbon (MBC), soil metabolic quotient (qCO2), and potentially mineralizable nitrogen (PMN). These properties were measured from soil core samples collected from the upper 7.6 cm in the five geo-referenced sampling points within the diagnostic soil of each experimental plot. For AWHC, undisturbed cores (7.6 cm in diam., 7.6 cm length) were slowly water saturated, placed in a low pressure outflow system , and desorbed to soil water potential of -10 KPa to determine field capacity. Bulk soil samples collected adjacent to the soil cores were desorbed in a high pressure outflow system to a soil water potential of -1500 KPa to determine permanent wilting point. Cores were then oven dried at 105oC to determine bulk density . AS was measured using the wet sieving procedure on air-dried aggregates of size 2.00 to 4.75 mm in diameter. Soil P was measured using the Mehlich-3 extraction method and pH using a 2:1 soil: water ratio. For MBC, field moist soil was sieved through a 3-mm sieve, immediately stored in sealed plastic bags at 4 °C and within 1 week of sampling the chloroform fumigation-direct extraction procedure was used. Total carbon was measured by combustion using a Perkin-Elmer 2400 CHN analyzer (Perkin-Elmer Corp., Norwalk, Conn.). PMN was determined using the anaerobic incubation method. Soil respiration measurements were made at each geo-referenced sampling point using the closed-chamber technique and used to determine qCO2; the proportion of soil respiration and MBC.
Analysis of variance was conducted on the soil property data and on property score values to test for system differences. The pair comparison test used was Fisher’s protected least significant difference. Results from the analysis of variance and pair comparison tests were interpreted at the 5% level of probability.
Pesticide risk assessment was a complex task because of the multidimensional nature of pesticide impacts. Product risk assessment on water resources focuses on leaching and runoff losses, both determined by pesticide persistence, water solubility, and mobility. Pesticide LD50 is the material dosage that would result in the death of 50 percent of a population of test species under standard conditions. Expressed as milligrams per kilograms of body weight it represents the acute effects of solids and liquids that are swallowed or affect the skin. We applied a measure proposed by Nelson and Bullock (2003), the number of LD50 doses applied per hectare, to represent a level of pesticide acute human risk. The pesticide Groundwater Ubiquity Score (GUS) was used to asses pesticide groundwater pollution risks (Gustafson, 1989).
Our assessment rationale for fertilizer use was based on the amount of fertilizer N and P used; i.e., less of an environmental impact as fertilizer N and P use is reduced. Fertilizer indicator values were determined as the average amount (kg/ha) of fertilizer N and P used in each system across all years (1999 – 2007).
Tillage indicator values were determined as the average number of tillage operations (e.g., plowing, chiseling, disking and cultivating) across years.

Soil quality index values were determined by SMAF as the average of all soil property scores. For pesticides, fertilizers and tillage, indicator values were normalized using the procedure of Diaz-Balteiro and Romero (2004) to produce aggregated index values.

Objective 3: to develop a tool for bringing together various indicators of sustainability in a single picture
A web graph (Gomiero and Giampietro, 2005; Krajnc and Glavic, 2005) was used to integrate all index values and delineate the overall risk in each system. Graph axes represent individual integrated risk indices. The external web of the graph represents the optimal condition for all indices. In other words, a system has less of an overall environmental risk as its web spreads more toward the outer boundary of the graph.

Research results and discussion:

No differences in soil properties were found between systems with the data collected when the study began in 1998 (baseline data – not shown). We point out that the soil quality differences presented with the data collected in fall 2007 are differences that developed over a period of nine years. System differences were found for all soil properties except for TOC (Table 2a). MBC was highest in BMP/NT, CAS COS and BMP/CT and lowest in PFS and SUS. The qCO2, which reflects the intensity of organic matter decomposition, was highest in COC and BMP/CT and lowest in BMP/NT. The higher decomposition activity in COS and BMP/CT was likely due to the increase availability of organic matter sources like crop residues and organic fertilizer which were tilled into the soil. Residues in the other four systems predominate at the soil surface and therefore decompose at a slower rate. Soil P and PMN followed the same system trend as qCO2. The greater PMN in COS and BMP/CT indicates greater availability of organic N for microbial use which also explains the greater decomposition activity.

Soil pH values were higher in the four agronomic systems (COS, BMP/CT, BMP/NT and CAS) and lowest in the two non-agronomic systems PFS and SUS. These differences are likely related to differences in organic matter dynamics and microbial activities. The highly diverse group of broadleaf plants and grasses that grow in SUS produce large plant biomass and thatch residue amounts. During the decomposition of this material acid functional groups are released from which H+ ions dissociate. Also, the carbonic acid released from microbial respiration accumulates with time acidifying the soil. In addition, grasses in these two systems with their massive root system likely produced a significant amount of carbonic acid from root respiration.
Soil bulk density was lowest in the tilled COS and BMP/CT systems and highest in the untilled CAS and BMP/NT systems. The higher Db in the untilled systems developed naturally by the settling and consolidation of soil. The baseline Db measured when the study began in 1998 was 1.36 g/cm3.
Overall score values for TOC, MBC, PMN, pH, qCO2 and AS were mostly greater than 0.90 indicating that in each system these properties were adequate (Table 2b). Lower score values and system differences were obtained for soil P, Db and AWHC. The score values of 0.12 for COS and 0.44 for BMP/CT indicate excessive soil P levels. For Db, the lowest score value was obtained with CAS, the system with the highest bulk density of 1.58 g/cm3. Studies have found that for the type of soils found at the study site, sandy loam to loamy sands, a bulk density of 1.60 g/cm3 or greater will limit root growth.
Results from the pesticide assessment are presented in table 3. The system data are means across years and crops while the crop data are means across years when that specific crop was planted. No pesticides were applied in COS, SUS, and PFS. Therefore, pesticide indicator values equal zero. The two BMP systems had the highest values for all risks. The CAS system resulted in somewhat lower risks because no pesticides were added in pasture years (2005 – 2007).
SUS and PFS are not listed in table 4 since no fertilizers were added in these systems. In COS, turkey litter was used as source of fertilizer while commercial fertilizers were used in the two BMP systems and in CAS. The two BMP systems and COS received the most N while COS received the most P. Therefore, CAS has a lower N risk while all systems except COS have low P risks. The amount of turkey litter applied that met crop N requirements in COS resulted in excess P applications. The N risk was highest with corn and sorghum and the P risk was highest with corn.
The tillage indicator value was highest in COS and lowest in BMP/NT (Table 5). A post harvest pass with a disk unit was done in BMP/NT on peanuts years (2000 and 2003) resulting in the 0.2 indicator value. More tillage was done in COS because of its requirements of primary tillage (chisel plowing), secondary tillage (disking), creation of raised beds on cabbage and sweet potato years, and cultivation for weed control. Tillage in BMP/CT and in CAS each year consisted of chisel plowing followed by disking several times until a smooth seedbed was achieved. The indicator value was lower in CAS because no tillage operations were conducted on pasture years.
Resulting index values are shown in table 6. Soil quality index values did not differ between systems averaging 0.91. This high value indicates that, based on the soil properties considered in the assessment, the overall soil quality in each system was adequate. This does not mean that all soil properties in each system were within the “ideal” range. As discussed earlier, some system properties were less than ideal and others were adequate with varying trends between systems. However, the average of property scores produced similar system soil quality index values and reflected adequate soil quality overall.
Pesticide index values were slightly lower in the two BMP systems compared with CAS where less pesticide was used. More fertilizer was used in COS and less in CAS as reflected by the respective index values of 0.60 and 0.79. The more tillage practices used in BMP/CT resulted in an index value of 0.45 while BMP/NT resulted in the highest 0.97 value.
The web graph (Figure 2) illustrates individual risk indices and the overall multi-index assessment for each system. The tillage risk was highest with COS and BMP/CT, pesticide risk with the two BMP’s, fertilizer risk with COS and no risk was associated with the soil quality factor. The two systems with the smallest web were COS and BMP/CT. Hence, the highest overall environmental impact risks are associated with these two systems. Conversely, there are lower risks with PFS and SUS, the two systems having the largest web. The two agronomic production systems with the lowest potential impact were web portrayed by BMP/NT and CAS. As illustrated in the graph the productivity among systems was similar.
The assessment identified COS and BMP/CT as the two systems with the highest environmental risk mostly because they had the highest use of fertilizers and tillage. This does not imply that these two systems are environmentally unsound, but that potential risk factors need to be well managed to avoid adverse environmental effects. Equally, a positive assessment does not imply that the system is environmentally sound. BMP/NT and CAS were characterized as low-risk relative to the agronomic practices considered, but they could be environmentally unsound due to other factors. For instance, the consolidated soil condition in the pastured CAS, as indicated by the high bulk density, may limit infiltration and therefore increase the potential for fertilizer runoff.
The soil quality assessment in this study included measurements of physical, chemical and biological properties that reflect the status of multiple soil functions affecting productivity and environmental health. However, because SMAF is limited to the use of 11 soil properties, we were unable to make the assessment using specific properties that relate to limited functions in the coarse-textured soils of the Atlantic Coastal Plains region. For instance, these soils have low nutrient retention due to low clay and low organic matter contents. Hence, humic matter and CEC would be good indicators of nutrient retention and of the potential risk for nutrient movement below the root zone. Unwanted soil conditions like compacted soil zones that restrict crop rooting and water movement also develop due to poor soil management. A standard measurement that identifies this condition would also make the assessment more resourceful.
Five of the soil properties, TOC, MBC, soil P, PMN, and AWHC, are used in SMAF on a weight basis rather than on a volume basis. Because soil bulk density changes as a function of soil management, the content of these measurements on a per soil volume basis will also change. This has been addressed in other studies and interpretation errors of 7 to 14% have been made when using soil data on a per weight basis.
The tillage assessment component can be improved by including measurements or estimations of surface crop residue cover which influences soil processes. As is, two systems each with two tillage operations per year will have the same tillage index value regardless of the amount of surface residue left post tillage. One would expect more benefits in a system with significant surface residue cover (e.g., a conservation tillage system) compared with a system with no residue cover (e.g., a conventional tillage system).
The fertilizer factor can also be improved by adding an availability component. Fertilizer availability varies with chemical composition, fertilizer type (organic versus inorganic), release activity (slow versus fast) and application methods (split, banded, etc.). In general, fertilizer rates may be high for some systems but the environmental risk can be low if the above factors are considered in an effective management scheme.

Participation Summary

Educational & Outreach Activities

Participation Summary:

Education/outreach description:

Sydorovych, O., C. Raczkowski, A. Wossink, J.P. Mueller, N. Creamer, S. Hu, M. Bell, and C. Tu. “A Technique for Assessing Environmental Impact Risks of Agricultural Systems.” Renewable Agriculture and Food Systems, forthcoming.
Sydorovych, O., and A.Wossink (2009). “The Contextual Meaning of Agricultural Sustainability: Evidence from a Conjoint Choice Survey.” Agricultural Systems, 98(1):10-20.
Sydorovych, O. and A. Wossink (2007). “Assessing Sustainability of Agricultural Systems: Evidence from a Conjoint Choice Survey.” Journal of Agricultural and Applied Economics, 39(2):416. Abstract.

Sydorovych, O. and A. Wossink (2008). “An Application of Conjoint Analysis in Agricultural Sustainability Assessment.” The Proceedings of the XIIth Congress of the European Association of Agricultural Economists. Ghent, Belgium, August 26-29.

The finding of our study were presented on several occasions: Southern Agricultural Economics Association meetings in Mobile, AL, February 2007; European Association of Agricultural Economics Congress, in Ghent, Belgium, August 2008; American Agricultural Economics Association meeting in Portland, OR, July 2007; SARE National Conference in Oconomowoc, WI, August 2006. We also shared our results while attending other project-related meetings.

Project Outcomes

Project outcomes:

Because of its complex, multi-dimensional nature, agricultural sustainability is most often assessed using numerous indicators, which make aggregate comparisons among different production systems difficult. We developed a methodology that could be beneficial in aggregate sustainability assessment presenting a flexible framework that could be adopted for a variety of specific applications and allowing comparisons among various groups. The proposed approach allows engaging scientists, policy makers, and other stakeholders in all stages of the analysis, which enhances understanding of the procedure and acceptance of the outcomes. The results of such analysis could be linked to the scientific indicators in order to derive aggregated sustainability assessment. Resulting information could also be used to draw up a scientific research agenda; i.e. to help decide where new indicators are needed.
Our assessment of the potential environmental impact risk of integrated system production practices and resulting soil conditions was relatively simple and useful. For soil quality, SMAF is user-friendly and many soil properties used are relatively easy to sample and measure. The normalization and conversion procedures used on fertilizer, pesticide, tillage and yield scores were also simple. The display of integrated indicators in a web graph gives a clear system performance overview and includes individual index values for system comparisons.


Areas needing additional study

In the proposed attribute elicitation and aggregation method, we concentrate our effort on weight elicitation and aggregation, and our stakeholder groups are selected for illustrative reasons only. The question of adequate stakeholder representation remains unsolved and should be addresses in future research.
Our assessment of sustainability of selected production systems was limited by data availability. We recognized that for a more complete assessment additional environmental performance indicators need to be developed and the indicators used can still be improved. We also would like to point out that aggregated index values for soil quality, pesticides and fertilizers were calculated as the average of individual normalized indicators. Such approach is informative about the general environmental impact trends, but with different set of relative weights assigned to individual indicators, aggregated results could vary. In addition, comprehensive assessments of this type should include economic and social indicators that would broaden the understanding of growers and determine any trade-offs between these indicators and environmental responsibility. A comprehensive study should assess systems over time to account for the temporal variations of the highly dynamic indicators used. In general, we believe that this assessment, with further development and the inclusion of additional indicators, can be a resourceful tool to promote agricultural sustainability and environmental stewardship, and to assist policy making processes. Finally, development of improved/additional indicators and proposed weight elicitation method based on CA could be linked in the future for comprehensive system assessment.

Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the U.S. Department of Agriculture or SARE.