Project Overview
Annual Reports
Commodities
- Agronomic: barley, canola, corn, cotton, flax, hops, millet, oats, peanuts, potatoes, rapeseed, rice, rye, safflower, soybeans, spelt, sugarbeets, sugarcane, sunflower, wheat, grass (misc. perennial), hay
- Fruits: apples, apricots, avocados, bananas, berries (other), cherries, berries (cranberries), figs, citrus, grapes, melons, olives, peaches, pears, pineapples, plums, quinces, berries (strawberries)
- Nuts: almonds, hazelnuts, pecans, walnuts
- Vegetables: sweet potatoes, artichokes, asparagus, beans, beets, broccoli, cabbages, carrots, cauliflower, celery, cucurbits, eggplant, garlic, greens (leafy), lentils, onions, parsnips, peas (culinary), peppers, rutabagas, sweet corn, tomatoes, turnips, brussel sprouts
- Additional Plants: tobacco, herbs, native plants, ornamentals, trees
- Animals: bees, bovine, poultry, goats, rabbits, sheep, swine, fish, ratite, shellfish
- Animal Products: dairy
- Miscellaneous: mushrooms
Practices
- Animal Production: feed/forage
- Education and Training: extension, focus group, networking, participatory research
- Farm Business Management: whole farm planning
- Production Systems: agroecosystems, holistic management
- Sustainable Communities: social capital, social networks, social psychological indicators, sustainability measures
Abstract:
This study evaluates three SARE programs from inception through 2002. 171 former SARE Principal Investigators (PIs) were surveyed, and 33 interviewed, representing three grant “families”–researchers, producers, and educators – to: 1) evaluate cascade impacts of SARE funding; 2) explore ways in which SARE influences knowledge, attitudes, and behaviors; 3) understand participants’ perception of a “successful” SARE project; 4) provide recommendations; and 5) compare responses between groups. Quantitative results indicated moderate impacts; however, interviews revealed considerable support for each program’s value. Producers favored more niche production research and rated the value of the final SARE reports significantly higher than the other two groups.
Introduction:
Evaluation of government-funded programs is necessary in order to help stakeholders understand the ways in which programs add value and where they can be improved to better serve their intended populations (Henry et al., 1998; SARE, 1997; Rog & Fournier, 1997; Rogers et al., 2000; Stufflebeam, 2001). This article provides selected findings from a recent evaluation of the North Central Region Sustainable Agriculture Research and Education (NCR-SARE) program conducted as a part of the program’s commitment to evaluation and program improvement (SARE, 1997).
In 1988, the USDA established the Low-Input Sustainable Agriculture (LISA) program to fund research and adoption of sustainable agricultural (SA) practices. Later the program was revised and renamed the Sustainable Agriculture Research and Education program, and it continues under that banner (SARE, 1997). While scholars and advocates continue to offer variations on the definition of the term, sustainable agriculture (Allen, 1993; Francis et al., 1990; Hall & Kuepper, 1999; Harwood, 1990; Kroma & Flora, 2001; MacRae et al., 1990; Pretty, 1998; SARE, 1997), sustainable agricultural practices are generally considered to include those that are economically viable, environmentally sound, and socially just. Appropriate methods for measuring “it” continue to be debated among the disciplines represented under the umbrella concept of sustainable agriculture (AAE, 1998; Francis et al., 1995; Grieshop & Raj, 1990; Lightfoot & Noble, 2001; Morse et al., 2002; SARE, 1997).
The initial SARE program provided $3.9 million to fund researchers conducting collaborative, on-farm trials through Research and Education (R & E) competitive grants (SARE, 1997). In 1992, a second program was initiated in the North Central Region under the SARE umbrella, which provided funds directly to producers to conduct their own applied, on-farm research (AEE, 1998). In 1994, Congress approved a third SARE program, Professional Development Program (PDP) grants, to support sustainable agricultural education and outreach activities for Cooperative Extension system educators and specialists, Natural Resources Conservation Service staff, and others who provide educational programming to farmers and ranchers (SARE, 1997).
From inception through implementation, all funded projects include a strong outreach component and significant involvement from their farmer/rancher audience or others they are intended to reach. Research and Education grants are led by universities or non-profit organizations that use an interdisciplinary approach to fund projects up to $150,000. Successful Producer grant applicants receive funding for on-farm research that typically costs between $6000 (individuals) and $18,000 (groups). The size of PDP grants varies, depending on the number of states involved and the overall scope of the education projects. By 2004, Congress had increased funding for SARE to $16.6 million, annually. That totals $176 million for sustainable agriculture programming to fund close to 3,000 projects since the program’s inception (Elaine Hauhn, SARE Program Analyst, personal communication, May 11, 2004).
Funded projects are selected from among applications that survive a rigorous review process unique to each respective grant “family.” The process involves review by various committees made up of industry experts from universities, non-profit and for-profit organizations and producers. Grants typically range from two-year to three-year duration. Final reports are required at the conclusion of all projects.
In 1997, the USDA the report, “Ten Years of SARE,” to highlight accomplishments of the SARE funding, nationwide, during the program’s first decade. The case study report provided brief descriptions of one project in each of the four regions (North Central, Northeast, South and West) that represented each of ten categories: Crop Production, Animal Production, Natural Resource Protection, Marketing, Community Development, Education, Pest Management, Horticulture, Professional Development, and Integrated Farm/Ranch Systems.
The next year, den Biggelar and Suvedi (AEE, 1998) published “An Evaluation of the North-Central Region SARE Producer Grant Program.” This report was the result of an extensive study by the AEE Center for Evaluative Studies at Michigan State University, which examined successful and unsuccessful Producer Grant applicants on a variety of research questions. Questions included opinions about the grant program, perceptions of problems with the program, and program needs for increasing growth of sustainable agriculture. Participants were asked about their preferred sources of information on conventional and sustainable agriculture, and the economic, environmental and social impacts of the grant program. To date, there has been no parallel evaluation to assess the impacts of either the R & E or the PDP grants.
To strengthen its understanding of SARE’s impacts and effectiveness, the North Central Region SARE program has identified evaluation as an on-going process that will receive priority (Bauer, 1998). Despite a lack of formal program evaluation, we have witnessed, over the years, numerous anecdotal examples of ways in which SARE funding has led to increased knowledge, shifts in attitudes, and the embracing of farming or ranching practices that demonstrate the presence of a sustainable agricultural paradigm. In 2001, Poincelot used the SARE program as an indicator that, indeed, sustainable agriculture had become accepted in mainstream American agriculture (Poincelot, 2001).
With indicators that sustainable agriculture is now on the radar screens of administrators and researchers in the U.S. (Allen, 1993; Francis, 2000; Francis et al., 1995; Poincelot, 2001), we became curious about ways to adequately evaluate the effectiveness of the SARE program. We wanted to look not only at the short-term impacts provided by the final reports, but also the longer-term, direct and indirect effects – herein referred to as the “cascade effects” – of the SARE funding over the course of the program’s existence. In addition, we were curious about whether or not perceived differences in attitudes and behavioral outcomes toward the SARE program existed among the funded groups (researchers, producers and educators). To date, no evaluation had attempted to look at common data across the spectrum of the grant programs.
A thorough evaluation of the North Central SARE program will help decision-makers (e.g., committees and advisory boards, policy makers, lobbyists, USDA), SARE administrators (e.g., program directors, regional directors), educators (e.g., Extension, Natural Resource and Conservation Service, Natural Resources Districts, environmental educators), and potential grant applicants make the most informed decisions regarding programming, funding, and future focus. The purpose of this study was to evaluate the North Central Region’s Sustainable Agricultural Research and Education (SARE) grants operated through the U.S. Department of Agriculture, from their inception through 2002, and to compare responses between producer grant and R & E and PDP grant recipients.
The present study was heavily influenced by the Henry et al. (1998) discussion of emergent realist evaluation (ERE). Emergent realist evaluation is “an approach to evaluation based both on a philosophy called neo-realism and on an integration of lessons drawn by many evaluators from practice, theory, and research (Henry et al., 1998, p. 4). Evaluators embracing ERE recognize that an evaluation of a social program can rarely measure all of the possible consequences that each stakeholder group and the public value. In this study, ERE is an appropriate theoretical base because it recognizes that different individuals and groups assign varying levels of importance to different values, and that social programs should include evaluation of these value positions in context of the program being studied and the populations in which the program exists.
The study employed a decision/accountability-oriented evaluation approach (Stufflebeam, 2001). The purpose of this approach is to provide a knowledge and value base for making and being accountable for decisions that result in developing, delivering, and making informed use of cost-effective services. According to Stufflebeam, the decision/accountability-oriented evaluation approach should be used proactively to improve a program, as well as retroactively to judge its merit and worth. The most important purpose of the decision/accountability-oriented evaluation approach is “not to prove, but to improve" (Stufflebeam, 2001, p. 56).
The decision/accountability-oriented approach is particularly effective for addressing questions such as: Has an appropriate beneficiary population been determined? What beneficiary needs should be addressed? What are the available alternative ways to address these needs and what are the comparative merits and costs? Is the program staff sufficiently qualified and credible? Are the participants effectively carrying out their assignments? Is the program working and should it be revised in any way? Is the program reaching all targeted beneficiaries? Is the program meeting participants’ needs? Is the program better than competing alternatives? Is it sustainable? Is it transportable? (Stufflebeam, 2001).
This study provides an assessment of the SARE funding that allows grant recipients to respond to relevant survey questions, and also to give their own attributions to the SARE experience. In designing the present evaluation, we were especially interested in discovering and uncovering cascade effects, “emergent properties” (Gliessman, 1998; Henry et al., 1998), or unexpected qualities or outcomes that result from trying new practices or designing new systems on the farm, in one’s field of study, or from one’s programming.
Goals of the study were: 1) to evaluate the perceived cascade social, economic and environmental impacts of SARE funding, from inception through 2002; 2) to explore ways in which the SARE experience influenced the knowledge, attitudes, and behaviors of past grant recipients; 3) to understand participants’ perception of a “successful” SARE project; 4) to provide recommendations for strengthening the SARE program and processes; and 5) to compare responses between groups in order to discern differing attitudes and experiences that might influence adoption and engagement in sustainable agricultural activities. This evaluation will help decision-makers understand the perceived strengths and weaknesses of the regional programs from the grant recipients’ perspective. This information is necessary in order to maintain areas of strength, modify areas identified as needing improvement, attract increased participation and funding, and provide guidelines potentially helpful to other SARE regions.
This study uses the earlier den Biggelar and Suvedi (AEE, 1998) evaluation of the producer program as a reference point for developing the evaluation plan. Their comprehensive treatment in the earlier evaluation was invaluable as we took this next step toward more fully understanding the extent to which SARE funding is perceive to lead to increased awareness and adoption of sustainable practices.
References
AEE Center for Evaluative Studies (1998). An Evaluation of the North-Central Region SARE Producer Grant Program. C. den Biggelaar & M. Suvedi (Eds.), East Lansing, MI.
Allen, P. (1993). Food for the Future: Conditions and Contradictions of Sustainability. New York, Wiley & Sons.
Bauer, L. (1998). North Central Region SARE 1998 Annual Report. North Central Region SARE. Lincoln, NE: NCR-SARE.
den Biggelaar, C., & Suvedi, M. (2000). Farmers' definitions, goals, and bottlenecks of sustainable agriculture in the North Central Region. Agriculture and Human Values, 17, 347-358.
Francis, C.A. (2000). How sustainable agriculture programs impact U.S. land grant universities: an update. Sustainable Agriculture Education Workshop, Univ. Wisconsin. Madison, Wisconsin: Center for Integrated Agricultural Systems.
Francis, C. A., Edwards, C., Gerber, J., Harwood, R., Keeney, D., Liebhardt, W., & Liebman, M. (1995). Impact of sustainable agriculture programs on U.S. landgrant universities. Journal of Sustainable Agriculture, 5, 19-33.
Francis, C.A., Flora, C.B., & King, L.D. (1990). Sustainable Agriculture in Temperate Zones. New York: Wiley and Sons.
Gliessman, S.R. (1998). Agroecology: Ecological Processes in Sustainable Agriculture. Chelsea, Michigan: Ann Arbor Press.
Grieshop, J. I., & Raj, A.K. (1990). Are California's farmers headed toward sustainable agriculture? California Agriculture, 46(2): 4-7.
Hall, B., & Kuepper, G. 1999. Making the Transition to Sustainable Farming. Appropriate Technology Transfer for Rural Areas (ATTRA).
Harwood, R. R. (1990). A history of sustainable agriculture. In C. A. Edwards, R. Lal, P. Madden, R. H. Miller, & G. House (Eds.), Sustainable Agricultural Systems. Pp. 3-19. Ankeny, IA: Soil and Water Conservation Society.
Henry, G.T., Julnes, G., & Mark, M.M. (1998). Realist evaluation: An emerging theory in support of practice. San Francisco: Jossey-Bass.
Kroma, M. M., & Flora, C.B. (2001). An assessment of SARE-funded farmer research on sustainable agriculture in the north central U.S. American Journal of Alternative Agriculture, 16(2): 73-80.
Lightfoot, C., & Noble, R. (2001). Tracking the ecological soundness of farming systems: Instruments and indicators. Journal of Sustainable Agriculture, 19(1): 9-30.
MacRae, R. J., Hill, S. B., Henning, J., & Bentley, A. J. (1990). Policies, programs, and regulations to support the transition to sustainable agriculture in Canada. American Journal of Alternative Agriculture, 5(2), 76-92.
Morse, S., McNamara, N., & Acholo, M. (2002). Agricultural sustainability: Comparing external and internal perspectives. Journal of Sustainable Agriculture, 20(4): 29 - 59.
Poincelot, R. P. (2001). Editorial. Journal of Sustainable Agriculture, 17(4): 1-5.
Pretty, J. N. (1998). Supporting policies and practice for scaling up sustainable agriculture. In N. G. Röling, & M.A.E. Wagemakers (Eds.) Facilitating Sustainable Agriculture. Pp. 23-45. New York: Cambridge University Press.
Rog, D.J. & Fournier, D. (1997). Progress and Future Directions in Evaluation: Perspectives on Theory, Practice, and Methods. San Francisco: Jossey-Bass.
Rogers, P.J., Hacsi, T.A., Petrosino, A., & Huebner, T.A. (2000). Program Theory in Evaluation: Challenges and Opportunities. San Francisco: Jossey-Bass.
SARE (1997). Ten Years of SARE.
Stufflebeam, D.L. (2001). Evaluation Models. San Francisco: Jossey-Bass.
Project objectives:
1. Assess the local and regional impacts of SARE grants on productivity, economic returns, environmental and social viability of farming in agricultural landscapes and rural communities.
2. Evaluate the cascade (indirect) impacts of SARE grants on research in universities and tests by farmers, and on the current research priorities identified by investigators and farmers.
3. Develop indicators of sustainability as well as return on investment of research funds invested in SARE projects to date, and involve a graduate student in the research process.