Final Report for LS05-214
Project Information
The purpose of this evaluation is to determine the quantifiable impacts of
S-SARE R&E grants and the reach or diffusion of the grantees’ efforts to farmers,
ranchers, and other pertinent user groups, particularly in the general locale
of the funded project.
The primary research objectives are to:
1. Determine if the scientific community accepted the research (refereed publications).
(Impacts)
2. Determine what impact adopting the change may have had on some facet of
farming/ranching operations (e.g., profitability, increase or decrease in labor
or management, fertilizer costs, yields per acre, soil/air/water quality
changes). (Impact)
3. Determine how many other farmers or ranchers attended field days or had
personal visits with the researchers/educators. (Reach)
4. Estimate how many of those visiting farmers or ranchers also tested or
adopted a practice or technology. (Reach)
5. Determine if changes in the grant-making, contracting or reporting process,
or requirements are necessary to make the program more user-friendly,
based on the comments of grantees. (Customer Service)
The Center for Evaluative Studies in Michigan State University’s Department of
Community, Agriculture, Recreation and Resource Studies (MSU CARRS)
responded to the 2006 Targeted Request for Proposals (RFP) From the Southern
SARE Region. A grant recipient survey instrument developed for Western
Region SARE was used for data collection. Only small and regionally-specific
changes were made to the prototype survey.
A survey was administered to all 1994 to 2004 Southern Region SARE R&E
grant recipients in winter and spring 2007. Principal investigators were contacted
through a series of postal and electronic mailings. In each mailing the
survey webpage link and unique access code were provided. In the electronic
mailings, respondents could link directly to the survey from within the emailed
message. Follow-up reminder mailings were used to increase overall response
rate.
During the 18 weeks of data collection, non-responders received a paper
version of the survey in their reminder mailing. The sequence of mailings was
initiated on February 9, 2007, and contacts continued through June 19, 2007.
The overall response rate was 60.75% by principal investigator (n=107) and
59.66% by project (n=119).
Data from on-line surveys was electronically entered in the survey database
established in Vovici. Data from paper surveys was entered manually by one
of the project team members. Data were downloaded into SPSS from Vovici.
Data were randomly checked for accuracy using frequency analysis. Data were
analyzed using SPSS. Threshold for reporting has been 5 respondents. Hence,
there is n/a when a question had four or less respondents.
Descriptive statistics including frequency counts, percentage, range, mean and
standard deviation were used to describe the findings. One-way analysis of
variance and t-tests were used, as appropriate, to determine differences in
mean scores by respondent groups.
(Editor's note: A very detailed final report complete with tables and figures that cannot fit into this reporting template is available from the Southern Region SARE office by calling (770) 412-4787.)
The purpose of this evaluation is to determine the quantifiable impacts of SSARE
R&E grants and the reach or diffusion of the grantees’ efforts to farmers,
ranchers, and other pertinent user groups, particularly in the general locale of
the funded project. The primary research objectives are to:
1. Determine if the scientific community accepted the research (refereed publications).
2. Determine what impact adopting the change may have had on some facet of
farming/ranching operations (e.g., profitability, increase or decrease in labor
or management, fertilizer costs, yields per acre, soil/air/water quality
changes). (Impact)
3. Determine how many other farmers or ranchers attended field days or had
personal visits with the researchers/educators. (Reach)
4. Estimate how many of those visiting farmers or ranchers also tested or
adopted a practice or technology. (Reach)
5. Determine if changes in the grant-making, contracting or reporting process,
or requirements are necessary to make the program more user-friendly,
based on the comments of grantees. (Customer Service)
The Southern Sustainable Agriculture Research and Education (S-SARE) program
consists of competitive grant programs funded by the United State
Department of Agriculture (USDA). The goal for these programs is to enable
the full spectrum of American farmers and ranchers to move profitably toward
production systems that are compatible with the concepts of sustainable agriculture.
The Southern SARE region is comprised of 13 U.S. states ranging from
Texas in the west to our nations capitol in the east, as well as several
Caribbean island nations.
The Southern SARE program administers grants in several categories that aim
to increase knowledge about sustainable agricultural practices and to help
farmers and ranchers adopt those practices. Sustainable agriculture is an integrated
system of plant and animal production practices that will, over the long
term, satisfy human food and fiber needs; enhance environmental quality;
make efficient use of non-renewable resources; sustain economic viability of
farm operations; and enhance the quality of life for farmers, ranchers, and
society as a whole. SARE works toward these goals by funding research, education,
on-farm research, and professional development activities.
The Southern Region SARE Research and Education (R&E) grants program
was established in 1988 to support farmers, ranchers, and agribusinesses
seeking to research and develop more sustainable agriculture production
systems and marketing approaches. The Southern SARE R&E grant program
funds projects that are research based, that also have outreach and educational
components to disseminate its findings to producers, extension personnel,
and other pertinent user groups. Many Southern SARE-funded projects involve
scientists, producers and others in an interdisciplinary approach. Projects may
involve on-farm research trials with crops and/or livestock; quality of life, agricultural
marketing, integrated farming systems, or soil and water conservation.
Each grant is selected through a competitive review process and operates on
an annual cycle. Southern SARE requires that all funded projects:
_ Address Southern SARE goals. Limited resource farmers; organic farming
systems; environmentally sound practices/agricultural ecosystems; marketing/
economic development; policy, program evaluation, and quality of life;
components of sustainable systems; and women in sustainable agriculture
are priority areas of research and education.
_ S-SARE program is particularly interested in funding projects that have social
science emphasis.
_ Outcomes must focus on developing sustainable agriculture systems or moving
existing systems toward sustainable agriculture.
_ The project be research based with an educational/outreach component to
extend the project findings to the public.
_ The project use holistic, systems research techniques (Southern SARE, 2008).
The Southern SARE program has funded approximately 124 Research and
Education projects throughout the South since its inception. Southern SARE
has traditionally gathered results-based data directly from grant recipients in
the form of reports. In 2005, Southern SARE began a more systematic process
to assess the reach and measurable impacts of all of its grants programs.
Research
Survey Implementation
The questionnaire was programmed into a web survey format using
Websurveyor (now Vovici). A paper and pencil version was also formatted and
printed. Each respondent was assigned a unique identification/access code for
the survey. This number provided a way to track returns, to follow up with
non-responders, and to limit access to the survey to only those who were
asked to participate.
Principal investigators were contacted through a series of postal and electronic
mailings. In each, of the mailings the survey webpage link and unique access
code were provided. In the electronic mailings, respondents could link directly
to the survey from within the emailed message. Follow-up reminder mailings
were used to increase overall response rates. During the fourth week of data
collection non-responders received a paper version of the questionnaire in
their reminder mailing. As a final plea to non-responders, and in order to further
increase response rates, telephone reminder calls were made towards the
end of the data collection period. The sequence of mailings was initiated on
January 26 and contacts continued through June 11, 2007.
Survey Description
The survey was based in large part on a similar survey developed by Western
Region SARE as was specified in the call for proposals to conduct this evaluation
project. MSU researchers reviewed the Western Region survey, suggesting
and making several minor changes.
Identification of Survey Participants
The Southern Region SARE office provided researchers with a spreadsheet
containing Research and Education project information from 1994 through
2006. There were 112 project leaders or principal investigators (PIs) who were
responsible for 124 distinct projects. After excluding five PI’s who were not
reached because of wrong/missing contact information, 107 PI’s with 119 projects
comprised the valid sample.
Survey Procedure
Researchers sent a letter to each individual who was identified by Southern
SARE as a Research and Extension project leader. The letter listed title(s) of
the project(s) for which they were PI and provided a brief description of the
survey and its purposes. The letter provided a web address where PI’s could
complete the survey as well as a telephone number and e-mail address where
they could request a paper copy of the survey. An identification number was
assigned each individual R&E project. PI’s were required to use that identification
number to enter the on-line survey.
The response rate was 60.75% by principal investigator (n=61) and 59.66% by
project (n=67).
Data Entry
Data from on-line surveys was electronically entered in the survey database
established in Vovici. Data from paper surveys was entered manually by one
of the project team members.
Data Analysis
Data were downloaded into SPSS from Vovici. Data were randomly checked
for accuracy using frequency analysis. Data were analyzed using SPSS.
Threshold for reporting was five respondents. Hence, there is n/a when a
question had four or fewer respondents.
Descriptive statistics including frequency counts, percentage, range, mean and
standard deviation were used to describe the findings. One-way analysis of
variance and t-tests were used, as appropriate, to determine differences in
mean scores by respondent groups.
Major findings:
Farmers and ranchers were actively involved in the S-SARE funded research
and education projects. Participation in project planning was the most common
type with 80.6% of respondents indicating farmers or ranchers involved in
this way. The next three most common types of farmer/rancher involvement
were providing for land/equipment for test plots or site for a tour (61.2%),
active involvement in on-farm research or demonstration (58.2%), and speaking
about the project at a meeting or field day (58.2%). Much less common
were farmers/ranchers co-authoring a paper or other project (14.9%) or acting
as the project’s principal investigator (11.9%). Although the percentage of
farmer/rancher PI’s was only 11.9%, that level is still noteworthy because most
PI’s for SARE Research and Education projects are university faculty members
or nonprofit organizational staff members.
Extension personnel were involved in the S-SARE funded R&E projects.
Participation in project planning was the most common type with 88.1% of
respondents indicating Extension personnel involved in this way. The next
three most common types of Extension personnel involvement were speaking
about the project at a meeting or field day (82.1%)providing for land/equipment,
involvement in on-farm research or demonstration (74.6%), and interactive
learning from project results (74.2%). 59.7% of respondents indicated
Extension personnel involvement in authoring or co-authoring Extension
materials and 49.3% reported Extension personnel involvement in authoring or
co-authoring scholarly articles or papers. 25.4% reported that Extension personnel
were principal investigators and 13.4% reported other types of
Extension involvement.
Graduate and undergraduate students were involved in R&E projects. Most
respondents reported student involvement of some sort with 81.8% reporting
that students were employed or otherwise worked on the project. 66.7% of
respondents reported that students made a presentation related to the project
and 53% that they authored or co-authored a scholarly paper or article. 34.8%
reported that students authored or co-authored Extension materials and 22.7%
reported student involvement in other ways. Although both graduate and
undergraduate students commonly worked on projects and make project presentations,
undergraduate students typically did not author or co-author either
scholarly or Extension materials nor were they generally involved in other
ways.
Respondents listed key impacts or results of their projects along a range from
increased awareness to systemic changes:
_ Increased awareness about aspects of sustainable agriculture,
2 responses. For example, “There were demonstrations; field days.
Not sure about farmer adoption.”
_ Increased interest in aspects of sustainable agriculture, 1 response.
For example, “Forest certification will become more important to many
businesses.”
_ Increased knowledge or skills in sustainable agriculture, 34 responses.
For example, “This project illustrated 5 concepts of sustainable agriculture
and portrayed those concepts through video footage of actual farms.
Extension agents gained knowledge and teaching materials to convey this
information to farmers. Farmers learned about potential sustainable management
techniques and practices to achieve their goals.”
_ Changed behavior, practices, 12 responses. For example, “Our recommendations
have reduced insecticide use to XXX,” or “When the producers
saw that they were not containing some of their nutrients, they actively
modified management.”
_ Systemic impacts, 9 responses. For example, “Creation of a long-term systems
experiment that is serving multiple uses for research, extension/training
and academic education.”
Nearly half (49.2%) of the respondents noted unanticipated impacts or results
that they attributed to their S-SARE research and education project. These
included creating publications, development of improved relationships, and
further research.
As comprehensive as any initiative tries to be, no project can resolve every
possible issue for every possible participant. Over half (58.7%) of respondents
noted issues which they felt had been left unresolved after their Southern SARE
project’s conclusion. Frequently mentioned issues included too short time
duration of project, altogether new questions that arose during the project,
important information that was not available, inadequate funding, and solutions
that were not cost effective.
SARE helps create networks that can help deal with future issues.
Respondents indicated that interpersonal collaborations were most frequently
reported: collaboration with a colleague previously worked with (77.9%), collaboration
with a new colleague (76.5%), and influence on the direction of a
colleague’s work (70.6%). Institutional collaborations were less frequently
reported: collaboration between an 1862 and 1890 Land Grant (51.5%), collaboration
between a Land Grant and a non-profit (48.5%), and collaboration
between an 1862 and 1994 Land Grant (7.4%). Respondents also reported
other types of influences or outcomes (19.1%).
All except one respondent indicated that they were able to use their S-SARE
project to leverage other funds. 78% of respondents noted that they were able
to derive funds from at least one other source. As displayed in Table 8, 57.4%
of respondents reported that they launched a new project that built on their
Southern SARE project and that used other sources of funds. 51.5% reported
that they were able to leverage other funds for their Southern SARE project.
11.8% reported other funding advantages that resulted from their Southern
SARE project.
Respondents who had reported any funding advantage from their S-SARE
funded project were further asked to indicate how many different funding
sources they had tapped and how much funding they had leveraged. 62.7% of
respondents listed funding sources. 37.9% reported funding from one source,
18.2% from two sources, and 7.6% from three sources. The amount of funding
varied considerably with 22.8% of respondents reporting $1 to $10,000, 19.3%
reporting $10,001 to $50.000, 14.0% reporting $100,000 to $200,000, and 21.1%
reporting $200,000 or more.
SARE outreach activities are crucial to ensure that the maximum number of
farmers is reached using provided resources. Findings indicate that each
S-SARE project has touched or reached a mean of 431 farmers, of which 52 are
female, 21 African American, 20 Hispanic and 1 Native American.
Spreading S-SARE’s message through various forms of media was assessed.
Journal articles, Extension publications, fact sheets, websites, books and book
chapters were frequently used media.
An attempt was made to document the perceived impacts on farmer production,
marketing and net income through an open-ended question. 67% of
respondents described production and/or marketing changes. The farmer
impacts were most frequently changed behaviors or practices, followed by
increased knowledge or skills. Also cited less frequently were systemic
impacts, increased interest in aspects of sustainable agriculture, and increased
awareness about aspects of sustainable agriculture. Only 27.9% of respondents
estimated the number of farmers who changed practices.
Over half (51.5%) reported that their S-SARE project did not focus on net
income change and 42.6% did not respond to the question. Some 5.9% of
respondents estimated that farmer net income increased.
Personal and Institutional Impacts
Respondents were asked how participating in a S-SARE project had affected
their promotion and tenure. Over half (52.2%) reported a somewhat or very
positive effect. 22.4% reported no effect and 3.0% said they did not know. 9%
reported that they did not work within a university setting.
When asked to assess any change in support for sustainable agriculture
research and extension at their institution over the past decade, nearly three
quarters (72.0%) reported that support had somewhat or greatly increased.
3.6% reported that support had somewhat or greatly decreased. 14.0% reported
no effect, 10.5% reported that the question did not apply to their situation.
Could support for sustainable agriculture within their institution be attributed
to the Southern SARE grants program? Of those responding, 64.9% strongly or
somewhat agreed; 3.6% strongly or somewhat disagreed; and 15.8% neither
agreed nor disagreed. In addition, 10.5% reported “does not apply” and 5.3%
reported that they did not work within a university setting.
Southern SARE funded projects have made systemic impacts in teaching agricultural
science. Two thirds (69.1%) of respondents reported that project
results had been used in special, one-time classroom presentations. Half
(50.0%) reported use in regular, ongoing college or university coursework and
6.5% in regular, ongoing kindergarten through twelfth grade coursework.
Respondents were asked to rate various aspects of applying for Southern SARE
funds. On a scale of 1 — very poor to 5 — excellent, the mean ratings for the
application process were between good and excellent. Respondents rated ease
of obtaining grant application forms highest, with a mean rating of 4.52.
Following closely were ease of finding information about S-SARE funding
opportunities, policies and procedures (4.48); ease of finding information about
S-SARE-funded projects (4.36); clarity of grant application forms, selection criteria
and instructions at 4.25; and adequate time between grant announcement
and deadline to submit proposals (4.15).
When asked to rate various aspects of S-SARE’s grant-making process using
the same scale as above, highest rated was interaction/ communication with
S-SARE staff members at a mean rating of 4.33, closely followed by responding
to your post-awarded requests in a timely manner (4.19). Next were providing
timely official notification of review process outcomes (4.00); keeping applicants
informed on the status of their application and the funding decision
(3.94); and reporting requirements that eliminate redundancy (3.93); timeliness
of distributing funds for awarded projects (3.90); and nature and number of
requirements and provisions relative to size of grants (3.85). Respondents
rated clarity of feedback from the review process lowest at 3.60.
How satisfied were investigators with S-SARE programs and services? 83.4%
of those responding reported being very or somewhat satisfied with S-SARE
programs and services, with only 13.4% reporting that they were very or somewhat
dissatisfied. 1.7% reported being neither satisfied nor dissatisfied and
1.7% reported no opinion.
To what extent did S-SARE programs and services meet their expectations?
43.3% of respondents reported that S-SARE programs and services meet their
expectations; 38.3% reported that they somewhat exceed their expectations,
and 8.3% reported that they greatly exceed their expectations. No respondents
reported that S-SARE programs and services greatly fall short of their expectations,
but 8.3 % reported that they somewhat fall short of their expectations
and 1.7% reported “no opinion.”
What one thing should Southern SARE do to improve its program services?
Nearly half (49.2%) of respondents provided suggestions. Number one was to
provide additional money and time. Other categories included making proposals
easier and less time consuming, less micro-management, more feedback
on proposals, quicker distribution of funds and better grant reviewers.
In summary, Southern Region SARE has funded approximately 124 grants.
Most were planned in collaboration with agricultural producers and Extension
staff. Graduate and undergraduate students were hired to help implement program
activities. The findings were widely shared with farmers, Extension personnel,
and students through journal articles, newsletters, bulletins, workshops
and classroom presentations. Farmers and ranchers have actively participated
in field days and workshops and received research-based information through
Extension publications.
Over the past several years, Southern SARE has made a systematic attempt to
develop sustainable agricultural technologies and practices and disseminate
these to farmers and ranchers and agribusinesses. Researchers have formulated
new ideas that prove to work and they have been able to share these ideas
through networking. Results of Southern SARE have been instrumental in shifting
market dynamics to more sustainable and/or local agriculture. Forestry
and marketing of agricultural products locally are some frequently mentioned
areas where researchers have made impacts. Impact on enhanced farm
income has not been the focus of many research projects. However, many
researchers indicated that their research project has contributed to increased
net returns.
The art and science of sustainable agriculture is rapidly evolving. Southern
SARE has been a pioneer in supporting this initiative through research and
education grants. Findings of this survey show that researchers, educators,
agricultural producers, agribusinesses and
Non-governmental organizations have been collaborating in testing new ideas
and developing a knowledge base to address agricultural sustainability.
Continuity of support to this novel effort is required to strengthen the knowledge
base and disseminate the new information to the end users.