Final Report for ENE00-058
The main objectives of this professional development project were: (1) increase the expertise of 30 extension staff to support farmers who want to make the transition to organic grain production; (2) increase access of extension agents to resources on organic grain production, so they can improve their local education programs by creating a handbook for organic grain production; (3) distribute organic grain videos for use in training farmers.
The key components of this project were the three training programs, held at the Rodale Institute Experimental Farm and on privately owned farms in Pennsylvania and Maryland. The programs provided training, forums, resources, and field experiences to participants, and developed a network of know-how in supporting the efforts for achieving a more sustainable agricultural food system. The three training programs covered topics including: components of an organic cropping system; the federal organic standards; criteria for choosing a certification agency; record keeping requirements for certification; economics of organic grain production; farmer-to farmer organic grain marketing; qualities of organic grains for food and feed required by buyers of organic grains; and management of organic nutrients in organic cropping systems.
A handbook titled “Organic Grain: Cropping System and Marketing” was produced as a result of this project and was distributed to extension agents. In the production of this handbook on organic grain production, we successfully gathered together all the information into one source. This reference handbook is a comprehensive source of information on the underlying concepts of organic agriculture, certification requirements, farm management decisions, production techniques, and marketing.
This professional development project has also served as a venue for publicizing and distributing the information developed as part of the “Organic Grain: Another Way” resource package produced at Cornell University and funded by Northeast SARE through the Professional Development Grant program. It consists of three videos produced as part of a SARE grant coordinated by John Hall in Maryland, and two SARE books from the Handbook series, books 3 and 4. The first video covers the decisions and actions farm families need to take to remain viable economically. The second video is based on interviews with organic grain producers who address the issues of (1) why they farm organically, (2) the decision to change, (3) marketing considerations, and (4) the certification process and peer pressure concerns. The third is a how-to video that addresses the various components of an organic grain production system. Other resource materials from SARE, ATTRA, and OTA, plus regionally specific fact sheets, were included with the videos. A set of videos was purchased and distributed to each participant to use in their education programs with farmers.
This professional development project had three objectives:
1. Increase the expertise of 30 extension staff to support farmers who want to make the transition to organic grain production.
2. Increase access of extension agents to resources on organic grain production, so they can improve their local education programs by creating a handbook for organic grain production.
3. Distribute organic grain videos for training farmers.
This project was designed to provide professional development opportunities for extension agents in the northeastern U.S. in the area of organic grain production. Three two-day field days were conducted in 2001 and 2002, focusing on all aspects of organic grain crop production: production practices, cost of production, certification, marketing, and market development. A variety of educational techniques were used to achieve the objectives of the project. These included small group discussions, farm visits, video presentations, presentations by producers, industry professionals, university professors, and Rodale Institute staff, plus time for informal interaction between agents and resource people. Agents also received a considerable amount of educational resources at each event to help them with future educational programs. Videos, certification guidelines, an organic grain production handbook, and organic grain production fact sheets were provided during the course of the project.
Evaluations of each event were conducted to help plan future events and to measure the success of the project. Based on these evaluations, we concluded that this project was successful. Extension agents increased their knowledge level significantly on all aspects of organic grain production and gained considerable knowledge of resources and resource people that they can go to for more information on organic grain production.
An organizing committee was formed at the beginning of the project with members representing the diverse perspectives needed to plan and carryout this project effectively. The organizing committee members represented several states in the mid-Atlantic region to ensure that we would meet the needs of agents in New York, Pennsylvania, Delaware, New Jersey, and Maryland. We also made sure that different areas of expertise (researchers, extension specialists, and field agents) were represented. The organizing committee met twice to plan the training and make all decisions concerning curriculum content, faculty, and logistics. A considerable amount of communication and coordination through telephone and Internet was involved in planning the educational component of this project. Additional planning was done at the field days. The members of the organizing committee are listed under the team members' section. The organizing committee made all decisions about the timing, content, and logistics of the field days.
Three different training programs were planned to cover organic methods of grain production, organic certification, and marketing. Each of the training sessions was designed to use a combination of passive and active presentation methods and to encourage exchanging of views and experiences with the aim of stimulating networking and establishing future working relations between participants. Lectures were focused on information transmission, a useful tool for introducing a topic or covering large amounts of data. The sessions were completed with experiential activities in the field to help agents integrate new information into their internal conceptual framework.
Small group discussion in each field day was used to facilitate the sharing of ideas and feedback. This was accomplished by providing opportunities for speakers and participants to interact on an informal basis. This mode was particularly appropriate for providing information on production techniques, since all of the participants had vast experience and had much to offer to each other.
In addition, a group of farmers participated in each field day, presenting their experiences in a panel discussion or giving a tour of their farm operation. Their role was to provide agents with specific examples of successful organic grain production systems and also to present the challenges they faced in the transition process from conventional to organic production.
Field day participants received educational materials to help them develop their own programs. These materials were covered in the educational session and their use was demonstrated to the participants. Educational materials consisted of a video package called “Organic Grain: Another Way,” copies of organic standards, and the “Organic Grain: Cropping System and Marketing” handbook, which was reviewed at the third field day.
In April 2001, the first field day was held at the Rodale Institute experimental farm. The objectives of this field day focused on marketing and production of organic grains. This field day consisted of discussions of the local market opportunities for organic grains and featured a presentation by McGreary Grain. The process of organic grain certification was also introduced to the group at this field day. Participants were also shown the organic grain videos, which shared some of the rationale for organic grain production. The field day then focused on organic grain production and economics and included presentations from organic grain producers Ed Fry and Mary-Howell and Klaas Martens, as well as the Rodale Institute farm manager Jeff Moyer, who has vast experience in organic grain production. Weed control and biological nutrient cycling aspects of organic grain production were also introduced to the group. The results of the Farming Systems Trial at the Rodale Institute were discussed, as well.
Participants in the first field day also received other educational resources: a summary report on the Rodale Institute farming systems trial and a folder of web resources on organic production featuring ATTRA and Organic Trade Association materials. Participants also received examples of adult educational techniques that were distributed by Mena Hautau from the Berks County Extension office. In addition, each participant received a set of the organic grain videos.
In September 2001, the second field day was organized at the Rodale Institute and at Tim and Ann Bock’s organic grain farm in Pennsylvania. This field day focused on organic certification and the new organic standards. Representatives from Pennsylvania Certified Organic presented an overview of the national organic standards and provided the participants with some reference materials describing the standards. They also reviewed a step-by-step process of the certification of an organic farm. Participants were provided with a sample certification certificate and an example of a farm review report. On the second day of the field day, the group visited the Bock organic grain farm near Kutztown, Pennsylvania. Tim and Ann Bock described their farming operation and exemplary record keeping system. The group then participated in a mock certification inspection with an organic inspector. The group was able to take pictures and ask questions during the entire process. Participants were provided a sample organic farm inspection report at the end of the tour.
The third field day was held in April 2002 at Fair Hill Farm in Chestertown, Maryland, an organic grain farm that serves as the classroom for state-of-the-art organic farming practices. The objective of this field day was to assess the potential of developing regional organic markets through cooperative development or through developing regional marketing partnerships. This program started with farm tours in Chester and Kent County, Maryland, where the Horizon Organic Dairy and the Chesapeake Fields Institute have developed markets for organic grains and forages. The group visited Fair Hill Farm, operated by Ed Fry, and he shared with the group the economic benefits of organic grain production and the feasibility of transitioning from conventional to organic production. Horizon Dairy shared the challenges of organic dairy production. On the second day of the field day, two of Horizon's management team talked with the group about their thoughts on growth and trends in the organic market. Bob Anderson from Sustainable Strategies and Kim Tait then shared their experiences in developing regional marketing partnerships that could benefit organic grain producers. William Nelson from North Dakota State University then presented some of the issues that producers face in developing value-added cooperatives. The program concluded with a discussion of the cooperative development effort that John Hall has been involved in with the Chesapeake Fields Institute. Participants were provided with educational materials from each presentation.
The goal of the program was to expand the knowledge and awareness of extension agents about organic grain production and marketing. To measure performance, the Rodale Institute staff developed and administered a comprehensive evaluation plan following a pre-test/post-test design. In order to estimate the effectiveness of the field days as educational tools we conducted before-and-after surveys measuring changes in participants’ level of expertise in organic grain production and marketing following each field day. To ascertain the overall impact of the project we also conducted a follow-up survey exploring how extension agents apply the acquired knowledge and skills in their educational activities with farmers six months after attending the training events.
To estimate changes in participant’s level of expertise in organic grains production practices and management strategies after the field days, we used self-assessment questionnaires. They consisted of 16 closed-ended questions asking participants to rate their level of understanding on topics pertaining to organic grains marketing and organic certification, as well as the economics and biology of organic grain production. Responses were recorded using a five point scale, ranging from 1= “no understanding” to 5=”excellent understanding.”
The post-test questionnaires contained an additional module of four open-ended questions designed to assess the future informational and educational needs on organic grains production topics of the participating extension agents. They solicited information about educational topics of interest and preferences for hands-on demonstrations and other field day activities.
The questionnaire following the second field day consisted of two modules of six closed-ended questions each asking participants to assess the usefulness and the quality of information presented at the field day, with a particular focus on whether the information was current. Responses were measured on a five-point scale from 1=“poor” to 5=“excellent.”
The six-month follow-up survey was mailed to all 17 participants from the April 2001 field day. The follow-up questionnaire contained three modules with a total of 20 closed-ended and seven open-ended questions. The closed-ended questions asked the participants to evaluate the general relevance and usefulness of the training as well as the relevance, benefits, and outcomes of the first training field day. Responses were recorded on a 5-point scale with 1=“lowest” and 5=“highest” evaluation. The open-ended questions were designed to assess future educational needs, as well as to gather information on how extension agents have applied or plan to apply the knowledge they acquired through this project in their work as farmer educators.
Response Rate and Data Management
Attendance at the three training events was 17 participants in the first field day, 19 in the second, and 24 in the third; the respective completed surveys were 17, 17, and 16, for a response rate of 71% or above for each of the training events. All 17 extension agents attending the first field day in April 2001 completed and returned valid follow-up questionnaires for a 100% response rate.
All closed-ended questions were coded following instructions developed by the research team. Data was entered in a computer file and analysis was conducted using SPSS10.1 on a PC. We applied paired t-test analyses to compare mean scores of responses on each of the individual questions from the before and after surveys. We also created additive scales to develop composite measure of level of expertise in select areas like organic grains marketing, organic certification, and biological processes. In addition, we created composite measures of relevance, usefulness, and benefits of the training activities. All composite measures that we used were subjective and passed a satisfactory reliability analysis.
Summary of results
The overall results from our evaluation clearly show that the activities carried out for this project were an effective training resource for extension specialists and agents. The project was designed to respond to the growing demand for organic food and the new marketing opportunities opening for grain producers in the mid-Atlantic region.
Empirical results from the end-of-session questionnaires demonstrate that the field days have significantly increased the knowledge and expertise of extension agents in this area and have made an important contribution in preparing them to meet the educational and training needs of farmers interested in the production and marketing of organic grains. Of particular importance here is the compelling empirical evidence of the high relevance of the educational materials. Indeed, survey results from the three field days consistently show that following each training event the largest statistically significant increases in mean knowledge scores were recorded across topics identified as “priority information needs” by the participating extension agents. Furthermore, the overwhelming majority of participants consistently rated their acquired expertise level in all educational areas as “above average.”
Results from the six-month follow-up survey further support our findings that the training curriculum and design of this program have proven highly effective in addressing current needs for in-service training. All survey respondents give above average scores in evaluating the relevance, usefulness, and overall professional benefits that this program has offered. In fact, about half of the participating extension agents have given highest marks in evaluating the educational and other benefits of this program. Overall, our empirical results provide compelling evidence of the effectiveness of the design of this training program.
A handbook titled “Organic Grain: Cropping System and Marketing” was produced as a result of this project and distributed to extension agents. This reference handbook is a comprehensive source of information on the underlying concepts of organic agriculture, certification requirements, farm management decisions, production techniques, and marketing. This publication is available in both hard copy and CD ROM formats. The electronic format of this publication was not initially a part of the project; the CD ROM version was added because it was determined through training participant review that a computer-friendly version would be most helpful for the extension offices.
Performance Target Outcomes
April 2001 Field Day
The pre-test results conducted prior to the first field day identified three topics with higher information needs as indicated by the lower means scores of the participants.
1. Food and feed grade qualities required by buyers of organic grains.
2. Criteria for choosing a certification agency.
3. Organic grain marketing options for farmers in the mid-Atlantic states.
This information was used to plan the second and third field day content.
The pretest also showed that overall, extension agents felt most confident in their expertise in the biological aspects of organic production and their overall capabilities as farmer educators. They rate their knowledge of the process of organic certification and the economics of organic production to be about average and they report they have the least expertise in the field of organic grain marketing.
Results from the post-test questionnaire at the first field day strongly suggest that the field day made an important contribution in increasing the knowledge and expertise of extension agents in the area of organic grain production and marketing. All participants in the field day reported an increase in their level of understanding in all information topics covered in the field day. Extension agents reported that they have improved their expertise even in the biology of organic grain production and education, the two areas where they felt most knowledgeable before the field day.
Of interest was the high relevance of the educational material as participants in the field day have significantly improved their knowledge and understanding of the three topics they felt least knowledgeable about before the field day. Indeed, the mean scores across the three areas show the largest statistically significant increase among all survey questions. In addition, results from the survey indicated that extension agents participating in the field day have significantly increased their overall expertise in organic certification and marketing of organic grains. The results strongly suggest that the design of the field day was relevant to the current educational and informational needs of extension personnel.
September 2001 Field Day
Nineteen extension agents participated in the second field day. Seventeen participants completed evaluation post-field day surveys. Results indicated that most participants considered the information and hands-on demonstration at the field day to be useful to their work, with mean scores rated “average” or above. Clearly, discussion and information exchange on the topic of organic certification was perceived to be the most useful component of the field day. It was closely followed by topics of transitioning and crop rotation. Somewhat surprisingly, soil health and pest management issues rank slightly below the average, as do topics in economics. The overall evaluation of the field day was positive, with the majority of participants rating it a good or excellent source of useful information.
Presenting and disseminating up-to-date information is of utmost importance for a successful and valuable field day. This is of particular importance for topics where changes occur continuously. Extension agents participating in this event were in agreement that information on the majority of topics covered at the field day was relevant and current. Results from the survey single out organic certification and transition to organic as the two areas providing most current information. Discussion and demonstrations about soil health and rotation were ranked about average, while topics on pest control and economics were perceived to contain somewhat dated information. The overall evaluation of the usefulness and innovative context of the discussions and field day demonstrations received a high mark as indicated by their high mean scores.
April 2002 Field Day
Twenty-four extension agents participated in this second field day, funded by the SARE grant. The evaluation was again conducted using a before-and-after design, with the original self-assessment questionnaire applied at the first field day. Seventeen participants completed the pre-field day survey, and 16 filled in the post-field day survey. Missing data on individual items reduced the sample size to 14 respondents. Analysis is based on mean scores of the individual questions and summary scales for marketing, certification, and biological processes.
Areas of main current information need:
1. Criteria for choosing a certification agency with the lowest mean score.
2. Food and feed grade qualities required by buyers of organic grains.
3. Organic grain options for farmers in the mid-Atlantic States.
Interestingly, participants in this second field day identified the same three individual items that were singled out in the evaluation survey for the April 2001 field day. Clearly, those particular topics in organic marketing and production need further exploration, and the demand for information is consistently high. It is important to note, however, that the mean levels across the three items are higher for the April 2002 pre-field day survey indicating that extension personnel have acquired more knowledge and expertise in this field over the course of the year.
Overall results from the 2002 surveys show a pattern similar to that of the surveys in 2001: The responses indicate that participants in the field day have advanced their understanding and knowledge in most of the individual topics discussed. Some notable differences possibly result from the overall higher expertise level reported before the field day. Thus, the overall self-reported increase in understanding is smaller compared to that reported after the 2001 field day. In addition, advancement in knowledge is least impressive in biological processes, and furthermore, the change is not statistically significant. On the other hand, following the pattern from the first field day evaluation, the most impressive gain in understanding is observed where most need was identified to begin with. Results show a statistically significant increase in expertise level in organic marketing and certification, the two areas of highest information demand. Interestingly, this time there is a more consistent pattern of improvement across all topics and issues regarding the marketing of organic grains. This pattern suggests that participants in the field day have acquired a more comprehensive understanding on the variety of topics and issues involved in organic grains production and marketing.
In summation, the evaluation of the field days suggests that the training program has been highly successful. The curriculum proves to be relevant and responding to the most pressing information needs of extension educators. The quality and format of the field days were highly evaluated by participants, and the self-reported increase in knowledge and understanding suggests that the field days also appear to have been highly effective as educational tools.
SARE Follow-up survey, January 2002
Results from the surveys provide very limited information on the long-term effects of the training. While a reaction assessment can capture basic impacts, such as the amount of learning that occurred, it cannot attest to behavior change. Nor can it register how much of the newly acquired knowledge actually stays with the participant over time, or how well it can be and has been applied in the field. To strengthen the evaluation design we added an impact evaluation component. A six-month follow-up survey was mailed to all seventeen participants in the April 2001 field day.
The first module of the questionnaire estimated the general value of the SARE training activities and asked participants to rate relevance, the focus on most needed information, and overall usefulness of the activities. All aspects of the training activities consistently received above average marks from the participating extension agents.
The second module assessed how participants evaluated the quality of the first SARE field day. Respondents were in agreement that the field day was relevant to the overall institutional needs of their respective offices and to their needs as educators. This pattern clearly suggests that the field days provided an educational and informational resource that is in demand and responds to the objective needs of extension agents and farmers.
Extension agents also highly value the contribution the field day has made to improve their appreciation of the importance and complexity of organic grains production. This result speaks to the effectiveness of the field day as an educational tool to increase participants’ confidence in their ability to help farmers undertake the transition to organic. Overall, extension agents consider the field day to be an important factor in providing tools and resources for their work of guiding and advising transitional farmers.
Results from the follow-up survey also suggest that the project has made a contribution not only in increasing extension agents' level of knowledge, but, even more importantly, in changing their behavior. This is shown by the 63% of the respondents who report using or sharing the videos they received as part of the training activities in their professional work. Furthermore, over 87% of the extension agents indicate that they have continued to seek information on organic agriculture after the field day, and that the field day has improved their overall ability and effectiveness in responding to their clients’ informational needs. Thus, the SARE program has both promoted further educational interest and has made an important contribution to the quality of the current educational work of the extension personnel. About 80% of the participating agents indicate that during the six months after the field day, they have already recommended to interested farmers some of the new practices learned during the field day An overwhelming majority of 94% indicate that they are very interested in participating in similar educational and training activities in the future.
The successes of this project are already being felt across the region, with the sharing of important information on organic grain production. The data we have collected through the field day surveys strongly indicates that the field day attendees benefited from the information they gleaned, and they have already passed much of the information and resource material along to their constituents. The format that was chosen to deliver the information and develop the handbook was extremely successful.
Based on this success and on feedback received from the extension agents and specialists attending the field days, the following issues have been identified as future professional development needs: (1) similar educational programs related to other categories of crops, like vegetables, fruits, cut flowers, and specialty crops; (2) continuing updates on the new national organic standards; (3) on an annual basis, exchanges between local certifying agencies and extension staff to learn about the challenges faced by farmers and discuss ways to resolve them.
Those who participated in the educational activities genuinely embraced the issues surrounding organic agriculture and the positive effects it can have on our region. It is our hope that we can build on this excitement, and that future support for expanding this model to include these topics will be strongly considered.