Progress report for GNE21-253
- Track AA Fellow changes: Track and describe how AA Fellows integrate climate adaptation concepts into their programming over the two years post program completion. Record actions they take both as a direct result of participation in CAF, as well as actions they would have undertaken without CAF. (This will allow me to estimate CAF's additionality, or the proportion of climate-related activities that can be attributed to CAF participation.)
- Track Farmer Fellow changes: Measure the progression of Farmer Fellows’ climate adaptation knowledge, confidence and further outreach efforts to peers over the two years post program completion. (Record changes due to CAF and not due to CAF to estimate CAF’s additionality, as in Obj.1.)
- Track on-farm changes: Observe and document Farmer Fellows’ adoption and use of CAF-inspired climate adaptation or mitigation practices on their farms, with particular attention to changes in practice use (tweaking), mal-adoption, or abandonment of practices.
- Map outreach and knowledge spread: Map and describe how Fellows’ outreach efforts reached and influenced additional contacts, by following the spread of concepts and practices through farmer-advisor networks. Fellows are required to do 3-5 outreach activities to other farmers as part of the CAF program. These activities can be in the form of writing (e.g. newsletter articles, blog posts), presentations to groups/conferences, or hosting farm visits. These will be documented during CAF and then integrated into network maps as part of this proposal. Fellows may also choose to do further ad-hoc outreach after the official CAF program period. These efforts will also be tracked and added to the network maps over the proposal period.)
- Evaluate learning model: Using CAF as a case study, evaluate the effectiveness of peer-to-peer learning models for advisors and growers engaged in sustainable management in the Northeast, with attention to its influence on knowledge spread and practice adoption, as above, and recommendations for future programming.
The purpose of this project is to understand the degree to which peer-to-peer programming influences communities of growers and agricultural advisors (AAs), specifically those actively engaged in sustainable management. While farmer-focused professional development training often includes a peer-to-peer element, and research suggests that farmers favor learning from other farmers, efforts to document the medium- and long-term effects of this mode of education are sparse. This is often due to funding and time constraints, with many evaluation efforts limited to assessing short-term changes in knowledge and behavior change intentions. What is missed is whether these changes translate into sustainable practice adoption, and whether these practices are employed continually, modified, or dropped in particular farm contexts. By not exploring the longer term outcomes, we lose the opportunity to refine our approaches, and diminish our capacity to support sustainable agriculture systems on a broader scale.
To address this need, I propose using a SARE-PDP funded project, ‘The Northeast Climate Adaptation Fellowship: Vegetable and Small Fruit Program’ (CAF), as the context of my study. Farmers in the Northeast are facing a wide range of challenges caused by climate change, including extreme rainfall and drought, new pest pressures, and greater risks to their operations. These are predicted to increase in the coming decades, and farmers will therefore need support to match this growing challenge – to build capacity for climate adaptation planning and management. This capacity will be crucial for the achievement of NE SARE’s vision of a diversified, profitable agriculture in which farmers steward resources sustainably.
The CAF program is taking an important step towards addressing this challenge – pairing 21 fruit and vegetable growers in the Northeast with 22 professional AAs and facilitating a 12-month partnership. Pairs are required to conduct an on-farm risk assessment, a financial analysis of an adaptation or mitigation practice, and do 3-5 outreach activities. The verification plan of the CAF program runs through January 2022, culminating in the submission of final reports from the Fellows and a post-program survey. Specifically, the approved verification plan aims to capture a) changes in knowledge, b) application of knowledge, and c) confidence working on climate-related topics - all compared to the baseline data collected one year prior.
The currently funded work will give an initial snapshot of the immediate changes resulting from the CAF program. However, it does not allow for any longer-term tracking of outcomes after the program ends. To accomplish this, I propose tracking the flow of knowledge and ideas, and the use of new practices at farm-level, for a two-year period after CAF program completion. This should highlight the benefits and shortcomings of this type of education program, and show how climate-adaptation information and motivations do or do not “ripple out” beyond program participants into the broader community of Northeast farmers and AAs. I am particularly interested in the additionality of the CAF program, meaning I wish to assess whether changes are made because of the program, or if farmers would have made these changes without CAF.
- (Educator and Researcher)
- (Educator and Researcher)
This research uses the CAF program, including the pre-existing CAF verification plan, as a case study to answer bigger questions about peer-to-peer learning in farmer-advisor networks. The CAF verification approach captures changes in Fellows knowledge about climate change, confidence in integrating climate change into their work, as well as Fellows’ intentions for applying knowledge gained through the program in their future farm management or technical assistance programs. My research proposal extends the established evaluation period by 23 months (extending the total assessment period to 3-years) and adds significant rigor. This will lead to a deeper and more meaningful assessment of peer-to-peer, network based outreach education.
With the additional mixed-methods research in this proposal, it will be possible to move beyond looking at intentions and observe individual’s behavior. I can also explore how generic adaptations (e.g. use more cover crops) are tailored to specific farm contexts, and by doing so improve our understanding of the potential benefits and limitations of these practices.
To meet each objective, I propose using the following methods:
Data Collection Methods:
- In-depth review of CAF materials (Objectives 1 - 5)
Some of these materials (bullets a-e, below) will be reviewed as part of the CAF verification plan. I will use the output of this review to serve as both a baseline to evaluate later change, as well as guide farm visits, in-depth interviews and follow-up surveys. Items described in bullets f,g, (below) are not included in the original CAF verification plan, but are summarized here to describe the baseline my research will build upon.
Baseline CAF verification materials:
a. Fellow’s application survey (July - Sept 2020) - Current farm practices, where the farmer gets his/her information or advice, how often they already talk about climate change related topics, their climate change beliefs, and perceived farm risks.
b. Post-workshop evaluations (January 2021 and January 2022) - The CAF program is bookended by two all-participant workshops in January 2021 and January 2022. Evaluations include responses from Fellows on climate change knowledge/confidence, changes in knowledge, evaluation of the workshop content, intention to make changes on the farm, and requests for future content.
c. Fellows’ work plans (submitted Feb 2022) - Including an on-farm risk assessment, financial analysis of an adaptation or mitigation practice, outreach activities, and other optional activities chosen by Fellows.
d. Outreach Log: (April 2021 - January 2022) - Documentation of outreach completed during the program time period, including mode of outreach, number of other farmers reached, and topics.
e. Fellow’s final reports (Jan 2022) - Results of on-farm assessment and analysis, farm trials, outreach products, and new contacts reached. Report template also asks Fellows to share their intentions for applying knowledge gained through the program in their future farm management or technical assistance programs.
Additional materials to be reviewed through this project:
f. Workshop notes and discussion logs (January 2021 and January 2022) - Agendas, personal notes taken during the workshops, chat logs, as well as CAF listserv archives.
g. Educator meetings notes (December 2020-January 2022) The Educator team is a group of 7 AAs, most of whom were involved in the development of the CAF curriculum. This group includes 5 University staff, a representative from the Maine Organic Farmers and Gardeners Association, and the USDA NE Climate Hub, who inform workshop design and support Fellow pairs through the 12-month process.
The analysis of the items described above will provide descriptive information on the Fellows and their experiences in the program as they stand in January 2022 - including differences within the group such as the magnitude of change in knowledge or confidence, future intentions, and satisfaction with the program. The review and resulting baseline will allow me to track how Fellows progress over the following 2-years and identify any diverange based on these results.
2) Cross-sectional surveys: (Objectives 1 - 5)
I will design and send out a survey to all CAF Fellows to capture changes in outcomes. This survey will be sent twice: 1 year post CAF (January 2023) and 2 years post CAF (January 2024).
These follow-up surveys will include some of the same questions as the three CAF surveys included in the established verification plan (application period, post-January 2021 workshop, post-final January 2022 workshop) so that I can follow changes in knowledge and perception over this entire period. I will also add new questions related to the Fellows’ experiences with their farm practices and any further outreach they have done.
3) In-depth semi-structured interviews: (Objectives 1 - 5)
I plan to conduct in-depth interviews with 15 Fellows in the Fall of 2022 and Fall of 2023. Through these conversations, I will document the following:
- Longer-term perception of educational programing (all Fellows): I will capture Fellow’s perceptions of the CAF program as it fits in with their other professional development experiences, after they have had time to see the results of new practices on their farms (Farmer Fellows) or results of their new programing (AA Fellows). I will:
- Document their experience with other professional development programs, and compare their experiences with CAF to other peer-to-peer education, advising or networking activities.
- Ask which specific aspects of the CAF educational programming have provided lasting changes for them (CAF curriculum, 1-1 support through AA-farmer pairing, CoP, and outreach).
- Probe for deeper explanations to these responses to add to the results from the cross-sectional surveys and triangulate findings.
- On-farm changes (Farmer Fellows): I will ask for information about any CAF-attributed farm trials, demonstrations or changes on their farm. I will document what they did, the results of their efforts, lessons they learned from the experience, and their plans for the future.
- Programming changes (AA Fellows): I will ask for information about any new information or concepts they have integrated into their programming. I will document what they added to existing programs or if they established new programs, the results of these changes, what they learned, and their plans for the future.
- Communication changes (all Fellows): I will ask Fellows to report on outreach activities they did because of CAF. Specifically, I will seek to understand if their communication about climate change and agriculture has changed because of CAF. I will ask about who they discuss climate-related topics with, what they talk about, and whether they have noticed any colleagues changing their practices or programming, perhaps because of their interactions.
4) Farm visits and case studies (in-person or video if necessary): (Objectives 2 -5)
Farm visits and in-depth conversations with four farmers will be used to further document changes the Farmer Fellows have made to their farming practices, and when possible, the results of those changes. Discussions at the farm-level will also provide context, including attributes of the environment and climate of the farm, the full staff team, and their specific risks and challenges. Visits can give a valuable visual on the crops and chosen management practices. I will document these visits using notes, photography, and video, with the farmers’ permission.
It will not be possible to visit every Farmer Fellow, and so I propose choosing a subset of farmers to visit and highlight. These case studies will include a deeper look into the story of a specific farm and farmer(s), and how the CAF program has impacted their farm management and communication related to climate change. Farmers will be selected with an aim to highlight diverse experiences - looking for a mix in geographic location, type of farm, gender/age/experience farming, as well as reported experience with the CAF program.
5) Collection of Fellows’ feedback: I will report my findings to the CAF Fellows in March of 2023 and March 2024, and gather their feedback as a form of data triangulation. Their responses will be used in 2023 to adapt remaining research methods as appropriate (survey and interview questions), and in 2024 to inform conclusions.
1) Outreach tracking and network mapping: (Objectives 4 and 5)
One of the key attributes of the CAF program is the requirement of Fellows to share what they have learned with their peers and the broader agricultural community. This can be in the form of in-person farm visits or group conversations, conference presentations, or written contributions to newsletters, blogs, etc. Each pair of Fellows are required to complete 3-5 outreach activities as part of the CAF program, and I anticipate that many may continue to share their learning after the official end of that program. I will track and map the outreach of Fellows over a 3-year period, starting in February 2021, through February 2024. I will track the mode of outreach, topic, and audience reached to create a series of maps of the spread of CAF-attributed knowledge and ideas.
Through the proposal period, I will track:
- In-person and other informal outreach conducted by the Fellows: I will utilize all contact points with Fellows (farm visits, in-depth interviews and surveys) to ask about any informal outreach that Fellows can recall and who they reached (to the best of their knowledge). I will follow-up by phone or email with as many of those new contacts as possible, to ask about how the information shared was perceived or used.
- Written, published and formal presentation outreach: I will attempt to follow available data such as website hits, social media views, newsletter or journal readership counts, and audience attendance at conferences. This will require coordination with publishers, conference organizers, and website managers. I will follow-up by phone or email with readers or audience members when possible, to ask about how the information shared was perceived or used.
Through this outreach mapping, I will record key descriptive attributes of the connections, as supported in the literature on Social Network Analysis (SNA) (Knoke and Yang 2020): a) frequency of outreach, b) duration (before or after CAF, length of contact period), and c) substance of outreach. This will allow me to create a series of ‘ego-centric’ network maps, showing all of the connections made by a Fellow related to CAF ideas. These maps can be made to show: a) network change - pre-existing and new connections of each Fellow, b) temporal change - how networks evolved and ideas spread over time, and c) spatial change - how connections evolve over a geographic area. Using this established approach, I will look at key characteristics of these new networks, including network density, centrality, presence of subgroups, homophily and bonding and bridging ties (Bourne at al. 2017). Examples of network maps created using SNA are included in the supporting documents as examples.
2) Data Analysis (Objectives 1 - 5)
Document review will be conducted through NVivo, a widely used qualitative coding software. Working with my advisor, I will develop a code book based on the project research questions. Themes from this review will be used to triangulate data collected in surveys and interviews (see below) and will be described in a narrative report.
Survey data will be analyzed using R (for quantitative results) and NVivo (for open-ended qualitative results). In addition to running descriptive statistics, I anticipate using R to do standard nonparametric statistical tests (e.g. Kruskal-Wallis H and related tests) to compare different groups of survey respondents (farmers and AAs) and how membership in these groups affects dependent variables described above (i.e. confidence in making decisions with climate change in mind, intentions to make changes on their farms or in programming, and the actual changes implemented). I will also use R to run SNA, using one of a variety of packages available (igraph, sna, or networks). These packages will allow me to assess network size, connectivity, and centrality. They also will allow me to create informative data visualizations.
Interviews will be transcribed and analyzed using NVivo. I will again work with my advisor to develop a qualitative codebook based on the project interview guide, assign codes, and assess pre-identified and emergent themes. With the assistance of an undergraduate research assistant, I will use a double coder approach for this portion of the analysis, an established method for reducing bias in qualitative analysis.
Research will commence in spring 2022. There are no results to report at this time.
Education & Outreach Activities and Participation Summary
The outreach plan for this project includes a mix of sharing and collecting feedback with the CAF Fellows, as well as formal and informal sharing with the broader agricultural advisory and research community in the Northeast and beyond. There are two primary audiences for my research findings:
First, I will report my findings to the CAF Fellows twice: in March of 2023, and at the end of research in March of 2024. In both of these cases, I will not only share, but also gather their feedback as a form of data triangulation, and to be able to use their insight to revise and improve upon the research methods and conclusions. This type of participatory evaluation increases accountability of the research team and can also lead to richer and more-user friendly findings (Jacobs et al. 2010). I anticipate that Fellows will be interested in the results of my research based on their feedback on a climate communication session that was part of the January 2021 CAF workshop. Specifically, more than 80% of Fellows reported that they would be interested in learning more about how to communicate with (other) farmers about climate change in future workshops. As one Fellow stated: “I feel like I always need strategies for addressing how to communicate these topics with diverse parties and this [session] was a great addition to my learning.”
Second, I will share the results of my work with agricultural advisors and researchers across the Northeast region. I will do so by preparing a research brief for the USDA Northeast Climate Hub University Partner network, give a presentation to the Maine Agriculture and Climate Network, and publish at least one peer-reviewed journal article in open access format. Target journals for a peer-reviewed paper include Global Environmental Change, PloS one, and Ecology and Society. In addition to this, I will put out shorter and less formal updates online, including one or more blog posts on Dr. Schattman’s Agroecology Lab website, and seek out other platforms to share blog posts and/or farmer case studies.
No outcomes to report at this time. Research will commence in spring 2022.
Research tools have been designed, and IRB approval will be sought in January 2022. Research will commence in spring 2022.
My future research will focus on documenting the post-program impact of the Climate Adaptation Fellowship, and what this can tell us more broadly about farmer and agricultural advisor learning and social networks. My research will begin in the spring of 2022, and continue through 2024. I plan to use my PhD research experience to explore next career steps after completing my degree. I hope to continue my career combining research and practice in the areas of agricultural education and extension to support farmers in climate adaptation and mitigation.
No assessment to report at this time. Research will commence in spring 2022.