Learning how to use communities of practice to address sustainable agriculture issues

Final Report for ENC08-101

Project Type: Professional Development Program
Funds awarded in 2008: $65,958.00
Projected End Date: 12/31/2010
Region: North Central
State: Iowa
Project Coordinator:
Richard Pirog
Leopold Center for Sustainable Agriculture
Co-Coordinators:
Beth Larabee
Leopold Center for Sustainable Agriculture
Expand All

Project Information

Abstract:

A Community of Practice (CoP) building curriculum was developed and made available on the web. Two workshops (one for the north-central region and a pilot for Iowa) were held in 2009 reaching more than 100 educators. Four follow-up calls on different aspects of building CoPs were completed in fall/winter 2009-2010 following the regional workshop. Pre- and post-evaluations were completed for both the training sessions. Changes in attitudes and knowledge gained were significant in the regional workshop; the Iowa impact results were adversely influenced by ISU Extension staffing changes announced the day of the workshop. Experiences of the facilitators and discussants and dialogue and interaction with fellow participants were critical for helping them understand the structure and function of the communities of practice and options for creating new ones. In 2010 the Leopold Center held a workshop at the request Iowa State University (ISU) Extension for more than 60 ISU Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center’s effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before. A follow up workshop was held on April of 2011 for the SARE PDP participants who attended the first workshop. This session focused on participants sharing best practices as their new CoPs develop.

Thirty-six percent reported they wanted to learn how to foster collaboration and how work together with other groups and network with their peers who were involved in CoPs. Project participants and local partners have been able to leverage over $300,000 to support their work in sustainable agriculture using the Community of Practice model. As a result, organizations, agencies, institutions, and businesses involved in this work say they are reaching new audiences and serving those audiences more effectively. As we concluded at the end of our first year evaluation report, over time, we hope to see more impacts trickle down to farmers and consumers through increased opportunities to create and consume farm products and services that contribute to the triple bottom line goals of creating environmental, social, and financial good. We’ve seen some of that already happening in Year Two and expect SARE and our other partners in this effort will see more of it in years to come.

Project Objectives:

This Value Chain Partnerships (VCP) project proposed to: (1) develop Community of Practice (CoP) curriculum and workshop materials; (2) pilot test CoP Workshop training and materials prior to its formal launch with 20-24 Iowa State University (ISU) Extension county and field staff participating; (3) provide the CoP Workshop in Year 1 to 48 Extension, agency, and/or non-profit educators and researchers in the north central states to address challenges in sustainable agriculture at a two-day workshop; (4) allow participants to participate in a community of practice meeting; (5) share all training materials on Communities of Practice on the web; (6) provide regular conference calls and creation of a listserv to provide networking and problem solving support; (7) develop a workshop in Year 2 where participants can share best practices as their new CoPs develop and evolve; (8) workshop materials and short presentations also will be shared on the web; (9) train an additional 120 Extension, agency, and/or non-profit educators and researchers in the north central states; and (10) evaluate activities using project surveys to measure short-term learning outcomes after each workshop in both Years 1 and 2. Year 2 evaluation activities will rely on face-to-face and/or telephone interviews to measure medium-term impacts of this project in terms of changes made by participants and representative organizations/agencies as a result of their participation.

Education & Outreach Initiatives

Objective:
Description:

Methods

Accomplishments/Milestones

VCP completed a written resource guide and workshop curriculum on creating, leading and managing CoPs. The Community of Practice Workbook was written in April 2009, revised in July 2009 and again in March 2010. The workbook is available at http://www.valuechains.org/files/resources/copresourceguide.pdf. A pilot workshop was held on April 30, 2009 with 38 Iowa State University Extension staff participating. The workshop was held the day that significant extension restructuring and job cuts were announced, which greatly reduced participants’ ability to focus on the workshop.

The SARE PDP Workshop Session was held July 21-22 with 43 NCR SARE scholars, plus 20 North Central Region colleagues and 13 NCR SARE speakers and partners. This exceeded by 19 the expected attendance of 24 SARE Educators. Educators from California, Oregon, Nevada, Washington, D.C., Virginia and Maine also attended. All of the NCR SARE scholars participated in one of three communities of practice meetings: Grass-based Livestock, Pork Niche Market Working Group, or Regional Food Systems Working Group. Over the past 12 months, we also have provided URLs, CoP workbooks and other information to at least 10 interested parties and added 14 members to the Community of Practice Ning networking site.

Web access to the CoP Curriculum has been accomplished by developing an on-line social networking site, http://communitiesofpractice.ning.com/. Participants have had access to the Community of Practice social networking site since June 2009. This site currently has 88 members from Illinois, Indiana, Iowa, Kansas, Maryland, Michigan, Minnesota, Missouri, Nebraska, New York, North Dakota, Ohio, Oregon, South Dakota, Virginia, Vermont and Wisconsin, and houses all the materials developed for the workshops plus a calendar, information on funding opportunities, additional related materials/activities as well as a blog.

Four calls were held in 2009 for the SARE PDP educators and other participants by the VCP workshop team. On September 8, 2009 the call was led by Rich Pirog who explored what topics should be covered during subsequent calls (13 participants, six team members). The second call took place on October 20, 2009 when Gary Huber led the discussion on initiating communities of practice (11 participants, seven team members). On November 10, 2009 the topic was funding communities of practice led by Andy Larson (9 participants, six team members). On December 8, 2009 the branding communities of practice discussion was led by consultant Sue Honkamp (seven participants, seven team members)

Budgeting for a large number of extra participants for the July workshop was challenging for the organizers given the reduced financial circumstances at Iowa State University and the Leopold Center for Sustainable Agriculture, but the opportunity to offer this workshop to so many interested people could not be passed up. Iowa SARE PDP funds and supplemental funding from a USDA NRI grant were used to cover the cost of extra participants and additional transaction cost (Table 1).

The social networking site continued to see moderate traffic and additional people joined the network. The winter workshop was planned, reconvening the SARE-PDP participants, with an agenda focused on lessons learned from using community of practice approaches. We requested the following re-budget to best use remaining resources in Year 2 (Table 2).

Catering costs for the workshop were higher than expected with an increased number of NCR-SARE PDP participants (43 instead of the 24 anticipated) resulting in an increase of $1,668 in Supplies and Materials. Cost of travel reimbursement was less than expected for the NCR-SARE PDP participants supported by the grant (decrease of $2,168). Participant feedback indicates that a speaker who is excelling in a sustainable agriculture-focused community of practice would be valuable for the second convening of the NCR-SARE PDP grant supported participants. That requires an increase in funds to procure a high-quality speaker ($500 increase in Other Direct Costs). The amount of $4,620 was posted in the wrong line item (Supplies/Materials) and will be moved to Other Direct Costs

Leopold Center’s close relationship with Extension led to a special workshop for ISU Extension staff which was not within the scope of the full proposal to SARE. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor were participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. This session allowed ISU Extension staff to focus on the content and invite community partners to further disseminate the Community of Practice concept. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented non-profit social service organizations and the business community. The Value Chains Partnership website (www.valuechains.org) continued to provide information on many communities of practice and their partner organizations and their services and activities. Funds for this workshop were provided by Iowa State University Extension and the Leopold Center for Sustainable Agriculture.

A follow-up workshop where participants could share best practices as their new CoPs develop and evolve was created and held in April, 2011. The morning session focused on what the participants were curious about after their experiences with CoPs. The afternoon session focused on challenges experienced by CoPs and group problem solving. Funds were rebudgeted (see Table 3)to strengthen evaluation of the grant and adjust for increased facilities/food costs. Seventeen returning SARE staff participated in the follow up CoP workshop. Travel honoraria were awarded to thirteen of the returning NCR SARE personnel. A number of Iowans were also invited to attend resulting in 16 Iowa participants including the facilitation team. Travel for five Iowa participants was paid with Iowa SARE funds.

Justification for Rebudget
$453 from Travel Domestic and $5616 from Other Direct Costs were reallocated $1182 to Salary/Hourly and $2581 to Payroll Benefit and $2306 to Supplies/Materials for meals. Please note meals are part of the days training and were approved with the original grant.

Remaining funds will be returned to SARE, See Table 4.

Outreach and Publications

Priog, R and Bregendahl, C. Creating Change in the Food System: The Role of Regional Food Networks in Iowa. MSU Center for Regional Food Systems. March 2012.

Outcomes and impacts:
INTRODUCTION

Communities of Practice (CoPs) or working groups for Extension, agency, and non-profit educators and researchers in the north central states. The workshop was part of the Leopold Center’s two-year NCR SARE PDP project to increase capacity in the north central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. A total of 63 participants engaged in the regional workshop, formally referred to as the Community of Practice Workshop. This workshop was preceded by a pilot workshop held in late April for Extension staff in Iowa to sample the methods and materials.

To evaluate the impact these workshops had on attendees, we developed two survey instruments: an electronic pre-test survey and a paper post-test survey. The purpose of these surveys was to: 1) prepare participants to think about issues addressed by the workshop prior to their arrival; 2) gauge their interest in particular types of working groups to help the training team design workshop sessions; 3) measure benchmark knowledge, attitudes, and experiences we could compare to post-test workshop survey results; and 4) measure medium-term changes in behavior that attendees might report at the end of the two-year project.

On April 19, 2010, the Leopold Center held a workshop for Iowa State University Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center’s effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. The project is supported by the North Central Sustainable Agriculture Research and Education Professional Development Program. Sixty people attended this event, which followed a regional workshop attended by 63 partners from the North Central Region in July, 2009.

In April 2011, the Leopold Center held a follow up workshop for sixteen returning NCR SARE PDP participants. In addition, 16 Iowans were invited to participate. Travel funds for 6 Iowans were paid by Iowa SARE funds. Panelists for the workshop were; Rich Pirog (Leopold Center) – Regional Food Systems Working Group; Madeline Schultz (ISU Extension), Cooperatives Community of Practice; Sarai Rice (Des Moines Area Religious Council), The Alban Institute; Eric DeLuca, National, Cooperative Business Association; Jim Dyer, Southwest Marketing Network; Teresa Engel, Wisconsin Local Food Network

EVALUATION RESULTS FROM THE PILOT WORKSHOP

The pilot Community of Practice workshop for Extension was the first workshop delivered. It was created and delivered to:

• Test pilot workshop materials and workshop community of practice curriculum prior to the July meeting with representatives from the North Central Region;
• Explain the purpose, function, and value of using working groups to address key issues that are part of ISU Extension’s work plan; and
• Use Iowa-based examples and lessons learned to share and discuss principles and considerations about initiating, managing, funding, and branding communities of practice as a process tool in ISU Extension programming.

The Leopold Center initially aimed to involve a maximum of 24 people at the pilot workshop on April 30, 2009. However, requests continued to stream in beyond the intended cap. Ultimately, 43 attended. We asked registrants to complete a pre-test survey electronically; 38 of which were completed, for a response rate of 88 percent.

Timing is critical, however. The day before the April 30 event, ISU Extension administrators informed the SARE project leader that on the day of the workshop, Extension would announce its new plan to restructure in response to severe budget cuts. The news was grim as positions would be eliminated. Despite this, administrators advised the SARE project leader to continue with the CoP training and asked to be added to the agenda so they could directly address attendees about the reorganization. They warned that some participants in attendance might not choose to stay for the training after hearing about the restructuring plan. Of the 43 attending, approximately seven did leave throughout the course of the day. From the 36 remaining, we received 26 useable post-workshop surveys, for a 72 percent response rate. The following tables and paragraphs briefly summarize what we learned from them.

Table 5 shows a breakdown of the position and program area affiliations of responding Extension staff participating in the pilot workshop, as well as basic demographic characteristics. The reduction of County Extension Directors (CED) from 47.1 percent to 30.4 percent in the pre- to the post-test responses is likely the result of the Extension announcement the day of the workshop, in which all County Extension Directors in Iowa were given notice they would be losing their jobs in the reorganization. Many Extension Directors did not come as planned or left the workshop in progress once the announcement had been made. Field specialists were deemed &”safe” and, therefore, we did not see any kind of attrition in this group based on numbers from the survey results.

Table 5 also shows the program area expertise of Extension participants. (Respondents could choose more than one area.) Extension staff working in the areas of Agriculture and Natural Resources and Families comprised the majority of respondents, creating a roughly 3 to 1 ratio of Families or ANR staff to CED or 4-H personnel.

Table 6 compares results from the pilot pre- and post-test questions that queried participants on the extent to which they agreed with 21 statements about their attitudes and knowledge of CoP-related issues. The table shows the pre-test survey mean of their responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. The t-statistics and p-values also are listed.

For every statement, the post-test mean increased once the workshop had been conducted, albeit only slightly for some. Yet, not all increases are statistically significant. Seven were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop

Interestingly, Table 6 shows that the workshop had no effect on whether Extension personnel were interested in forming a CoP. These results differ from the July workshop results with representatives from the region (described later), which show a statistically significant (positive) change in participants’ willingness to form a CoP after they had received the training. Also worthwhile to compare with the regional results is the lack of change in attitudes after the training among Extension personnel regarding the capacity of their organization to organize and lead the creation and facilitation of a CoP. Participants in the pilot training experienced no change in attitude regarding this point after attending the workshop (where p=.866), whereas the July workshop attendees were more likely to agree that their organization had the capacity once they had been through the training (p=.018). The day’s announcement to reorganize Extension and cut 100 County Extension Directors from the payroll almost certainly had an effect on these results, and most likely had an immediate impact on participants’ attitudes about Extension having the capacity to function in the future.

Questions related to participants’ attitudes and activities regarding partnerships, while not statistically significant above in points r), s), and t), reveal that Extension already is looking outside itself to connect with other people and resources. The majority (87 percent) of respondents to the pre-test survey responded affirmatively when asked whether they had tried to leverage resources in the past two years in partnership with another organization, group or agency to further their work. Among the partners listed were local farmers; local businesses; Chambers of Commerce; local elected officials; county health officials; churches; food pantries; County Conservation boards; Farm Bureau; banks; insurance companies; community colleges; producer associations; City Parks and Recreation departments; local hospitals; schools; rural advocacy groups, such as the Center for Rural Affairs; state agencies, including IDALS and DNR, Workforce Development, and the Department of Health and Human Services; non- profits such as Resource Conservation and Development; councils/agencies, Practical Farmers of Iowa, the Iowa Network for Community Agriculture; and low-income family service providers, such as Iowa Community Action agencies; and University-based partners including Extension and the Leopold Center. The range represented in this list of organizations, agencies, and institutions suggests Extension is already striving to make better use of local community resources.

The notion of doing better outreach or some variation on that theme was listed by one in three participants as one of the most significant things learned in the workshop. “[We need to] spend more time bringing folks together,” said one participant. Another remarked he had learned that “CoPs are a good way to bring diverse groups with common goals together to work towards those goals.” Still others mentioned that the most significant thing they learned was the need to involve actual clients in working groups and how important working groups are as a means for “reaching and relating to clientele.” This was the reason some Extension staff chose to attend: “I thought I could learn more about building collaboration and sponsorship [for Extension work].”

Clearly, the notion of creating CoPs or working groups as a strategy for better engaging clients and communities in the work of Extension appealed to workshop participants. However, many were concerned about the availability of Extension resources to support such an approach. One participant prefaced his comments with, “If funding is available…” and another finished her comments with, “…if I am still involved with Extension.” Although there were positive evaluation comments about the workshop and content delivered, there also were some tepid reactions, which one participant chalked up to “the context of the budget cut information [presented by Extension administrators that] affected us all and our participation. It was a distraction we couldn’t control.” While some found it “difficult to engage in light of the news about Extension,” others found value in the training and materials. “The resource workbook will be a great tool. Hearing from the various CoP working groups and having their examples was very helpful.” For others, the workshop articulated “[what] I think is new language for what I have [already] done in communities in the past.”

Evaluation Results From the First Regional Workshop

After modifying the design and format of the regional workshop based on experience and feedback from the pilot, the regional workshop was set for July 21-22 in Des Moines. At the time we first surveyed the regional workshop participants, 76 were signed up. Of those participants, 52 provided useable pre-test surveys, for a 68 percent response rate. We administered the pre-test survey electronically using Survey Monkey, an on-line survey tool. Of the 63 who actually attended the workshop, 51 completed a useable post-test survey, for a response rate of 81 percent. The post-test survey was administered on-site using paper copies included with the workshop handouts and materials. However, not all participants answered all questions all of the time. Therefore, tables throughout this report list the number of respondents (in parentheses) who answered each question.

About the Respondents

Table 7 shows that most respondents are women, particularly those responding to the post-test survey. This closely aligns with the gender of the attendees, which was 63 percent female, based on a review of names in the registration lists. However, men also were a strong contingent at the workshop, as were participants from several different age groups. Table 8 shows the average attendee age is 48 but the range spanned six decades, showing a mix of interest from nearly all age groups.

Table 9 shows the occupational affiliations of participants, who were invited to choose multiple categories if applicable. Non-profit representation was the strongest by far, comprising one-third of workshop attendees. More than one in four attendees represented Cooperative Extension, while government employees and farm-related business each totaled nearly 14 percent of those present.

Table 10 shows where workshop attendees live and/or work. The majority are from Iowa, followed by Illinois, Minnesota, Nebraska, and Michigan. And, although we have no pre- or post-test results from Kansas and no post-test results from North Dakota, one representative from each state did indeed attend the workshop, so that all 12 states in the North Central Region were represented.

Impact on Regional Attitudes and Knowledge

As we did for the ISU Extension pilot, in both the pre- and post-test surveys of regional representatives we asked participants to indicate the extent to which they agreed with 21 statements about their attitudes and knowledge of CoP-related issues. Table 11 shows the pre-test survey mean of their responses on a five-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. We assumed normality of results given our sample size was over 100 (103) and therefore used parametric statistics (t-tests of means) to determine whether any rise or drop was statistically significant. The t-statistics and p-values also are shown in Table 11.

For each of the 21 statements, the post-test mean increased once the workshop training had been delivered. For all but three statements, results were statistically significant where p<.05, meaning that the increase in the means purely due to chance and/or error is so small that we can attribute the change to the workshop. To interpret this in practical terms, use statement c) as an example. Here, the chance that the mean would increase from 3.71 to 4.20 without the workshop intervention is four in 1000, suggesting that the workshop is related to the increase. The lower the p value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different. If the means were equal as hypothesized, there is a 4 in 1000 chance that a more extreme test statistic would be obtained using data from this population.

These results are heartening, and show that knowledge and attitude changes can be attributed to the workshop. Statistics show that the workshop increased participants’ familiarity with the CoP concept, increased awareness about CoP benefits, and increased participants’ attitudes about their viability as a way to help their respective organizations achieve their goals. Participants also believe that CoPs can help their organization work more efficiently and can connect them to people and communities they serve. And while we saw a jump in participants’ interest in forming a CoP as a result of the workshop, we did not see a significant increase in their interest to simply participate in a CoP. These results suggest that participants are looking to take on leadership roles in starting CoPs and already had a definite interest in participating in a CoP both before and after the workshop.

The other two statements that did not generate statistically significant results were “I currently partner with other groups in my area of work to advance opportunities related to sustainable agriculture” and “I am willing to work with multiple organizations to create change in the area of sustainable agriculture. “Although the means for both did increase, it was not enough of a bump to give credit to the workshop because participants attending the workshop already partnered with multiple organizations to further their work in sustainable agriculture. Such assertions are evident in responses to the question we posed about whether workshop registrants (pre-test) had tried to leverage resources in partnership with another organization, group, or agency to further work in sustainable agriculture. Nearly all (92.3 percent) said yes (n=52). Workshop participants provided even more details in the post-test survey, citing a mean of 14.4 new groups, organizations, agencies, and businesses they collaborated with in the past year to further their work (n=45), ranging from 0 to 150. The mode (most common response) was six new groups. If we remove the outlier of 150 groups, the mean drops to a still impressive 11.4 groups (where the mode remained the same and the maximum topped out at 41).

Despite these impressive partnering figures, the workshop appeared to strengthen participants’ pre-existing collaborative orientation, as indicated by their affirmative responses to the following questions:

• Has this workshop prompted you to consider using the CoP model to work with groups with whom you have never partnered to further your work in sustainable agriculture? (83.3 percent “yes” n=42) and
• Has this workshop prompted you to consider using the CoP model when working with groups with whom you are already partnering to further your work in sustainable agriculture? (93.3 percent “yes” n=45)

Respondents cited other groups they would now consider working with in a community of practice setting (beyond the “choir” groups such as local food groups and regional produce distributors). More than a handful mentioned “big ag” interests such as Farm Bureau, pork producers’ associations, beef producers’ associations, and corn and soybean growers’ associations as possible collaborators. “I am very inspired to give this a whirl with two groups of ag types I am working with.” Others said they were now considering, but had never partnered with entities such as elected officials, city planners, or economic development agencies, councils, or banks. Segments of the health sector such as hospitals and public health agencies also emerged as potential partners in moving sustainable agriculture work forward collectively.

Regional Workshop Structure and Format

The way in which training is delivered can impact reported outcomes. Evaluating the process and design of knowledge building can provide us with clues about impacts we measure. In light of this, we asked specific questions about characteristics of the workshop and how important each one was in contributing to learning outcomes or connecting participants to resources.

Table 12 shows that for all the workshop structure and format questions asked in the post-test survey, roughly 3 in 4 participants felt each component was important (rating it either a 4 or 5 which have been combined in the far right column). Most notably, results show that 9 in 10 participants feel the experiences of the facilitators and discussants were critical for helping them understand the structure and function of working groups and options for creating new ones. Nearly the same percentage viewed the dialogue and interaction with fellow participants as a contribution to their learning.

Next Steps for Regional Participants

When we asked the regional participants an open-ended question about any actions they were planning to take as a result of attending the workshop, nearly one-third (29 percent) answered the question. That one in three participants was able to articulate concrete ideas about how to use information learned in the workshop immediately after its delivery demonstrates to some degree the relevance of the curriculum as well as the contemplative nature of participants.

Of those who did respond, 20 percent expressed an interest in forming new working groups; however, the majority (47 percent) expressed an interest in using what they learned to reconfigure and optimize the performance of groups with whom they are already working. “I am going to [try to] reconfigure local working groups, share with food system council members, and work more on documentation.” Still others said they were rethinking leadership in terms of how to better engage local and regional interests in their work.

Next Steps for the Project

It is generally expected that changes in knowledge lead to changes in attitudes, both of which are apparent from the post-workshop quantitative survey data. Qualitative data gleaned from open-ended questions confirm this claim. As one respondent remarked, the knowledge she learned in the workshop changed her attitudes about working groups. “I came with no knowledge of working groups. I thought of them more like socialism–now I see them as democratic.” The training appears to have created a more accepting environment for using CoPs as a model for working with varied partners. In addition and as already noted in the discussion on the pilot workshop, the regional workshop gave participants with a common language that provided clarity about the way some participants say they are already working. “[The workshop] was a wonderful idea. It blended learning about the concept of Communities of Practice in a setting where we were immediately able to apply it–perhaps to what we are already doing.” However, there is still work to be done. As some participants noted, they want to see more discussion of “tacit knowledge” associated with working group formation and maintenance and more guidance on trouble shooting problems and finding funding to support working groups.

This is valuable information since the CoP workshop was one of the first several steps over a two-year period to assist Extension, agency and non-profit educators and researchers in the North Central Region in using CoPs or working groups to address issues in sustainable agriculture and food systems. The second step of the process is to hold a series of conference calls to network and support interested workshop participants as they develop CoPs of their own. To prepare for this, we asked workshop participants what they wanted to learn more about. This information will allow us to better target future efforts on items of the greatest and most intense interest (Table 13).The topic that aroused the most interest among participants was funding a working group (87.5 percent agreed they wanted to learn more about funding), followed by branding and conditions for success when selecting a working group.

Given the obvious desire for more learning, we also queried participants about their interest in participating in future conference calls. Eighty-five percent (or 41 out of 48) said they were interested, all of whom were willing to connect on a quarterly basis. One in three was willing to connect by phone on a monthly basis. About half (52 percent; n=44) said they were interested in one or more individual phone consultations with members of the Value Chain Partnerships team. Participants were most interested in phone consultations with the:

• Regional Food Systems Working Group (57 percent )
• Fruit and Vegetable Working Group (22 percent )
• Grass-Based Livestock Working Group (13 percent )
• Branding consultant (9 percent )
• Evaluator or any CoP Facilitator (8 percent )
• Pork Niche Market Working Group (4 percent )

The first conference call was held on August 11, garnering participation from 17 people located around the region and seven members of the SARE CoP training team. Call participants were asked to discuss one priority CoP topic area they wanted the most help with and how they wanted that help to be delivered (e.g., one-on-one consultation with VCP or others in network, group discussion, addition to resource guide, webinar, etc.) The other topic of discussion was use of the social networking site developed for this group, available at http://communitiesofpractice.ning.com/. The Ning site was developed to supplement the face-to-face and phone interactions and provide a forum for those interested in CoPs to communicate in an asynchronous learning environment. The site was developed partially in response to the regional pre-test survey results which show that half (49 percent) of the respondents said they currently participate in electronic social networking sites to inform their work (n=50), the majority of whom (72 percent; n=25) said that these sites were “somewhat valuable” One in four (24 percent) reported these sites were “quite” or “very” valuable. With this in mind, the CoP training team led by the Leopold Center created the above-referenced Ning site to provide participants access to print and social resources to support the creation and maintenance of food and agriculture related CoPs.

Although a special workshop for ISU Extension staff was not within the scope of the full proposal to SARE, the Leopold Center’s close relationship with ISU Extension and discussions with ISU Extension administrators made it a natural outgrowth of the work and mutually beneficial. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor were participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. With the passage of time and additional experience conducting the Regional workshop, the Leopold Center’s CoP training team once again offered interested Extension staff a workshop. To strengthen Extension’s community-based partnerships, the CoP training team issued invitations to Extension staff encouraging them to attend the workshop with a community-based partner with whom they were working or wanted to work. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented nonprofit social service organizations and the business community.

This report documents results of the evaluation of the April 2010 Extension CoP workshop. We developed two survey instruments adapted from instruments used to evaluate the Regional workshop; the first was an electronic pre-test survey designed in Survey Monkey, whose link was e-mailed to registrants about a week prior to their arrival. The second was a paper survey distributed at the end of the workshop. The purpose of the surveys was 1) to prepare participants to think about issues addressed by the workshop before coming and 2) to develop benchmark measures on knowledge, attitudes, and experiences we would use to compare with post-test workshop survey results and longer-term follow up surveys.

Respondents

The response rate to the post-workshop survey was 53 percent (we received 32 completed surveys). Of the 68 who initially registered, 8 did not show up for an attendance count of 60. Of the pre-registrants, 31 completed the pre-workshop survey for a response rate of 46%. For the purpose of this evaluation, we therefore had a total of 63 surveys upon which to base our analysis.

Extension personnel comprised 58% of respondents, many of whom were field specialists (38%), followed by regional extension directors (1 in 4) and state specialists (1 in 4). Not quite half of Extension respondents attending the workshop were based in Agriculture and Natural Resources (46.2%), while one in four (23%) were based in the area of Families and Consumer Science. Communities and economic development was the focus area of 15% of Extension respondents while the same proportion reported specialties in more than one Extension area.

The majority of respondents (67%) attended the workshop with a partner. Most of the non Extension partners attending represented nonprofit social service providers and the business community. Table 1 shows the occupational affiliation of partners (who could choose all affiliations that applied). More than half of the non Extension respondents attending the workshop were affiliated with community-based nonprofit social services (54%) including food banks, substance abuse centers, and various community resource centers. Government employees comprised 46% of non Extension partners attending the workshop. Nearly two in five (39%) are involved in either youth or adult education and 15% were either farmers or entrepreneurs.

Results

Respondents were motivated to attend the workshop mostly for content reasons: to learn about creating, facilitating, managing, and funding CoPs (51%) (Figure 1). However, answers to this open-ended question also yielded more interesting results: 36 percent reported they wanted to learn how to foster collaboration and how work together with other groups—in other words, they wanted to learn how to be good partners and learn about process. Another top reason for coming was to network with others (22%).

For the majority of respondents, the workshop met their expectations (78%) (Figure 2). For one in five, however, the workshop only met their expectations “somewhat” and one said it did not. Among those who selected “somewhat” or “no” responses regarding expectations, most said they still learned something but not all the material was new to them and they wanted more time to spend on it.

We evaluated workshop delivery and mechanics by asking respondents to rate the usefulness of each session using a five-point scale. Results are shown in Table 15. For the most part, respondents found the discussion and definition of CoPs most useful, followed by the small group exercise, which emphasized action steps necessary for focusing, initiating, managing, funding, and evaluating a CoP. The sessions on debriefing and branding were less useful to respondents.

Learning what information workshop participants find most significant is helpful for informing workshop designers and facilitators how well they gauged the needs and interest of their audience and whether the training team and respondents value the same information. To assess this, we included a straight-forward, open-ended question asking respondents to identify the most significant thing they learned. One in four (26.1%) identified as most significant the lesson on the importance of managing the CoP in terms of relationships and communication (Figure 3). The need to articulate common goals and partner with a variety of others with diverse backgrounds and perspectives were also significant lessons reported by nearly two in five respondents (17.4% each). Branding also surfaced, as did the distinction between different degrees of cooperation and the notion of nested networks (13% each). This is much aligned with the major thrust of the workshop, which emphasized the need for effectively engaging partners in collaborative work and maximizing collaborative outcomes.

Armed with new information, we were curious to know what respondents were planning to do with the information after attending the workshop. Although immediately after the workshop is still too early to tell, asking them about their plans provides benchmark data from which to compare information on actual action taken that we gathered in the longer-term follow-up survey, which was administered in December, 2010, eight months.

At the end of the workshop one in three planned to apply the concepts they had learned to existing collaborations to improve their work and partnerships (Figure 4). Fewer (one in four) were planning to initiate discussions with local stakeholders to consider starting new CoPs. Additionally, one in five planned to reach out to include new partners in their work. Although medium and long-term outcomes are virtually impossible to assess immediately after a workshop, administering a similar survey both before and after the workshop does enable us to measure short-term changes in attitudes and knowledge, especially if some questions remain identical in both surveys. This was a strategy we used to test the statistical significance of any short-term change that might have occurred as a result of participation in the workshop. Table 3 compares results from the pre-workshop survey with responses to the surveys distributed after the workshop. In both surveys, the questions were the same. We asked respondents to rate the extent to which they agreed with 23 statements about their attitudes toward and knowledge of CoP related issues. The table shows the pre-test survey mean of responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. T-statistics and p-values are also shown.

For every single statement, the post-test mean increased once the workshop had been conducted. Seventeen or 74% of responses to the 23 statements were statistically significant changes. However, six were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop “intervention” or “treatment.” To interpret this in practical terms, take statement d) for example. Here, the chance that the mean would increase from 3.77 to 4.32 without the workshop intervention is 3 in 1000, strongly suggesting that the workshop is related to the increase. The lower the p-value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different.

The results in Table 16 show that important knowledge and attitude changes are attributable to the workshop. In summary, statistically significant changes among respondents include:

• Greater familiarization with the concept of CoPs and their benefits;
• Increased interest in starting and leading a new CoP;
• Increased interest in joining a CoP (but not necessarily leading);
• Greater sense that CoPs can help Extension (more efficiently) achieve its mission and goals;
• Stronger attitude that CoPs are a viable way to address key issues while better connecting respondents and their organizations to people they serve;
• Better understanding of different ways to fund and need to brand CoPs;
• Greater knowledge about the range of options for setting up a CoP; and
• Greater confidence in respondents’ respective organizations to have the capacity and resources to initiate and facilitate a CoP.

Statements that did not generate statistically significant results are worth examining as well. The mean level of agreement for the statement, “I am interested in applying concepts of the CoP model to work I am already doing” did not increase dramatically after the workshop because respondents generally agreed with this statement to start with (4.23 pre-workshop; 4.52 post-workshop). The same holds true for “I believe the CoP model can inform the work I do” (3.87 pre-workshop; 4.20 post-workshop). Before attending the workshop, respondents were already philosophically dedicated to working with multiple organizations and the need for working with partners from different sectors to address salient issues. Indeed, the role of Extension is to serve as a facilitator and catalyst within the communities and regions where based. And while there was an increase in respondents’ perception that different organizations in their area of work would be willing to actively engage in a CoP to address those issues, the difference was not significant (p=.066).

Workshop participants are clearly a collaboratively minded crew. Results in Table 3 above, as well as other results, provide evidence of this. Figure 5 provides further evidence, particularly as it relates to collaboration between Extension and their workshop partners. All responding partners reported a willingness to partner with Extension in the future (as indicated, additionally, by their mere presence at the workshop) and 91 percent report they have already partnered with Extension in the past. About half are currently partnering with Extension.

Asking participants to attend the workshop with a partner was a deliberate strategy on our part to strengthen existing relationships Extension staff already had in place with partners from their region or community. To this end, the workshop served its purpose. One of the benefits of attending the workshop with a partner was a better appreciation and understanding of partner organizations and resources (Figure 6). One in three workshop participants who attended with a partner also reported they learned how to collaborate better. Perhaps most valuable was the response from one respondent who said they learned they were not ready to play an active role in a Community of Practice yet. Sometimes CoP work is premature and knowing when the time is (or is not) right is just as important as understanding how to make it work.

Despite respondents’ obvious commitment to collaborative processes, there is always room for improvement. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before. Nearly one in five (18.2%) answering this question was considering working with producers, Extension, regional development groups, funders, and health-based groups. When we asked respondents what would be needed for their organization or business to fully participate in CoP work, 60 percent said first and foremost they needed administrative support and permission to spend time on pursuing and nurturing collaborative relationships. The participation of others and the better articulation of organizational goals were also needed (13.3%, respectively).

Evaluation Results from Iowa State University Extension CoP Workshop

On April 19, 2010, the Leopold Center held a workshop for Iowa State University Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center’s effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. The project is supported by the North Central Sustainable Agriculture Research and Education Professional Development Program. Sixty people attended this event, which followed a regional workshop attended by 63 partners from the North Central Region in July, 2009.

Although a special workshop for ISU Extension staff was not within the scope of the full proposal to SARE, the Leopold Center’s close relationship with Extension and discussions with ISU Extension administrators made it a natural outgrowth of the work and mutually beneficial. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor were participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. With the passage of time and additional experience conducting the Regional workshop, the Leopold Center’s CoP training team once again offered interested Extension staff a workshop. To strengthen Extension’s community-based partnerships, the CoP training team issued invitations to Extension staff encouraging them to attend the workshop with a community-based partner with whom they were working or wanted to work. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented non-profit social service organizations and the business community.

This report documents results of the evaluation of the 2010 Extension CoP workshop. We developed two survey instruments adapted from instruments used to evaluate the Regional workshop; the first was an electronic pre-test survey designed in Survey Monkey, whose link was e-mailed to registrants about a week prior to their arrival. The second was a paper survey distributed at the end of the workshop. The purpose of the surveys was 1) to prepare participants to think about issues addressed by the workshop before coming and 2) to develop benchmark measures on knowledge, attitudes, and experiences we would use to compare with post-test workshop survey results and longer-term follow up surveys.

Respondents

The response rate to the post-workshop survey was 53 percent (we received 32 completed surveys). Of the 68 who initially registered, 8 did not show up for an attendance count of 60. Of the pre-registrants, 31 completed the pre-workshop survey for a response rate of 46%. For the purpose of this evaluation, we therefore had a total of 63 surveys upon which to base our analysis.

Characteristics of Respondents/Participants

Table 17 shows that most survey respondents were women (71%), a figure that exceeds the proportion of women who actually attended the workshop (60%). Table 17 also shows the mean age and occupational affiliations of participants.

specialists (38%), followed by regional extension directors (1 in 4) and state specialists (1 in 4). Not quite half of Extension respondents attending the workshop were based in Agriculture and Natural Resources (46.2%), while one in four (23%) were based in the area of Families and Consumer Science. Communities and economic development was the focus area of 15% of Extension respondents while the same proportion reported specialties in more than one Extension area.

The majority of respondents (67%) attended the workshop with a partner. Most of the non Extension partners attending represented nonprofit social service providers and the business community. Table 1 shows the occupational affiliation of partners (who could choose all affiliations that applied). More than half of the non Extension respondents attending the workshop were affiliated with community-based nonprofit social services (54%) including food banks, substance abuse centers, and various community resource centers. Government employees comprised 46% of non Extension partners attending the workshop. Nearly two in five (39%) are involved in either youth or adult education and 15% were either farmers or entrepreneurs.

RESULTS

Respondents were motivated to attend the workshop mostly for content reasons: to learn about creating, facilitating, managing, and funding CoPs (51%) (Figure 7). However, answers to this open-ended question also yielded more interesting results: 36 percent reported they wanted to learn how to foster collaboration and how work together with other groups—in other words, they wanted to learn how to be good partners and learn about process. Another top reason for coming was to network with others (22%).

For the majority of respondents, the workshop met their expectations (78%) (Figure 8). For one in five, however, the workshop only met their expectations “somewhat” and one said it did not. Among those who selected “somewhat” or “no” responses regarding expectations, most said they still learned something but not all the material was new to them and they wanted more time to spend on it.

We evaluated workshop delivery and mechanics by asking respondents to rate the usefulness of each session using a five-point scale. Results are shown in Table 18. For the most part, respondents found the discussion and definition of CoPs most useful, followed by the small group exercise, which emphasized action steps necessary for focusing, initiating, managing, funding, and evaluating a CoP. The sessions on debriefing and branding were less useful to respondents.

Learning what information workshop participants find most significant is helpful for informing workshop designers and facilitators how well they gauged the needs and interest of their audience and whether the training team and respondents value the same information. To assess this, we included a straight-forward, open-ended question asking respondents to identify the most significant thing they learned. One in four (26.1%) identified as most significant the lesson on the importance of managing the CoP in terms of relationships and communication (Figure 8). The need to articulate common goals and partner with a variety of others with diverse backgrounds and perspectives were also significant lessons reported by nearly two in five respondents (17.4% each). Branding also surfaced, as did the distinction between different degrees of cooperation and the notion of nested networks (13% each). This is much aligned with the major thrust of the workshop, which emphasized the need for effectively

Armed with new information, we were curious to know what respondents were planning to do with the information after attending the workshop. Although immediately after the workshop is still too early to tell, asking them about their plans provides benchmark data from which to compare information on actual action taken that we gathered in the longer-term followup survey, which was administered in December, 2010, eight months.

At the end of the workshop one in three planned to apply the concepts they had learned to existing collaborations to improve their work and partnerships (Figure 9). Fewer (one in four) were planning to initiate discussions with local stakeholders to consider starting new CoPs. Additionally, one in five planned to reach out to include new partners in their work.

Although medium and long-term outcomes are virtually impossible to assess immediately after a workshop, administering a similar survey both before and after the workshop does enable us to measure short-term changes in attitudes and knowledge, especially if some questions remain identical in both surveys. This was a strategy we used to test the statistical significance of any short-term change that might have occurred as a result of participation in the workshop. Table 3 compares results from the pre-workshop survey with responses to the surveys distributed after the workshop. In both surveys, the questions were the same. We asked respondents to rate the extent to which they agreed with 23 statements about their attitudes toward and knowledge of CoP related issues. The table shows the pre-test survey mean of responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. T-statistics and p-values are also shown.

For every single statement, the post-test mean increased once the workshop had been conducted. Seventeen or 74% of responses to the 23 statements were statistically significant changes. However, six were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop “intervention&” or “treatment.” To interpret this in practical terms, take statement d) for example. Here, the chance that the mean would increase from 3.77 to 4.32 without the workshop intervention is 3 in 1000, strongly suggesting that the workshop is related to the increase. The lower the p-value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different.

The results in Table 19 show that important knowledge and attitude changes are attributable to the workshop. In summary, statistically significant changes among respondents include:

• Greater familiarization with the concept of CoPs and their benefits;
• Increased interest in starting and leading a new CoP;
• Increased interest in joining a CoP (but not necessarily leading);
• Greater sense that CoPs can help Extension (more efficiently) achieve its mission and goals;
• Stronger attitude that CoPs are a viable way to address key issues while better connecting respondents and their organizations to people they serve;
• Better understanding of different ways to fund and need to brand CoPs;
• Greater knowledge about the range of options for setting up a CoP; and
• Greater confidence in respondents’ respective organizations to have the capacity and resources to initiate and facilitate a CoP.

Statements that did not generate statistically significant results are worth examining as well. The mean level of agreement for the statement, “I am interested in applying concepts of the CoP model to work I am already doing” did not increase dramatically after the workshop because respondents generally agreed with this statement to start with (4.23 pre-workshop; 4.52 post-workshop). The same holds true for “I believe the CoP model can inform the work I do” (3.87 pre-workshop; 4.20 post-workshop). Before attending the workshop, respondents were already philosophically dedicated to working with multiple organizations and the need for working with partners from different sectors to address salient issues. Indeed, the role of Extension is to serve as a facilitator and catalyst within the communities and regions where based. And while there was an increase in respondents’ perception that different organizations in their area of work would be willing to actively engage in a CoP to address those issues, the difference was not significant (p=.066).

Workshop participants are clearly a collaboratively minded crew. Results in Table 19 above, as well as other results, provide evidence of this. Figure 10 provides further evidence, particularly as it relates to collaboration between Extension and their workshop partners. All responding partners reported a willingness to partner with Extension in the future (as indicated, additionally, by their mere presence at the workshop) and 91 percent report they have already partnered with Extension in the past. About half are currently partnering with Extension.

Asking participants to attend the workshop with a partner was a deliberate strategy on our part to strengthen existing relationships Extension staff already had in place with partners from their region or community. To this end, the workshop served its purpose. One of the benefits of attending the workshop with a partner was a better appreciation and understanding of partner organizations and resources (Figure 11). One in three workshop participants who attended with a partner also reported they learned how to collaborate better. Perhaps most valuable was the response from one respondent who said they learned they were not ready to play an active role in a Community of Practice yet. Sometimes CoP work is premature and knowing when the time is (or is not) right is just as important as understanding how to make it work.

Despite respondents’ obvious commitment to collaborative processes, there is always room for improvement. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before. Nearly one in five (18.2%) answering this question was considering working with producers, Extension, regional development groups, funders, and health-based groups. When we asked respondents what would be needed for their organization or business to fully participate in CoP work, 60 percent said first and foremost they needed administrative support and permission to spend time on pursuing and nurturing collaborative relationships. The participation of others and the better articulation of organizational goals were also needed (13.3%, respectively).

Discussion and Conclusions

Community of Practice work is a natural fit for Extension and its partners. The level of understanding of this audience about the need for engaging diverse partners to address key issues in sustainable agriculture and create healthy communities, and the need for managing the resulting collaborative relationships that result from outreach, made it appear this training wasn’t needed for this group. We did not have to convince them of the value of this work. However, our challenge and their challenge lies beyond the workshop: To effectively engage critical partners who may be less enthusiastic about the difficult process work needed to improve their communities and landscapes. Even within the workshop, we discovered various levels of interest based on participation. Communities and Economic Development field staff and to a lesser extent, staff with expertise in Families and Consumer Science were less represented than those in Agriculture and Natural Resources. Given the relatively low participation from Families and Consumer Science, we were surprised at the variety of representation of community partners who accompanied Extension partners and the social service experiences they brought with them.

The CoP work of the Leopold Center and its partners in Iowa has focused largely on the area of food systems and sustainable agriculture. We were delighted to learn that the principles of our work could easily be transferable to another set of activities in the state that served the same or a similar purpose: to improve community health across the state. Apparently, the ease of transferability has to do with the process. The same general principles of the process hold true, regardless of the content of focus. Organizing and coordinating a working group comprised of different partners with different perspectives, experiences, and backgrounds offers the same opportunities and challenges, regardless of the topic—whether that topic is addiction, irrigation, immigration, or food access. What this work builds on is the tacit knowledge we have gained through experience and the tacit knowledge we have learned from others doing similar work. Essentially, the CoP workshop is a vehicle or structure for developing content about process, something that collaborative-minded individuals and groups are hungry for, most of which remains hidden in the realm of specialized journals or otherwise inaccessible publications. With this project, we tried to make tacit knowledge on process explicit and accessible to practitioners who have the capacity to lead this work forward and thus, creating positive change in sustainable agriculture or other arenas.

Based on the evaluation results, this workshop did not offer participants groundbreaking information that would change the course of their lives forever. For this already innovative group of collaborators, some with more experience than the training team, the workshop did serve to build confidence in their and their organization’s ability to do such work and reinforced what they knew about building collaborations. Evaluation results suggest the workshop formally validated those experiences, added new information to their portfolio, and increased interest in starting, joining, or leading a CoP. We expect the combination of these changes to lead to leadership changes in communities as Extension and its partners renew or continue their vows to work together and bring positive change to their communities.

Measuring Medium-Term Impacts: Followup Survey Results

In December 2010, we invited participants who had attended the North Central Regional Community of Practice (CoP) workshop in July, 2009 and ISU Extension personnel who attended the April, 2010 workshop to complete a brief electronic survey on progress they made or challenges they encountered related in their CoP work since attending the workshop. We hoped that with the passage of time (8 months in the case of the April 2010 workshop and a year and a half in the case of the July, 2009 workshop) we would be able to measure behavioral change as opposed to simply short-term changes in attitudes, skills, or knowledge.

Of the 117 recipients, we received 33 usable responses for a response rate of 28 percent. In terms of those who responded, most were women (61%). And while we received responses from participants in 7 of the 12 North Central states, most were from Iowa (65%). States NOT represented included North Dakota, Kansas, Michigan, Nebraska, and Indiana. In addition, half of the respondents worked for Cooperative Extension as state specialists, field specialists, or (former) county extension directors, with most (68%) working in the area of agriculture and natural resources. The other non Extension half represented non profits, the farming community or farm-based entrepreneurs, government, Resource Conservation and Development, education, research, and economic development.

The survey was designed to include a list of 12 possible actions respondents could have taken “in part as a result of attending the workshop.” Respondents could also write in actions they took that were not included in the list. If respondents selected one or more actions, they were asked relevant follow up questions to explore their responses. For example, if they indicated they started or helped start a new CoP, they were asked what kind of CoP they started. In addition, respondents were also asked to describe what happened as a result of each action. For each indicated, respondents were asked to share why the action did not work IF it was not successful. From this line of questioning, we tallied and coded responses described in the middle and right hand columns in Table 20.

Summary and Discussion

Table 20 summarizes the medium term impacts of this project. Perhaps one of the most significant impacts of the workshops was demonstrating the importance and benefits of forging new partnerships. More than half (58%) of the respondents reported they internalized this message by actually reaching out to include new groups as partners. As a result, seven respondents said they development new partnerships with other organizations and one reported that doing so led to a major new initiative. Furthermore, a clear majority (79%) conveyed the message of including new partners to fellow colleagues within their own and nearly two in three (61%) went even further by encouraging others outside their organization to include new partners. The result of this effort was 9 new collaborations, an increase in funding for partners, applications for new grants, and greater support for their work.

We were also interested in learning whether new CoPs formed as a result of this project or existing ones were strengthened. More than one in three respondents to the survey (38%) joined or became more actively involved in an existing CoP while nearly one in four (24%) started or helped start a new CoP. Furthermore, one in three (34%) applied for a grant that included funding for development of a new group or enhancement of an existing group. More than half who applied for funding (55%) actually received a grant.

A composite snapshot of the actions listed in Table 21 shows that carefully designing and implementing an inclusive, collaborative process does indeed net tangible results. The medium-term follow up survey provides evidence that participants applied key CoP concepts to their work and changed behaviors and practices. Doing so yielded a variety of positive results, from gaining an expanded support base with new partners, to new funding, expanded programming, greater opportunities for sharing the work and decision making, greater project participation, and new leadership. Longer-term changes to the landscape and built environment trailed the list of impacts, but the mere fact that respondents actually reported some was most (and pleasantly) surprising given that only 8-18 months had elapsed since the workshop.

While the benefits are clear, there are still challenges to overcome. Some respondents reported they don’t always succeed in their CoP work because this work can be too demanding in terms of managing collaborative relationships, organizing effective leadership, finding funding, making time “to be active with other priorities,” and simply trying to do too much with slim-staffed organizations with a limited capacity to make things happen on shoestring budgets. Given these rather formidable obstacles, the successes reported by this group become all the more impressive.

Project Outcomes

Project outcomes:
Final CoP Workshop

On April 14-15, a final workshop was held in Ames. The primary thrust of the workshop was to identify priority issues for beginning and advanced Community of Practice work, co-discover ways to address challenges, and make existing Communities of Practice more effective. The workshop was attended by 44 participants from 10 of the 12 North Central states including Iowa, Minnesota, Nebraska, South Dakota, Illinois, Missouri, Kansas, Wisconsin, Michigan, and Ohio. We also had participants from Massachusetts, Virginia, and Colorado. We administered a paper survey at the end of the conference to measure short-term outcomes and possible medium-term outcomes in case participants had taken part in the project’s early and ongoing events. We received 22 completed surveys for a response rate of 50 percent. Respondents represent non profits, Cooperative Extension, the education sector, farming, consultants, and conservation groups/agencies.

Participants report the workshop was very useful, giving it a high mean score of 4.8 on a 5-point scale where 1=not at all useful and 5=very useful. Nearly 70 percent have been with the project for some time, having used project resources or attended at least one event prior (for an average of using/attending two previously held events or resources listed in Figure 11).

In addition to questions about respondents’ involvement in the project, we also wanted to know what kind of actions they had taken as a result of the project. Table 21 below shows that one in three reported developing new contacts and/or partners for their work, and 90% reported they would do so in the future. Nearly one in three also reported they had already incorporated new ideas and/or information into their regular programming with three in four reporting they planned to in the future. At least nine in ten also reported they planned to use information from this project to answer client questions (94%) and publicize their experience with this project in newsletters or other media outlets (92%).

The last set of questions in the post workshop survey queried respondents about even longer term outcomes as a result of their participation in the project (Figure 12). Nearly half (46%) for whom it was applicable (at least 17 in all cases) report that their organization or business is actively partnering with a broader base of partners as a result of their CoP work and is also serving a broader more diverse set of people (46%). Two in five (41%) also report that farmers and farm-based business owners are either getting involved in sustainable agriculture work or are taking on new leadership roles in sustainable agriculture work as a result of their CoP efforts. Moreover, one in three report new agencies, institutions, and organizations are getting involved in sustainable agriculture work in their area because of respondents’ CoP work. Lastly, nearly one in four say that their participation in this project improved their organization’s ability to leverage funding related to their CoP work. When asked a followup question on the estimated total financial support respondents or their CoP partners have been able to leverage for their joint CoP work since attending their first SARE-funded CoP event with us, four respondents reported a total of $328,000 leveraged for their CoP work.

Conclusions

Although this was only a two-year project, multiple sources of feedback from respondents across the region participating in various aspects of this project demonstrate the medium-term impact of this work in the region, which has long term implications well beyond the evaluation scope of this project.

Our evaluation results show that project participants and local partners have been able to leverage over $300,000 to support their work in sustainable agriculture using the Community of Practice model, which emphasizes the value of engaging new partners in new ways and turning over leadership and decision making to sustain the work, placing it squarely in the hands of farmers who can participate and the support organizations and businesses serving them. As a result, organizations, agencies, institutions, and businesses involved in this work say they are reaching new audiences and serving those audiences more effectively. As we concluded at the end of our first year evaluation report, over time, we hope to see more impacts trickle down to farmers and consumers through increased opportunities to create and consume farm products and services that contribute to the triple bottom line goals of creating environmental, social, and financial good. We’ve seen some of that already happening in Year Two and expect SARE and our other partners in this effort will see more of it in years to come.

Recommendations:

Potential Contributions

Overall Summary and Conclusion

YEAR 1

The CoP training, made possible through a grant from the SARE Professional Development Program, was attended by 106 Extension, non-profit, government agency, university, farmer, and farm-related business representatives from each of the 12 states in the North Central Region, as well as states on each of the coasts. For both the pilot (held specifically for Extension staff) and the regional workshops, Extension Service staff had a strong presence although non-profits were better represented at the July regional meeting. Women were the largest group in attendance at both the pilot and regional workshops. Their strong presence, leadership, and interest in the working group model have implications for future strategies that will require a deep level of collaboration to accomplish goals in sustainable agriculture.

According to results from the surveys, the workshops succeeded in moving the project forward in terms of increasing the capacity of individuals and organizations in the north central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. Presentations and discussions, participant interaction, and training materials developed for the workshops helped make explicit tacit knowledge of the training group. This contributed to increasing participants’ confidence in forming and managing CoPs, provided ideas on how to fund a CoP, and brought to the forefront important skills and qualities needed to create and maintain a successful CoP. Statistics show that the workshop increased participants’ familiarity with the CoP concept, increased awareness about CoP benefits, and increased regional participants’ attitudes about their viability as a way to help their respective organizations achieve their goals. Participants also believe that CoPs can help their respective organizations work more efficiently and can connect them to people and communities they serve. Results from the regional workshop also show a jump in participants’ interest in forming a CoP as a result of the workshop, suggesting that participants in the region are looking to take on leadership roles in developing and supporting CoPs.

The workshop also prepared participants to organize and facilitate a CoP in their area of expertise and positively impacted participants’ attitudes about the capacity of their organization to create and facilitate a CoP (Extension staff cuts and restructuring notwithstanding). The surveys clearly showed that participants believed and continue to believe that building capacity to support CoP development necessarily involves partnering with other organizations to help them access and maximize use of important additional resources. Conservative estimates show participating individuals and organizations from the region reported partnering with an average of 11 groups in the past two years to further work in sustainable agriculture. These data suggest that these groups already are philosophically aligned with the working group approach and, therefore, are poised to take it one step further. An additional contribution of the workshop is its role in prompting participants to consider using the CoP model with new groups and groups with whom they are already partnering. In terms of new groups, besides the “choir” groups such as local food interests, more than a handful mentioned they were now considering working with “big ag” such as Farm Bureau, pork producers’ associations, beef producers’ associations, and corn and soybean growers’ associations.

Capacity building means not only building bridges with unlikely partners in industry, but also intentionally localizing decision making to better engage a wide variety of stakeholders at the community level. As one participant put it, “[We will] continue to move forward and give up more ownership of projects to communities [as a result of the workshop].”

The Leopold Center will continue to engage and support individuals and organizations in the region interested in implementing the working group model. The Center will hold regular conference calls on topics participants themselves identify, host the social networking Ning site, provide consulting services one-on-one as requested, convene a second meeting in fall or early winter of 2010-2011 to continue the process of learning and discovery, and jointly reflect on the results of the two-year journey together.

After only one year, we have measured benefits this project has provided to participating individuals and groups; but, it is difficult at this point to measure the extent to which it has benefited individuals and groups not participating in the workshops. However, we expect that as time elapses and participants start to put some of the CoP concepts into practice, evaluation activities in Year 2 will demonstrate the impact of using this model to build the capacity of multiple stakeholders involved in sustainable agriculture to maximize their impacts by working together with others. Over time, we will start to see impacts that will reach farmers and consumers through increased opportunities to create and consume farm products and services that contribute to the triple bottom line goals of creating environmental, social, and financial good.

YEAR 2

Community of Practice work is a natural fit for Extension and its partners. The level of understanding of this audience about the need for engaging diverse partners to address key issues in sustainable agriculture and create healthy communities, and the need for managing the resulting collaborative relationships that result from outreach, made it appear this training wasn’t needed for this group. We did not have to convince them of the value of this work. However, our challenge and their challenge lies beyond the workshop: To effectively engage critical partners who may be less enthusiastic about the difficult process work needed to improve their communities and landscapes. Even within the workshop, we discovered various levels of interest based on participation. Communities and Economic Development field staff and to a lesser extent, staff with expertise in Families and Consumer Science were less represented than those in Agriculture and Natural Resources. Given the relatively low participation from Families and Consumer Science, we were surprised at the variety of representation of community partners who accompanied Extension partners and the social service experiences they brought with them.

The CoP work of the Leopold Center and its partners in Iowa has focused largely on the area of food systems and sustainable agriculture. We were delighted to learn that the principles of our work could easily be transferrable to another set of activities in the state that served the same or a similar purpose: to improve community health across the state. Apparently, the ease of transferability has to do with the process. The same general principles of the process hold true, regardless of the content of focus. Organizing and coordinating a working group comprised of different partners with different perspectives, experiences, and backgrounds offers the same opportunities and challenges, regardless of the topic—whether that topic is drug addiction, irrigation, immigration, or food access. What this work builds on is the tacit knowledge we have gained through experience and the tacit knowledge we have learned from others doing similar work. Essentially, the CoP workshop is a vehicle or structure for developing content about process, something that collaborative-minded individuals and groups are hungry for, most of which remains hidden in the realm of sociological theory on organization in specialized journals or otherwise inaccessible publications. With this project, we tried to make tacit knowledge on process explicit and accessible to practitioners who have the capacity to lead this work forward and thus, creating positive change in sustainable agriculture or other arenas.

Based on the evaluation results, the workshop did serve to build confidence in their and their organization’s ability to do such work and reinforced what they knew about building collaborations. Evaluation results suggest the workshop formally validated those experiences, added new information to their portfolio, and increased interest in starting, joining, or leading a CoP. We expect the combination of these changes to lead to leadership changes in communities as Extension and its partners renew or continue their vows to work together and bring positive change to their communities.

Measuring Medium-Term Impacts: Follow-up Survey Results

In December 2010, we invited participants, who had attended the North Central Regional Community of Practice (CoP) workshop in July, 2009 and ISU Extension personnel who attended the April, 2010 workshop to complete a brief electronic survey on progress they made or challenges they encountered related in their CoP work since attending the workshop. We hoped that with the passage of time (8 months in the case of the April 2010 workshop and a year and a half in the case of the July, 2009 workshop) we would be able to measure behavioral change as opposed to simply short-term changes in attitudes, skills, or knowledge.

Of the 117 recipients, we received 33 usable responses for a response rate of 28 percent. In terms of those who responded, most were women (61%). And while we received responses from participants in 7 of the 12 North Central states, most were from Iowa (65%). States NOT represented included North Dakota, Kansas, Michigan, Nebraska, and Indiana. In addition, half of the respondents worked for Cooperative Extension as state specialists, field specialists, or (former) county extension directors, with most (68%) working in the area of agriculture and natural resources. The other non Extension half represented non profits, the farming community or farm-based entrepreneurs, government, Resource Conservation and Development, education, research, and economic development.

The survey was designed to include a list of 12 possible actions respondents could have taken “in part as a result of attending the workshop.” Respondents could also write in actions they took that were not included in the list. If respondents selected one or more actions, they were asked relevant follow up questions to explore their responses. For example, if they indicated they started or helped start a new CoP, they were asked what kind of CoP they started. In addition, respondents were also asked to describe what happened as a result of each action. For each indicated, respondents were asked to share why the action did not work IF it was not successful. From this line of questioning, we tallied and coded responses described in the middle and right hand columns in Table 23.

Summary and Discussion

Table 23 summarizes the medium term impacts of this project. Perhaps one of the most significant impacts of the workshops was demonstrating the importance and benefits of forging new partnerships. More than half (58%) of the respondents reported they internalized this message by actually reaching out to include new groups as partners. As a result, seven respondents said they development new partnerships with other organizations and one reported that doing so led to a major new initiative. Furthermore, a clear majority (79%) conveyed the message of including new partners to fellow colleagues within their own and nearly two in three (61%) went even further by encouraging others outside their organization to include new partners. The result of this effort was 9 new collaborations, an increase in funding for partners, applications for new grants, and greater support for their work.
?
We were also interested in learning whether new CoPs formed as a result of this project or existing ones were strengthened. More than one in three respondents to the survey (38%) joined or became more actively involved in an existing CoP while nearly one in four (24%) started or helped start a new CoP. Furthermore, one in three (34%) applied for a grant that included funding for development of a new group or enhancement of an existing group. More than half who applied for funding (55%) actually received a grant.

A composite snapshot of the actions listed in Table 22 shows that carefully designing and implementing an inclusive, collaborative process does indeed net tangible results. The medium-term follow up survey provides evidence that participants applied key CoP concepts to their work and changed behaviors and practices. Doing so yielded a variety of positive results, from gaining an expanded support base with new partners, to new funding, expanded programming, greater opportunities for sharing the work and decision making, greater project participation, and new leadership. Longer-term changes to the landscape and built environment trailed the list of impacts, but the mere fact that respondents actually reported some was most (and pleasantly) surprising given that only 8-18 months had elapsed since the workshop.

While the benefits are clear, there are still challenges to overcome. Some respondents reported they don’t always succeed in their CoP work because this work can be too demanding in terms of managing collaborative relationships, organizing effective leadership, finding funding, making time “to be active with other priorities,” and simply trying to do too much with slim-staffed organizations with a limited capacity to make things happen on shoestring budgets. Given these rather formidable obstacles, the successes reported by this group become even more amplified.

Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the U.S. Department of Agriculture or SARE.