Learning how to use communities of practice to address sustainable agriculture issues

Communities of Practice (CoPs) or working groups for Extension, agency, and non-profit educators and researchers in the north central states. The workshop was part of the Leopold Center's two-year NCR SARE PDP project to increase capacity in the north central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. A total of 63 participants engaged in the regional workshop, formally referred to as the Community of Practice Workshop. This workshop was preceded by a pilot workshop held in late April for Extension staff in Iowa to sample the methods and materials.

To evaluate the impact these workshops had on attendees, we developed two survey instruments: an electronic pre-test survey and a paper post-test survey. The purpose of these surveys was to: 1) prepare participants to think about issues addressed by the workshop prior to their arrival; 2) gauge their interest in particular types of working groups to help the training team design workshop sessions; 3) measure benchmark knowledge, attitudes, and experiences we could compare to post-test workshop survey results; and 4) measure medium-term changes in behavior that attendees might report at the end of the two-year project.

On April 19, 2010, the Leopold Center held a workshop for Iowa State University Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center's effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. The project is supported by the North Central Sustainable Agriculture Research and Education Professional Development Program. Sixty people attended this event, which followed a regional workshop attended by 63 partners from the North Central Region in July, 2009.

In April 2011, the Leopold Center held a follow up workshop for sixteen returning NCR SARE PDP participants. In addition, 16 Iowans were invited to participate. Travel funds for 6 Iowans were paid by Iowa SARE funds. Panelists for the workshop were; Rich Pirog (Leopold Center) - Regional Food Systems Working Group; Madeline Schultz (ISU Extension), Cooperatives Community of Practice; Sarai Rice (Des Moines Area Religious Council), The Alban Institute; Eric DeLuca, National, Cooperative Business Association; Jim Dyer, Southwest Marketing Network; Teresa Engel, Wisconsin Local Food Network

The pilot Community of Practice workshop for Extension was the first workshop delivered. It was created and delivered to:

• Test pilot workshop materials and workshop community of practice curriculum prior to the July meeting with representatives from the North Central Region;
• Explain the purpose, function, and value of using working groups to address key issues that are part of ISU Extension’s work plan; and
• Use Iowa-based examples and lessons learned to share and discuss principles and considerations about initiating, managing, funding, and branding communities of practice as a process tool in ISU Extension programming.

The Leopold Center initially aimed to involve a maximum of 24 people at the pilot workshop on April 30, 2009. However, requests continued to stream in beyond the intended cap. Ultimately, 43 attended. We asked registrants to complete a pre-test survey electronically; 38 of which were completed, for a response rate of 88 percent.

Timing is critical, however. The day before the April 30 event, ISU Extension administrators informed the SARE project leader that on the day of the workshop, Extension would announce its new plan to restructure in response to severe budget cuts. The news was grim as positions would be eliminated. Despite this, administrators advised the SARE project leader to continue with the CoP training and asked to be added to the agenda so they could directly address attendees about the reorganization. They warned that some participants in attendance might not choose to stay for the training after hearing about the restructuring plan. Of the 43 attending, approximately seven did leave throughout the course of the day. From the 36 remaining, we received 26 useable post-workshop surveys, for a 72 percent response rate. The following tables and paragraphs briefly summarize what we learned from them.

Table 5 shows a breakdown of the position and program area affiliations of responding Extension staff participating in the pilot workshop, as well as basic demographic characteristics. The reduction of County Extension Directors (CED) from 47.1 percent to 30.4 percent in the pre- to the post-test responses is likely the result of the Extension announcement the day of the workshop, in which all County Extension Directors in Iowa were given notice they would be losing their jobs in the reorganization. Many Extension Directors did not come as planned or left the workshop in progress once the announcement had been made. Field specialists were deemed &"safe" and, therefore, we did not see any kind of attrition in this group based on numbers from the survey results.

Table 5 also shows the program area expertise of Extension participants. (Respondents could choose more than one area.) Extension staff working in the areas of Agriculture and Natural Resources and Families comprised the majority of respondents, creating a roughly 3 to 1 ratio of Families or ANR staff to CED or 4-H personnel.

Table 6 compares results from the pilot pre- and post-test questions that queried participants on the extent to which they agreed with 21 statements about their attitudes and knowledge of CoP-related issues. The table shows the pre-test survey mean of their responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. The t-statistics and p-values also are listed.

For every statement, the post-test mean increased once the workshop had been conducted, albeit only slightly for some. Yet, not all increases are statistically significant. Seven were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop

Interestingly, Table 6 shows that the workshop had no effect on whether Extension personnel were interested in forming a CoP. These results differ from the July workshop results with representatives from the region (described later), which show a statistically significant (positive) change in participants' willingness to form a CoP after they had received the training. Also worthwhile to compare with the regional results is the lack of change in attitudes after the training among Extension personnel regarding the capacity of their organization to organize and lead the creation and facilitation of a CoP. Participants in the pilot training experienced no change in attitude regarding this point after attending the workshop (where p=.866), whereas the July workshop attendees were more likely to agree that their organization had the capacity once they had been through the training (p=.018). The day’s announcement to reorganize Extension and cut 100 County Extension Directors from the payroll almost certainly had an effect on these results, and most likely had an immediate impact on participants' attitudes about Extension having the capacity to function in the future.

Questions related to participants' attitudes and activities regarding partnerships, while not statistically significant above in points r), s), and t), reveal that Extension already is looking outside itself to connect with other people and resources. The majority (87 percent) of respondents to the pre-test survey responded affirmatively when asked whether they had tried to leverage resources in the past two years in partnership with another organization, group or agency to further their work. Among the partners listed were local farmers; local businesses; Chambers of Commerce; local elected officials; county health officials; churches; food pantries; County Conservation boards; Farm Bureau; banks; insurance companies; community colleges; producer associations; City Parks and Recreation departments; local hospitals; schools; rural advocacy groups, such as the Center for Rural Affairs; state agencies, including IDALS and DNR, Workforce Development, and the Department of Health and Human Services; non- profits such as Resource Conservation and Development; councils/agencies, Practical Farmers of Iowa, the Iowa Network for Community Agriculture; and low-income family service providers, such as Iowa Community Action agencies; and University-based partners including Extension and the Leopold Center. The range represented in this list of organizations, agencies, and institutions suggests Extension is already striving to make better use of local community resources.

The notion of doing better outreach or some variation on that theme was listed by one in three participants as one of the most significant things learned in the workshop. "[We need to] spend more time bringing folks together," said one participant. Another remarked he had learned that "CoPs are a good way to bring diverse groups with common goals together to work towards those goals." Still others mentioned that the most significant thing they learned was the need to involve actual clients in working groups and how important working groups are as a means for "reaching and relating to clientele." This was the reason some Extension staff chose to attend: "I thought I could learn more about building collaboration and sponsorship [for Extension work]."

Clearly, the notion of creating CoPs or working groups as a strategy for better engaging clients and communities in the work of Extension appealed to workshop participants. However, many were concerned about the availability of Extension resources to support such an approach. One participant prefaced his comments with, "If funding is available..." and another finished her comments with, "...if I am still involved with Extension." Although there were positive evaluation comments about the workshop and content delivered, there also were some tepid reactions, which one participant chalked up to "the context of the budget cut information [presented by Extension administrators that] affected us all and our participation. It was a distraction we couldn’t control." While some found it "difficult to engage in light of the news about Extension," others found value in the training and materials. "The resource workbook will be a great tool. Hearing from the various CoP working groups and having their examples was very helpful." For others, the workshop articulated "[what] I think is new language for what I have [already] done in communities in the past."

After modifying the design and format of the regional workshop based on experience and feedback from the pilot, the regional workshop was set for July 21-22 in Des Moines. At the time we first surveyed the regional workshop participants, 76 were signed up. Of those participants, 52 provided useable pre-test surveys, for a 68 percent response rate. We administered the pre-test survey electronically using Survey Monkey, an on-line survey tool. Of the 63 who actually attended the workshop, 51 completed a useable post-test survey, for a response rate of 81 percent. The post-test survey was administered on-site using paper copies included with the workshop handouts and materials. However, not all participants answered all questions all of the time. Therefore, tables throughout this report list the number of respondents (in parentheses) who answered each question.

About the Respondents

Table 7 shows that most respondents are women, particularly those responding to the post-test survey. This closely aligns with the gender of the attendees, which was 63 percent female, based on a review of names in the registration lists. However, men also were a strong contingent at the workshop, as were participants from several different age groups. Table 8 shows the average attendee age is 48 but the range spanned six decades, showing a mix of interest from nearly all age groups.

Table 9 shows the occupational affiliations of participants, who were invited to choose multiple categories if applicable. Non-profit representation was the strongest by far, comprising one-third of workshop attendees. More than one in four attendees represented Cooperative Extension, while government employees and farm-related business each totaled nearly 14 percent of those present.

Table 10 shows where workshop attendees live and/or work. The majority are from Iowa, followed by Illinois, Minnesota, Nebraska, and Michigan. And, although we have no pre- or post-test results from Kansas and no post-test results from North Dakota, one representative from each state did indeed attend the workshop, so that all 12 states in the North Central Region were represented.

Impact on Regional Attitudes and Knowledge

As we did for the ISU Extension pilot, in both the pre- and post-test surveys of regional representatives we asked participants to indicate the extent to which they agreed with 21 statements about their attitudes and knowledge of CoP-related issues. Table 11 shows the pre-test survey mean of their responses on a five-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. We assumed normality of results given our sample size was over 100 (103) and therefore used parametric statistics (t-tests of means) to determine whether any rise or drop was statistically significant. The t-statistics and p-values also are shown in Table 11.

For each of the 21 statements, the post-test mean increased once the workshop training had been delivered. For all but three statements, results were statistically significant where p<.05, meaning that the increase in the means purely due to chance and/or error is so small that we can attribute the change to the workshop. To interpret this in practical terms, use statement c) as an example. Here, the chance that the mean would increase from 3.71 to 4.20 without the workshop intervention is four in 1000, suggesting that the workshop is related to the increase. The lower the p value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different. If the means were equal as hypothesized, there is a 4 in 1000 chance that a more extreme test statistic would be obtained using data from this population.

These results are heartening, and show that knowledge and attitude changes can be attributed to the workshop. Statistics show that the workshop increased participants' familiarity with the CoP concept, increased awareness about CoP benefits, and increased participants' attitudes about their viability as a way to help their respective organizations achieve their goals. Participants also believe that CoPs can help their organization work more efficiently and can connect them to people and communities they serve. And while we saw a jump in participants' interest in forming a CoP as a result of the workshop, we did not see a significant increase in their interest to simply participate in a CoP. These results suggest that participants are looking to take on leadership roles in starting CoPs and already had a definite interest in participating in a CoP both before and after the workshop.

The other two statements that did not generate statistically significant results were "I currently partner with other groups in my area of work to advance opportunities related to sustainable agriculture" and "I am willing to work with multiple organizations to create change in the area of sustainable agriculture. "Although the means for both did increase, it was not enough of a bump to give credit to the workshop because participants attending the workshop already partnered with multiple organizations to further their work in sustainable agriculture. Such assertions are evident in responses to the question we posed about whether workshop registrants (pre-test) had tried to leverage resources in partnership with another organization, group, or agency to further work in sustainable agriculture. Nearly all (92.3 percent) said yes (n=52). Workshop participants provided even more details in the post-test survey, citing a mean of 14.4 new groups, organizations, agencies, and businesses they collaborated with in the past year to further their work (n=45), ranging from 0 to 150. The mode (most common response) was six new groups. If we remove the outlier of 150 groups, the mean drops to a still impressive 11.4 groups (where the mode remained the same and the maximum topped out at 41).

Despite these impressive partnering figures, the workshop appeared to strengthen participants' pre-existing collaborative orientation, as indicated by their affirmative responses to the following questions:

• Has this workshop prompted you to consider using the CoP model to work with groups with whom you have never partnered to further your work in sustainable agriculture? (83.3 percent "yes" n=42) and
• Has this workshop prompted you to consider using the CoP model when working with groups with whom you are already partnering to further your work in sustainable agriculture? (93.3 percent "yes" n=45)

Respondents cited other groups they would now consider working with in a community of practice setting (beyond the "choir" groups such as local food groups and regional produce distributors). More than a handful mentioned "big ag" interests such as Farm Bureau, pork producers' associations, beef producers' associations, and corn and soybean growers' associations as possible collaborators. "I am very inspired to give this a whirl with two groups of ag types I am working with." Others said they were now considering, but had never partnered with entities such as elected officials, city planners, or economic development agencies, councils, or banks. Segments of the health sector such as hospitals and public health agencies also emerged as potential partners in moving sustainable agriculture work forward collectively.

Regional Workshop Structure and Format

The way in which training is delivered can impact reported outcomes. Evaluating the process and design of knowledge building can provide us with clues about impacts we measure. In light of this, we asked specific questions about characteristics of the workshop and how important each one was in contributing to learning outcomes or connecting participants to resources.

Table 12 shows that for all the workshop structure and format questions asked in the post-test survey, roughly 3 in 4 participants felt each component was important (rating it either a 4 or 5 which have been combined in the far right column). Most notably, results show that 9 in 10 participants feel the experiences of the facilitators and discussants were critical for helping them understand the structure and function of working groups and options for creating new ones. Nearly the same percentage viewed the dialogue and interaction with fellow participants as a contribution to their learning.

Next Steps for Regional Participants

When we asked the regional participants an open-ended question about any actions they were planning to take as a result of attending the workshop, nearly one-third (29 percent) answered the question. That one in three participants was able to articulate concrete ideas about how to use information learned in the workshop immediately after its delivery demonstrates to some degree the relevance of the curriculum as well as the contemplative nature of participants.

Of those who did respond, 20 percent expressed an interest in forming new working groups; however, the majority (47 percent) expressed an interest in using what they learned to reconfigure and optimize the performance of groups with whom they are already working. "I am going to [try to] reconfigure local working groups, share with food system council members, and work more on documentation." Still others said they were rethinking leadership in terms of how to better engage local and regional interests in their work.

Next Steps for the Project

It is generally expected that changes in knowledge lead to changes in attitudes, both of which are apparent from the post-workshop quantitative survey data. Qualitative data gleaned from open-ended questions confirm this claim. As one respondent remarked, the knowledge she learned in the workshop changed her attitudes about working groups. "I came with no knowledge of working groups. I thought of them more like socialism--now I see them as democratic." The training appears to have created a more accepting environment for using CoPs as a model for working with varied partners. In addition and as already noted in the discussion on the pilot workshop, the regional workshop gave participants with a common language that provided clarity about the way some participants say they are already working. "[The workshop] was a wonderful idea. It blended learning about the concept of Communities of Practice in a setting where we were immediately able to apply it--perhaps to what we are already doing." However, there is still work to be done. As some participants noted, they want to see more discussion of "tacit knowledge" associated with working group formation and maintenance and more guidance on trouble shooting problems and finding funding to support working groups.

This is valuable information since the CoP workshop was one of the first several steps over a two-year period to assist Extension, agency and non-profit educators and researchers in the North Central Region in using CoPs or working groups to address issues in sustainable agriculture and food systems. The second step of the process is to hold a series of conference calls to network and support interested workshop participants as they develop CoPs of their own. To prepare for this, we asked workshop participants what they wanted to learn more about. This information will allow us to better target future efforts on items of the greatest and most intense interest (Table 13).The topic that aroused the most interest among participants was funding a working group (87.5 percent agreed they wanted to learn more about funding), followed by branding and conditions for success when selecting a working group.

Given the obvious desire for more learning, we also queried participants about their interest in participating in future conference calls. Eighty-five percent (or 41 out of 48) said they were interested, all of whom were willing to connect on a quarterly basis. One in three was willing to connect by phone on a monthly basis. About half (52 percent; n=44) said they were interested in one or more individual phone consultations with members of the Value Chain Partnerships team. Participants were most interested in phone consultations with the:

• Regional Food Systems Working Group (57 percent )
• Fruit and Vegetable Working Group (22 percent )
• Grass-Based Livestock Working Group (13 percent )
• Branding consultant (9 percent )
• Evaluator or any CoP Facilitator (8 percent )
• Pork Niche Market Working Group (4 percent )

The first conference call was held on August 11, garnering participation from 17 people located around the region and seven members of the SARE CoP training team. Call participants were asked to discuss one priority CoP topic area they wanted the most help with and how they wanted that help to be delivered (e.g., one-on-one consultation with VCP or others in network, group discussion, addition to resource guide, webinar, etc.) The other topic of discussion was use of the social networking site developed for this group, available at http://communitiesofpractice.ning.com/. The Ning site was developed to supplement the face-to-face and phone interactions and provide a forum for those interested in CoPs to communicate in an asynchronous learning environment. The site was developed partially in response to the regional pre-test survey results which show that half (49 percent) of the respondents said they currently participate in electronic social networking sites to inform their work (n=50), the majority of whom (72 percent; n=25) said that these sites were "somewhat valuable" One in four (24 percent) reported these sites were "quite" or "very" valuable. With this in mind, the CoP training team led by the Leopold Center created the above-referenced Ning site to provide participants access to print and social resources to support the creation and maintenance of food and agriculture related CoPs.

Although a special workshop for ISU Extension staff was not within the scope of the full proposal to SARE, the Leopold Center’s close relationship with ISU Extension and discussions with ISU Extension administrators made it a natural outgrowth of the work and mutually beneficial. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor were participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. With the passage of time and additional experience conducting the Regional workshop, the Leopold Center’s CoP training team once again offered interested Extension staff a workshop. To strengthen Extension’s community-based partnerships, the CoP training team issued invitations to Extension staff encouraging them to attend the workshop with a community-based partner with whom they were working or wanted to work. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented nonprofit social service organizations and the business community.

This report documents results of the evaluation of the April 2010 Extension CoP workshop. We developed two survey instruments adapted from instruments used to evaluate the Regional workshop; the first was an electronic pre-test survey designed in Survey Monkey, whose link was e-mailed to registrants about a week prior to their arrival. The second was a paper survey distributed at the end of the workshop. The purpose of the surveys was 1) to prepare participants to think about issues addressed by the workshop before coming and 2) to develop benchmark measures on knowledge, attitudes, and experiences we would use to compare with post-test workshop survey results and longer-term follow up surveys.

Respondents

The response rate to the post-workshop survey was 53 percent (we received 32 completed surveys). Of the 68 who initially registered, 8 did not show up for an attendance count of 60. Of the pre-registrants, 31 completed the pre-workshop survey for a response rate of 46%. For the purpose of this evaluation, we therefore had a total of 63 surveys upon which to base our analysis.

Extension personnel comprised 58% of respondents, many of whom were field specialists (38%), followed by regional extension directors (1 in 4) and state specialists (1 in 4). Not quite half of Extension respondents attending the workshop were based in Agriculture and Natural Resources (46.2%), while one in four (23%) were based in the area of Families and Consumer Science. Communities and economic development was the focus area of 15% of Extension respondents while the same proportion reported specialties in more than one Extension area.

The majority of respondents (67%) attended the workshop with a partner. Most of the non Extension partners attending represented nonprofit social service providers and the business community. Table 1 shows the occupational affiliation of partners (who could choose all affiliations that applied). More than half of the non Extension respondents attending the workshop were affiliated with community-based nonprofit social services (54%) including food banks, substance abuse centers, and various community resource centers. Government employees comprised 46% of non Extension partners attending the workshop. Nearly two in five (39%) are involved in either youth or adult education and 15% were either farmers or entrepreneurs.

Results

Respondents were motivated to attend the workshop mostly for content reasons: to learn about creating, facilitating, managing, and funding CoPs (51%) (Figure 1). However, answers to this open-ended question also yielded more interesting results: 36 percent reported they wanted to learn how to foster collaboration and how work together with other groups—in other words, they wanted to learn how to be good partners and learn about process. Another top reason for coming was to network with others (22%).

For the majority of respondents, the workshop met their expectations (78%) (Figure 2). For one in five, however, the workshop only met their expectations “somewhat” and one said it did not. Among those who selected “somewhat” or “no” responses regarding expectations, most said they still learned something but not all the material was new to them and they wanted more time to spend on it.

We evaluated workshop delivery and mechanics by asking respondents to rate the usefulness of each session using a five-point scale. Results are shown in Table 15. For the most part, respondents found the discussion and definition of CoPs most useful, followed by the small group exercise, which emphasized action steps necessary for focusing, initiating, managing, funding, and evaluating a CoP. The sessions on debriefing and branding were less useful to respondents.

Learning what information workshop participants find most significant is helpful for informing workshop designers and facilitators how well they gauged the needs and interest of their audience and whether the training team and respondents value the same information. To assess this, we included a straight-forward, open-ended question asking respondents to identify the most significant thing they learned. One in four (26.1%) identified as most significant the lesson on the importance of managing the CoP in terms of relationships and communication (Figure 3). The need to articulate common goals and partner with a variety of others with diverse backgrounds and perspectives were also significant lessons reported by nearly two in five respondents (17.4% each). Branding also surfaced, as did the distinction between different degrees of cooperation and the notion of nested networks (13% each). This is much aligned with the major thrust of the workshop, which emphasized the need for effectively engaging partners in collaborative work and maximizing collaborative outcomes.

Armed with new information, we were curious to know what respondents were planning to do with the information after attending the workshop. Although immediately after the workshop is still too early to tell, asking them about their plans provides benchmark data from which to compare information on actual action taken that we gathered in the longer-term follow-up survey, which was administered in December, 2010, eight months.

At the end of the workshop one in three planned to apply the concepts they had learned to existing collaborations to improve their work and partnerships (Figure 4). Fewer (one in four) were planning to initiate discussions with local stakeholders to consider starting new CoPs. Additionally, one in five planned to reach out to include new partners in their work. Although medium and long-term outcomes are virtually impossible to assess immediately after a workshop, administering a similar survey both before and after the workshop does enable us to measure short-term changes in attitudes and knowledge, especially if some questions remain identical in both surveys. This was a strategy we used to test the statistical significance of any short-term change that might have occurred as a result of participation in the workshop. Table 3 compares results from the pre-workshop survey with responses to the surveys distributed after the workshop. In both surveys, the questions were the same. We asked respondents to rate the extent to which they agreed with 23 statements about their attitudes toward and knowledge of CoP related issues. The table shows the pre-test survey mean of responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. T-statistics and p-values are also shown.

For every single statement, the post-test mean increased once the workshop had been conducted. Seventeen or 74% of responses to the 23 statements were statistically significant changes. However, six were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop "intervention" or "treatment." To interpret this in practical terms, take statement d) for example. Here, the chance that the mean would increase from 3.77 to 4.32 without the workshop intervention is 3 in 1000, strongly suggesting that the workshop is related to the increase. The lower the p-value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different.

The results in Table 16 show that important knowledge and attitude changes are attributable to the workshop. In summary, statistically significant changes among respondents include:

• Greater familiarization with the concept of CoPs and their benefits;
• Increased interest in starting and leading a new CoP;
• Increased interest in joining a CoP (but not necessarily leading);
• Greater sense that CoPs can help Extension (more efficiently) achieve its mission and goals;
• Stronger attitude that CoPs are a viable way to address key issues while better connecting respondents and their organizations to people they serve;
• Better understanding of different ways to fund and need to brand CoPs;
• Greater knowledge about the range of options for setting up a CoP; and
• Greater confidence in respondents’ respective organizations to have the capacity and resources to initiate and facilitate a CoP.

Statements that did not generate statistically significant results are worth examining as well. The mean level of agreement for the statement, “I am interested in applying concepts of the CoP model to work I am already doing” did not increase dramatically after the workshop because respondents generally agreed with this statement to start with (4.23 pre-workshop; 4.52 post-workshop). The same holds true for “I believe the CoP model can inform the work I do” (3.87 pre-workshop; 4.20 post-workshop). Before attending the workshop, respondents were already philosophically dedicated to working with multiple organizations and the need for working with partners from different sectors to address salient issues. Indeed, the role of Extension is to serve as a facilitator and catalyst within the communities and regions where based. And while there was an increase in respondents’ perception that different organizations in their area of work would be willing to actively engage in a CoP to address those issues, the difference was not significant (p=.066).

Workshop participants are clearly a collaboratively minded crew. Results in Table 3 above, as well as other results, provide evidence of this. Figure 5 provides further evidence, particularly as it relates to collaboration between Extension and their workshop partners. All responding partners reported a willingness to partner with Extension in the future (as indicated, additionally, by their mere presence at the workshop) and 91 percent report they have already partnered with Extension in the past. About half are currently partnering with Extension.

Asking participants to attend the workshop with a partner was a deliberate strategy on our part to strengthen existing relationships Extension staff already had in place with partners from their region or community. To this end, the workshop served its purpose. One of the benefits of attending the workshop with a partner was a better appreciation and understanding of partner organizations and resources (Figure 6). One in three workshop participants who attended with a partner also reported they learned how to collaborate better. Perhaps most valuable was the response from one respondent who said they learned they were not ready to play an active role in a Community of Practice yet. Sometimes CoP work is premature and knowing when the time is (or is not) right is just as important as understanding how to make it work.

Despite respondents’ obvious commitment to collaborative processes, there is always room for improvement. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before. Nearly one in five (18.2%) answering this question was considering working with producers, Extension, regional development groups, funders, and health-based groups. When we asked respondents what would be needed for their organization or business to fully participate in CoP work, 60 percent said first and foremost they needed administrative support and permission to spend time on pursuing and nurturing collaborative relationships. The participation of others and the better articulation of organizational goals were also needed (13.3%, respectively).

On April 19, 2010, the Leopold Center held a workshop for Iowa State University Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center's effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. The project is supported by the North Central Sustainable Agriculture Research and Education Professional Development Program. Sixty people attended this event, which followed a regional workshop attended by 63 partners from the North Central Region in July, 2009.

Although a special workshop for ISU Extension staff was not within the scope of the full proposal to SARE, the Leopold Center’s close relationship with Extension and discussions with ISU Extension administrators made it a natural outgrowth of the work and mutually beneficial. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor were participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. With the passage of time and additional experience conducting the Regional workshop, the Leopold Center’s CoP training team once again offered interested Extension staff a workshop. To strengthen Extension’s community-based partnerships, the CoP training team issued invitations to Extension staff encouraging them to attend the workshop with a community-based partner with whom they were working or wanted to work. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented non-profit social service organizations and the business community.

This report documents results of the evaluation of the 2010 Extension CoP workshop. We developed two survey instruments adapted from instruments used to evaluate the Regional workshop; the first was an electronic pre-test survey designed in Survey Monkey, whose link was e-mailed to registrants about a week prior to their arrival. The second was a paper survey distributed at the end of the workshop. The purpose of the surveys was 1) to prepare participants to think about issues addressed by the workshop before coming and 2) to develop benchmark measures on knowledge, attitudes, and experiences we would use to compare with post-test workshop survey results and longer-term follow up surveys.

Respondents

The response rate to the post-workshop survey was 53 percent (we received 32 completed surveys). Of the 68 who initially registered, 8 did not show up for an attendance count of 60. Of the pre-registrants, 31 completed the pre-workshop survey for a response rate of 46%. For the purpose of this evaluation, we therefore had a total of 63 surveys upon which to base our analysis.

Characteristics of Respondents/Participants

Table 17 shows that most survey respondents were women (71%), a figure that exceeds the proportion of women who actually attended the workshop (60%). Table 17 also shows the mean age and occupational affiliations of participants.

specialists (38%), followed by regional extension directors (1 in 4) and state specialists (1 in 4). Not quite half of Extension respondents attending the workshop were based in Agriculture and Natural Resources (46.2%), while one in four (23%) were based in the area of Families and Consumer Science. Communities and economic development was the focus area of 15% of Extension respondents while the same proportion reported specialties in more than one Extension area.

The majority of respondents (67%) attended the workshop with a partner. Most of the non Extension partners attending represented nonprofit social service providers and the business community. Table 1 shows the occupational affiliation of partners (who could choose all affiliations that applied). More than half of the non Extension respondents attending the workshop were affiliated with community-based nonprofit social services (54%) including food banks, substance abuse centers, and various community resource centers. Government employees comprised 46% of non Extension partners attending the workshop. Nearly two in five (39%) are involved in either youth or adult education and 15% were either farmers or entrepreneurs.

RESULTS

Respondents were motivated to attend the workshop mostly for content reasons: to learn about creating, facilitating, managing, and funding CoPs (51%) (Figure 7). However, answers to this open-ended question also yielded more interesting results: 36 percent reported they wanted to learn how to foster collaboration and how work together with other groups—in other words, they wanted to learn how to be good partners and learn about process. Another top reason for coming was to network with others (22%).

For the majority of respondents, the workshop met their expectations (78%) (Figure 8). For one in five, however, the workshop only met their expectations “somewhat” and one said it did not. Among those who selected “somewhat” or “no” responses regarding expectations, most said they still learned something but not all the material was new to them and they wanted more time to spend on it.

We evaluated workshop delivery and mechanics by asking respondents to rate the usefulness of each session using a five-point scale. Results are shown in Table 18. For the most part, respondents found the discussion and definition of CoPs most useful, followed by the small group exercise, which emphasized action steps necessary for focusing, initiating, managing, funding, and evaluating a CoP. The sessions on debriefing and branding were less useful to respondents.

Learning what information workshop participants find most significant is helpful for informing workshop designers and facilitators how well they gauged the needs and interest of their audience and whether the training team and respondents value the same information. To assess this, we included a straight-forward, open-ended question asking respondents to identify the most significant thing they learned. One in four (26.1%) identified as most significant the lesson on the importance of managing the CoP in terms of relationships and communication (Figure 8). The need to articulate common goals and partner with a variety of others with diverse backgrounds and perspectives were also significant lessons reported by nearly two in five respondents (17.4% each). Branding also surfaced, as did the distinction between different degrees of cooperation and the notion of nested networks (13% each). This is much aligned with the major thrust of the workshop, which emphasized the need for effectively

Armed with new information, we were curious to know what respondents were planning to do with the information after attending the workshop. Although immediately after the workshop is still too early to tell, asking them about their plans provides benchmark data from which to compare information on actual action taken that we gathered in the longer-term followup survey, which was administered in December, 2010, eight months.

At the end of the workshop one in three planned to apply the concepts they had learned to existing collaborations to improve their work and partnerships (Figure 9). Fewer (one in four) were planning to initiate discussions with local stakeholders to consider starting new CoPs. Additionally, one in five planned to reach out to include new partners in their work.

Although medium and long-term outcomes are virtually impossible to assess immediately after a workshop, administering a similar survey both before and after the workshop does enable us to measure short-term changes in attitudes and knowledge, especially if some questions remain identical in both surveys. This was a strategy we used to test the statistical significance of any short-term change that might have occurred as a result of participation in the workshop. Table 3 compares results from the pre-workshop survey with responses to the surveys distributed after the workshop. In both surveys, the questions were the same. We asked respondents to rate the extent to which they agreed with 23 statements about their attitudes toward and knowledge of CoP related issues. The table shows the pre-test survey mean of responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. T-statistics and p-values are also shown.

For every single statement, the post-test mean increased once the workshop had been conducted. Seventeen or 74% of responses to the 23 statements were statistically significant changes. However, six were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop "intervention&" or "treatment." To interpret this in practical terms, take statement d) for example. Here, the chance that the mean would increase from 3.77 to 4.32 without the workshop intervention is 3 in 1000, strongly suggesting that the workshop is related to the increase. The lower the p-value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different.

The results in Table 19 show that important knowledge and attitude changes are attributable to the workshop. In summary, statistically significant changes among respondents include:

• Greater familiarization with the concept of CoPs and their benefits;
• Increased interest in starting and leading a new CoP;
• Increased interest in joining a CoP (but not necessarily leading);
• Greater sense that CoPs can help Extension (more efficiently) achieve its mission and goals;
• Stronger attitude that CoPs are a viable way to address key issues while better connecting respondents and their organizations to people they serve;
• Better understanding of different ways to fund and need to brand CoPs;
• Greater knowledge about the range of options for setting up a CoP; and
• Greater confidence in respondents’ respective organizations to have the capacity and resources to initiate and facilitate a CoP.

Statements that did not generate statistically significant results are worth examining as well. The mean level of agreement for the statement, "I am interested in applying concepts of the CoP model to work I am already doing" did not increase dramatically after the workshop because respondents generally agreed with this statement to start with (4.23 pre-workshop; 4.52 post-workshop). The same holds true for "I believe the CoP model can inform the work I do" (3.87 pre-workshop; 4.20 post-workshop). Before attending the workshop, respondents were already philosophically dedicated to working with multiple organizations and the need for working with partners from different sectors to address salient issues. Indeed, the role of Extension is to serve as a facilitator and catalyst within the communities and regions where based. And while there was an increase in respondents’ perception that different organizations in their area of work would be willing to actively engage in a CoP to address those issues, the difference was not significant (p=.066).

Workshop participants are clearly a collaboratively minded crew. Results in Table 19 above, as well as other results, provide evidence of this. Figure 10 provides further evidence, particularly as it relates to collaboration between Extension and their workshop partners. All responding partners reported a willingness to partner with Extension in the future (as indicated, additionally, by their mere presence at the workshop) and 91 percent report they have already partnered with Extension in the past. About half are currently partnering with Extension.

Asking participants to attend the workshop with a partner was a deliberate strategy on our part to strengthen existing relationships Extension staff already had in place with partners from their region or community. To this end, the workshop served its purpose. One of the benefits of attending the workshop with a partner was a better appreciation and understanding of partner organizations and resources (Figure 11). One in three workshop participants who attended with a partner also reported they learned how to collaborate better. Perhaps most valuable was the response from one respondent who said they learned they were not ready to play an active role in a Community of Practice yet. Sometimes CoP work is premature and knowing when the time is (or is not) right is just as important as understanding how to make it work.

Despite respondents’ obvious commitment to collaborative processes, there is always room for improvement. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before. Nearly one in five (18.2%) answering this question was considering working with producers, Extension, regional development groups, funders, and health-based groups. When we asked respondents what would be needed for their organization or business to fully participate in CoP work, 60 percent said first and foremost they needed administrative support and permission to spend time on pursuing and nurturing collaborative relationships. The participation of others and the better articulation of organizational goals were also needed (13.3%, respectively).

Discussion and Conclusions

Community of Practice work is a natural fit for Extension and its partners. The level of understanding of this audience about the need for engaging diverse partners to address key issues in sustainable agriculture and create healthy communities, and the need for managing the resulting collaborative relationships that result from outreach, made it appear this training wasn’t needed for this group. We did not have to convince them of the value of this work. However, our challenge and their challenge lies beyond the workshop: To effectively engage critical partners who may be less enthusiastic about the difficult process work needed to improve their communities and landscapes. Even within the workshop, we discovered various levels of interest based on participation. Communities and Economic Development field staff and to a lesser extent, staff with expertise in Families and Consumer Science were less represented than those in Agriculture and Natural Resources. Given the relatively low participation from Families and Consumer Science, we were surprised at the variety of representation of community partners who accompanied Extension partners and the social service experiences they brought with them.

The CoP work of the Leopold Center and its partners in Iowa has focused largely on the area of food systems and sustainable agriculture. We were delighted to learn that the principles of our work could easily be transferable to another set of activities in the state that served the same or a similar purpose: to improve community health across the state. Apparently, the ease of transferability has to do with the process. The same general principles of the process hold true, regardless of the content of focus. Organizing and coordinating a working group comprised of different partners with different perspectives, experiences, and backgrounds offers the same opportunities and challenges, regardless of the topic—whether that topic is addiction, irrigation, immigration, or food access. What this work builds on is the tacit knowledge we have gained through experience and the tacit knowledge we have learned from others doing similar work. Essentially, the CoP workshop is a vehicle or structure for developing content about process, something that collaborative-minded individuals and groups are hungry for, most of which remains hidden in the realm of specialized journals or otherwise inaccessible publications. With this project, we tried to make tacit knowledge on process explicit and accessible to practitioners who have the capacity to lead this work forward and thus, creating positive change in sustainable agriculture or other arenas.

Based on the evaluation results, this workshop did not offer participants groundbreaking information that would change the course of their lives forever. For this already innovative group of collaborators, some with more experience than the training team, the workshop did serve to build confidence in their and their organization’s ability to do such work and reinforced what they knew about building collaborations. Evaluation results suggest the workshop formally validated those experiences, added new information to their portfolio, and increased interest in starting, joining, or leading a CoP. We expect the combination of these changes to lead to leadership changes in communities as Extension and its partners renew or continue their vows to work together and bring positive change to their communities.

In December 2010, we invited participants who had attended the North Central Regional Community of Practice (CoP) workshop in July, 2009 and ISU Extension personnel who attended the April, 2010 workshop to complete a brief electronic survey on progress they made or challenges they encountered related in their CoP work since attending the workshop. We hoped that with the passage of time (8 months in the case of the April 2010 workshop and a year and a half in the case of the July, 2009 workshop) we would be able to measure behavioral change as opposed to simply short-term changes in attitudes, skills, or knowledge.

Of the 117 recipients, we received 33 usable responses for a response rate of 28 percent. In terms of those who responded, most were women (61%). And while we received responses from participants in 7 of the 12 North Central states, most were from Iowa (65%). States NOT represented included North Dakota, Kansas, Michigan, Nebraska, and Indiana. In addition, half of the respondents worked for Cooperative Extension as state specialists, field specialists, or (former) county extension directors, with most (68%) working in the area of agriculture and natural resources. The other non Extension half represented non profits, the farming community or farm-based entrepreneurs, government, Resource Conservation and Development, education, research, and economic development.

The survey was designed to include a list of 12 possible actions respondents could have taken "in part as a result of attending the workshop." Respondents could also write in actions they took that were not included in the list. If respondents selected one or more actions, they were asked relevant follow up questions to explore their responses. For example, if they indicated they started or helped start a new CoP, they were asked what kind of CoP they started. In addition, respondents were also asked to describe what happened as a result of each action. For each indicated, respondents were asked to share why the action did not work IF it was not successful. From this line of questioning, we tallied and coded responses described in the middle and right hand columns in Table 20.

Summary and Discussion

Table 20 summarizes the medium term impacts of this project. Perhaps one of the most significant impacts of the workshops was demonstrating the importance and benefits of forging new partnerships. More than half (58%) of the respondents reported they internalized this message by actually reaching out to include new groups as partners. As a result, seven respondents said they development new partnerships with other organizations and one reported that doing so led to a major new initiative. Furthermore, a clear majority (79%) conveyed the message of including new partners to fellow colleagues within their own and nearly two in three (61%) went even further by encouraging others outside their organization to include new partners. The result of this effort was 9 new collaborations, an increase in funding for partners, applications for new grants, and greater support for their work.

We were also interested in learning whether new CoPs formed as a result of this project or existing ones were strengthened. More than one in three respondents to the survey (38%) joined or became more actively involved in an existing CoP while nearly one in four (24%) started or helped start a new CoP. Furthermore, one in three (34%) applied for a grant that included funding for development of a new group or enhancement of an existing group. More than half who applied for funding (55%) actually received a grant.

A composite snapshot of the actions listed in Table 21 shows that carefully designing and implementing an inclusive, collaborative process does indeed net tangible results. The medium-term follow up survey provides evidence that participants applied key CoP concepts to their work and changed behaviors and practices. Doing so yielded a variety of positive results, from gaining an expanded support base with new partners, to new funding, expanded programming, greater opportunities for sharing the work and decision making, greater project participation, and new leadership. Longer-term changes to the landscape and built environment trailed the list of impacts, but the mere fact that respondents actually reported some was most (and pleasantly) surprising given that only 8-18 months had elapsed since the workshop.

While the benefits are clear, there are still challenges to overcome. Some respondents reported they don’t always succeed in their CoP work because this work can be too demanding in terms of managing collaborative relationships, organizing effective leadership, finding funding, making time “to be active with other priorities,” and simply trying to do too much with slim-staffed organizations with a limited capacity to make things happen on shoestring budgets. Given these rather formidable obstacles, the successes reported by this group become all the more impressive.

• Table 5
• Table 7
• Table 9
• Table 14
• Figure 3
• Figure 4
• Figure 6
• Figure 7
• Table 18
• Table 21
• Table 6
• Table 12
• Figure 2
• Table 16
• Table 17
• Figure 9
• Table 20
• Figure 11
• Table 8
• Table 11
• Figure 1
• Figure 5
• Figure 8
• Table 19
• Figure 12
• Table 10
• Table 13
• Figure 10
• Figure 11
• Table 15