2010 Annual Report for ENC08-101
Learning how to use communities of practice to address sustainable agriculture issues
Summary
On April 19, 2010, the Leopold Center held a workshop at the request Iowa State University (ISU) Extension for more than 60 ISU Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center's effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before.
Objectives/Performance Targets
This Value Chain Partnerships (VCP) project proposed to: (1) develop Community of Practice (CoP) curriculum and workshop materials; (2) pilot test CoP Workshop training and materials prior to its formal launch with 20-24 Iowa State University (ISU) Extension county and field staff participating; (3) provide the CoP Workshop in Year 1 to 48 Extension, agency, and/or non-profit educators and researchers in the north central states to address challenges in sustainable agriculture at a two-day workshop; (4) allow participants to participate in a community of practice meeting; (5) share all training materials on Communities of Practice on the web; (6) provide regular conference calls and creation of a listserv to provide networking and problem solving support; (7) develop a workshop in Year 2 where participants can share best practices as their new CoPs develop and evolve; (8) workshop materials and short presentations also will be shared on the web; (9) train an additional 120 Extension, agency, and/or non-profit educators and researchers in the north central states; and (10) evaluate activities using project surveys to measure short-term learning outcomes after each workshop in both Years 1 and 2. Year 2 evaluation activities will rely on face-to-face and/or telephone interviews to measure medium-term impacts of this project in terms of changes made by participants and representative organizations/agencies as a result of their participation.
Performance target 7, Community of Practice Workshop, Session II will take place April 14th and 15th, 2011 at the Memorial Union in Ames Iowa; currently twenty-five participants from nine states are registered for the workshop.
Accomplishments/Milestones
Leopold Center’s close relationship with Extension led to a special workshop for ISU Extension staff which was not within the scope of the full proposal to SARE. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor was participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. This session allowed ISU Extension staff to focus on the content and invite community partners to further disseminate the Community of Practice concept. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented non-profit social service organizations and the business community. The Value Chains Partnership website (www.valuechains.org) continues to provide information on many communities of practice and their partner organizations and their services and activities.
Impacts and Contributions/Outcomes
On April 19, 2010, the Leopold Center held a workshop for Iowa State University Extension staff and their community partners on creating and supporting Communities of Practice (CoPs) or working groups. The workshop was part of the Leopold Center's effort to increase capacity in the North Central states to initiate, manage, fund, and brand working groups to further work in sustainable agriculture. The project is supported by the North Central Sustainable Agriculture Research and Education Professional Development Program. Sixty people attended this event, which followed a regional workshop attended by 63 partners from the North Central Region in July, 2009.
Although a special workshop for ISU Extension staff was not within the scope of the full proposal to SARE, the Leopold Center’s close relationship with ISU Extension and discussions with ISU Extension administrators made it a natural outgrowth of the work and mutually beneficial. In April, 2009 a pilot CoP workshop was held for ISU Extension staff to test workshop materials and methods. Due to an announcement that day on the reorganization of Extension in Iowa, the workshop forged on but it was not well attended nor were participants well focused, given that some attendees and/or their colleagues learned they had just lost their jobs. With the passage of time and additional experience conducting the Regional workshop, the Leopold Center’s CoP training team once again offered interested Extension staff a workshop. To strengthen Extension’s community-based partnerships, the CoP training team issued invitations to Extension staff encouraging them to attend the workshop with a community-based partner with whom they were working or wanted to work. As a result, 18 workshop participants (30 percent) attended who were not affiliated with Extension, but instead represented nonprofit social service organizations and the business community.
This report documents results of the evaluation of the April 2010 Extension CoP workshop. We developed two survey instruments adapted from instruments used to evaluate the Regional workshop; the first was an electronic pre-test survey designed in Survey Monkey, whose link was e-mailed to registrants about a week prior to their arrival. The second was a paper survey distributed at the end of the workshop. The purpose of the surveys was 1) to prepare participants to think about issues addressed by the workshop before coming and 2) to develop benchmark measures on knowledge, attitudes, and experiences we would use to compare with post-test workshop survey results and longer-term follow up surveys.
The response rate to the post-workshop survey was 53 percent (we received 32 completed surveys). Of the 68 who initially registered, 8 did not show up for an attendance count of 60. Of the pre-registrants, 31 completed the pre-workshop survey for a response rate of 46%. For the purpose of this evaluation, we therefore had a total of 63 surveys upon which to base our analysis.
Characteristics of Respondents/Participants
Table 1 shows that most survey respondents were women (71%), a figure that exceeds the proportion of women who actually attended the workshop (60%). Table 1 also shows the mean age and occupational affiliations of participants.
Extension personnel comprised 58% of respondents, many of whom were field specialists (38%), followed by regional extension directors (1 in 4) and state specialists (1 in 4). Not quite half of Extension respondents attending the workshop were based in Agriculture and Natural Resources (46.2%), while one in four (23%) were based in the area of Families and Consumer Science. Communities and economic development was the focus area of 15% of Extension respondents while the same proportion reported specialties in more than one Extension area.
The majority of respondents (67%) attended the workshop with a partner. Most of the non Extension partners attending represented nonprofit social service providers and the business community. Table 1 shows the occupational affiliation of partners (who could choose all affiliations that applied). More than half of the non Extension respondents attending the workshop were affiliated with community-based nonprofit social services (54%) including food banks, substance abuse centers, and various community resource centers. Government employees comprised 46% of non Extension partners attending the workshop. Nearly two in five (39%) are involved in either youth or adult education and 15% were either farmers or entrepreneurs.
Respondents were motivated to attend the workshop mostly for content reasons: to learn about creating, facilitating, managing, and funding CoPs (51%) (Figure 1). However, answers to this open-ended question also yielded more interesting results: 36 percent reported they wanted to learn how to foster collaboration and how work together with other groups—in other words, they wanted to learn how to be good partners and learn about process. Another top reason for coming was to network with others (22%).
For the majority of respondents, the workshop met their expectations (78%) (Figure 2). For one in five, however, the workshop only met their expectations “somewhat” and one said it did not. Among those who selected “somewhat” or “no” responses regarding expectations, most said they still learned something but not all the material was new to them and they wanted more time to spend on it.
We evaluated workshop delivery and mechanics by asking respondents to rate the usefulness of each session using a five-point scale. Results are shown in Table 2. For the most part, respondents found the discussion and definition of CoPs most useful, followed by the small group exercise, which emphasized action steps necessary for focusing, initiating, managing, funding, and evaluating a CoP. The sessions on debriefing and branding were less useful to respondents.
Learning what information workshop participants find most significant is helpful for informing workshop designers and facilitators how well they gauged the needs and interest of their audience and whether the training team and respondents value the same information. To assess this, we included a straight-forward, open-ended question asking respondents to identify the most significant thing they learned. One in four (26.1%) identified as most significant the lesson on the importance of managing the CoP in terms of relationships and communication (Figure 3). The need to articulate common goals and partner with a variety of others with diverse backgrounds and perspectives were also significant lessons reported by nearly two in five respondents (17.4% each). Branding also surfaced, as did the distinction between different degrees of cooperation and the notion of nested networks (13% each). This is much aligned with the major thrust of the workshop, which emphasized the need for effectively engaging partners in collaborative work and maximizing collaborative outcomes.
Armed with new information, we were curious to know what respondents were planning to do with the information after attending the workshop. Although immediately after the workshop is still too early to tell, asking them about their plans provides benchmark data from which to compare information on actual action taken that we gathered in the longer-term follow-up survey, which was administered in December, 2010, eight months.
At the end of the workshop one in three planned to apply the concepts they had learned to existing collaborations to improve their work and partnerships (Figure 4). Fewer (one in four) were planning to initiate discussions with local stakeholders to consider starting new CoPs. Additionally, one in five planned to reach out to include new partners in their work.
Although medium and long-term outcomes are virtually impossible to assess immediately after a workshop, administering a similar survey both before and after the workshop does enable us to measure short-term changes in attitudes and knowledge, especially if some questions remain identical in both surveys. This was a strategy we used to test the statistical significance of any short-term change that might have occurred as a result of participation in the workshop. Table 3 compares results from the pre-workshop survey with responses to the surveys distributed after the workshop. In both surveys, the questions were the same. We asked respondents to rate the extent to which they agreed with 23 statements about their attitudes toward and knowledge of CoP related issues. The table shows the pre-test survey mean of responses on a 5-point scale (where 1=strongly disagree and 5=strongly agree) and the post-test survey means once the workshop had been conducted. T-statistics and p-values are also shown.
For every single statement, the post-test mean increased once the workshop had been conducted. Seventeen or 74% of responses to the 23 statements were statistically significant changes. However, six were not statistically significant at the p=.05 level, meaning that the increase in the means purely due to chance or error was high enough that we could not attribute knowledge and attitude changes to the workshop "intervention" or "treatment." To interpret this in practical terms, take statement d) for example. Here, the chance that the mean would increase from 3.77 to 4.32 without the workshop intervention is 3 in 1000, strongly suggesting that the workshop is related to the increase. The lower the p-value (i.e., closer to 0), the stronger the evidence we have that the means are indeed different.
The results in Table 3 show that important knowledge and attitude changes are attributable to the workshop. In summary, statistically significant changes among respondents include:
• Greater familiarization with the concept of CoPs and their benefits;
• Increased interest in starting and leading a new CoP;
• Increased interest in joining a CoP (but not necessarily leading);
• Greater sense that CoPs can help Extension (more efficiently) achieve its mission and goals;
• Stronger attitude that CoPs are a viable way to address key issues while better connecting respondents and their organizations to people they serve;
• Better understanding of different ways to fund and need to brand CoPs;
• Greater knowledge about the range of options for setting up a CoP; and
• Greater confidence in respondents’ respective organizations to have the capacity and resources to initiate and facilitate a CoP.
Statements that did not generate statistically significant results are worth examining as well. The mean level of agreement for the statement, “I am interested in applying concepts of the CoP model to work I am already doing” did not increase dramatically after the workshop because respondents generally agreed with this statement to start with (4.23 pre-workshop; 4.52 post-workshop). The same holds true for “I believe the CoP model can inform the work I do” (3.87 pre-workshop; 4.20 post-workshop). Before attending the workshop, respondents were already philosophically dedicated to working with multiple organizations and the need for working with partners from different sectors to address salient issues. Indeed, the role of Extension is to serve as a facilitator and catalyst within the communities and regions where based. And while there was an increase in respondents’ perception that different organizations in their area of work would be willing to actively engage in a CoP to address those issues, the difference was not significant (p=.066).
Workshop participants are clearly a collaboratively minded crew. Results in Table 3 above, as well as other results, provide evidence of this. Figure 5 provides further evidence, particularly as it relates to collaboration between Extension and their workshop partners. All responding partners reported a willingness to partner with Extension in the future (as indicated, additionally, by their mere presence at the workshop) and 91 percent report they have already partnered with Extension in the past. About half are currently partnering with Extension.
Asking participants to attend the workshop with a partner was a deliberate strategy on our part to strengthen existing relationships Extension staff already had in place with partners from their region or community. To this end, the workshop served its purpose. One of the benefits of attending the workshop with a partner was a better appreciation and understanding of partner organizations and resources (Figure 6). One in three workshop participants who attended with a partner also reported they learned how to collaborate better. Perhaps most valuable was the response from one respondent who said they learned they were not ready to play an active role in a Community of Practice yet. Sometimes CoP work is premature and knowing when the time is (or is not) right is just as important as understanding how to make it work.
Despite respondents’ obvious commitment to collaborative processes, there is always room for improvement. After the workshop, 90% of all respondents reported the workshop had prompted them to consider working with groups or individuals with whom they had never partnered before. Nearly one in five (18.2%) answering this question was considering working with producers, Extension, regional development groups, funders, and health-based groups. When we asked respondents what would be needed for their organization or business to fully participate in CoP work, 60 percent said first and foremost they needed administrative support and permission to spend time on pursuing and nurturing collaborative relationships. The participation of others and the better articulation of organizational goals were also needed (13.3%, respectively).
Community of Practice work is a natural fit for Extension and its partners. The level of understanding of this audience about the need for engaging diverse partners to address key issues in sustainable agriculture and create healthy communities, and the need for managing the resulting collaborative relationships that result from outreach, made it appear this training wasn’t needed for this group. We did not have to convince them of the value of this work. However, our challenge and their challenge lies beyond the workshop: To effectively engage critical partners who may be less enthusiastic about the difficult process work needed to improve their communities and landscapes. Even within the workshop, we discovered various levels of interest based on participation. Communities and Economic Development field staff and to a lesser extent, staff with expertise in Families and Consumer Science were less represented than those in Agriculture and Natural Resources. Given the relatively low participation from Families and Consumer Science, we were surprised at the variety of representation of community partners who accompanied Extension partners and the social service experiences they brought with them.
The CoP work of the Leopold Center and its partners in Iowa has focused largely on the area of food systems and sustainable agriculture. We were delighted to learn that the principles of our work could easily be transferrable to another set of activities in the state that served the same or a similar purpose: to improve community health across the state. Apparently, the ease of transferability has to do with the process. The same general principles of the process hold true, regardless of the content of focus. Organizing and coordinating a working group comprised of different partners with different perspectives, experiences, and backgrounds offers the same opportunities and challenges, regardless of the topic—whether that topic is drug addiction, irrigation, immigration, or food access. What this work builds on is the tacit knowledge we have gained through experience and the tacit knowledge we have learned from others doing similar work. Essentially, the CoP workshop is a vehicle or structure for developing content about process, something that collaborative-minded individuals and groups are hungry for, most of which remains hidden in the realm of sociological theory on organization in specialized journals or otherwise inaccessible publications. With this project, we tried to make tacit knowledge on process explicit and accessible to practitioners who have the capacity to lead this work forward and thus, creating positive change in sustainable agriculture or other arenas.
Based on the evaluation results, the workshop did serve to build confidence in their and their organization’s ability to do such work and reinforced what they knew about building collaborations. Evaluation results suggest the workshop formally validated those experiences, added new information to their portfolio, and increased interest in starting, joining, or leading a CoP. We expect the combination of these changes to lead to leadership changes in communities as Extension and its partners renew or continue their vows to work together and bring positive change to their communities.
In December 2010, we invited participants who had attended the North Central Regional Community of Practice (CoP) workshop in July, 2009 and ISU Extension personnel who attended the April, 2010 workshop to complete a brief electronic survey on progress they made or challenges they encountered related in their CoP work since attending the workshop. We hoped that with the passage of time (8 months in the case of the April 2010 workshop and a year and a half in the case of the July, 2009 workshop) we would be able to measure behavioral change as opposed to simply short-term changes in attitudes, skills, or knowledge.
Of the 117 recipients, we received 33 usable responses for a response rate of 28 percent. In terms of those who responded, most were women (61%). And while we received responses from participants in 7 of the 12 North Central states, most were from Iowa (65%). States NOT represented included North Dakota, Kansas, Michigan, Nebraska, and Indiana. In addition, half of the respondents worked for Cooperative Extension as state specialists, field specialists, or (former) county extension directors, with most (68%) working in the area of agriculture and natural resources. The other non Extension half represented non profits, the farming community or farm-based entrepreneurs, government, Resource Conservation and Development, education, research, and economic development.
The survey was designed to include a list of 12 possible actions respondents could have taken “in part as a result of attending the workshop.” Respondents could also write in actions they took that were not included in the list. If respondents selected one or more actions, they were asked relevant follow up questions to explore their responses. For example, if they indicated they started or helped start a new CoP, they were asked what kind of CoP they started. In addition, respondents were also asked to describe what happened as a result of each action. For each indicated, respondents were asked to share why the action did not work IF it was not successful. From this line of questioning, we tallied and coded responses described in the middle and right hand columns in Table 4.
Summary and Discussion
Table 4 summarizes the medium term impacts of this project. Perhaps one of the most significant impacts of the workshops was demonstrating the importance and benefits of forging new partnerships. More than half (58%) of the respondents reported they internalized this message by actually reaching out to include new groups as partners. As a result, seven respondents said they development new partnerships with other organizations and one reported that doing so led to a major new initiative. Furthermore, a clear majority (79%) conveyed the message of including new partners to fellow colleagues within their own and nearly two in three (61%) went even further by encouraging others outside their organization to include new partners. The result of this effort was 9 new collaborations, an increase in funding for partners, applications for new grants, and greater support for their work.
We were also interested in learning whether new CoPs formed as a result of this project or existing ones were strengthened. More than one in three respondents to the survey (38%) joined or became more actively involved in an existing CoP while nearly one in four (24%) started or helped start a new CoP. Furthermore, one in three (34%) applied for a grant that included funding for development of a new group or enhancement of an existing group. More than half who applied for funding (55%) actually received a grant.
A composite snapshot of the actions listed in Table 4 shows that carefully designing and implementing an inclusive, collaborative process does indeed net tangible results. The medium-term follow up survey provides evidence that participants applied key CoP concepts to their work and changed behaviors and practices. Doing so yielded a variety of positive results, from gaining an expanded support base with new partners, to new funding, expanded programming, greater opportunities for sharing the work and decision making, greater project participation, and new leadership. Longer-term changes to the landscape and built environment trailed the list of impacts, but the mere fact that respondents actually reported some was most (and pleasantly) surprising given that only 8-18 months had elapsed since the workshop.
While the benefits are clear, there are still challenges to overcome. Some respondents reported they don’t always succeed in their CoP work because this work can be too demanding in terms of managing collaborative relationships, organizing effective leadership, finding funding, making time “to be active with other priorities,” and simply trying to do too much with slim-staffed organizations with a limited capacity to make things happen on shoestring budgets. Given these rather formidable obstacles, the successes reported by this group become even more amplified.