Final report for LNE16-349
Project Information
The food safety documentation required by the Food Safety Modernization Act (FSMA) places a burden on small-scale cheesemakers that have only a few employees. Finding time and money for training and writing food safety documents is difficult, and most courses and resources are focused on larger processors and pasteurized products that do not meet the needs of small-scale, artisan cheesemakers. Amish cheesemakers are at a further disadvantage because they don’t have easy access to internet resources. To fill this gap, resources tailored to very small-scale and raw milk cheesemakers were developed to help them to comply with federal food safety regulations.
This project included a research component to develop the materials and an educational component to deliver the materials at in-person workshops. To ensure that the resources met the needs of the cheesemakers, they were developed and refined in collaboration with five raw milk cheesemakers, two of which were Amish. The research component hypothesized that engaging cheesemakers in the resource development would result in long term changes in sanitation practices and reduced microbial counts in the environment. The educational approach expected that cheesemakers would conduct their hazard analysis and develop preventive controls on their own within three months of attending a workshop and receiving tailored resources.
The resources developed in this project were a Guide to Implementing a Food Safety System, a Raw Milk Gouda Cheese Food Safety Plan Teaching Example, and the curriculum for the workshop, Conducting a Hazard Analysis and Science-based Risk Assessment and Developing Preventive Controls for Small-scale Cheesemakers. The workshop was held three times in different locations in Pennsylvania, and attended by 37 people from Pennsylvania, Vermont, New Jersey, New York, Maryland, and West Virginia.
During the project, we discovered that we had underestimated the level of fundamental food safety practices in place for these cheesemakers. The tools were revised to include much more background on Good Manufacturing Practices (GMPs), prerequisite programs (PPs), and basic sanitation, in addition to the information on conducting a hazard analysis. By collaborating with several cheesemakers, we observed practices we had not anticipated, such as the hand-carrying of wheels of cheese from the make room to the aging room in a different building on a sidewalk open to the environment. These observations allowed us to include different scenarios to the example food safety plan hazard analysis that may be unique to small-scale cheesemakers. We also noted several steps in the hazard analysis that constitute a multiple-hurdle approach to food safety for the cheesemakers making raw milk cheese, which does not have a kill step.
The hypothesized outcome in the research project of reduced microbial counts in the environment, and the education outcome of the workshop participants conducting their hazard analyses and writing preventive controls were not met within the timeframe of this project, due to the underestimation food safety practices already in place. However, the cheesemaker collaborators showed improvements in GMPs, sanitation practices and documentation, and several fixed problem areas (floors) in their facilities as a result of participating in this study. Likewise, the workshop participants had more written GMPs and sanitation practices in place three months after they attended the workshop based on the pre-workshop and follow up surveys. Cheesemakers indicated that the workshop materials, guide to implementing a food safety system, and example food safety plan were helpful as they work on their own to write documents to meet the federal food safety requirements.
Fifty-five raw milk cheesemakers conduct their own science-based risk assessment and create 2 process controls and 3 sanitation controls as a result of attending the workshop and participating in the research component. The cheesemakers save 20 hours in developing their FDA-required food safety plans by using the resources provided.
The enactment of the Food Safety Modernization Act (FSMA) places a time and economic burden on very small-scale cheesemakers to develop science-based risk assessments and food safety plans. Very small cheese plants often have only a few employees with many responsibilities so finding time and money for training and to write a risk-based food safety plan can be challenging.
Most food safety courses and resources are focused on larger processors and pasteurized products. Training programs focused on artisan cheesemakers are increasing, and there are some food safety resources written specifically for cheesemakers. Amish cheesemakers do not have easy access to internet resources that can help them with their food safety plans. A food safety needs assessment of Pennsylvania dairy processors combined with questions asked by artisan cheesemakers in dairy food safety workshops illustrate a clear need for more in-depth resources for conducting science-based risk assessments.
Syrko and Kaylegian (2015) reported that 40% of Pennsylvania dairy processors surveyed had less than 5 employees. The Pennsylvania Dairy Plant and Raw Milk Permit Directory lists 119 cheesemakers, of which 66 are making raw milk cheese and 11 of these are Plain Sect (Amish). There are about 300 artisan cheesemakers in the Northeast and Mid-Atlantic regions of the U.S.
Tools tailored to very small-scale cheesemakers will simplify the process and reduce the time needed to conduct science-based risk assessments and develop preventive controls to comply with new federal regulations. A step-by-step guide for conducting risk assessments in small-scale and raw milk cheese plants will be developed in conjunction with raw milk cheesemakers to ensure that it is written in a way that is helpful to them. A raw milk hazard guide will contain scientific references used to justify the handling of hazards in a raw milk cheese plant. Three workshops on "Conducting a Science-based Risk Assessment for Small-Scale Cheesemakers” will be held in various locations in Pennsylvania to provide information, including the above guides, and practice hands-on exercises.
The research portion of this project hypothesizes that a better engagement during the risk assessment and preventive controls development will result in long term improvements in sanitation practices and reduced microbial counts in the environment in raw milk cheese plants. Cheesemaker collaborators will participate in pre- and post-study questionnaires and environmental testing of their facility five times over the course of 1 year to validate the effectiveness of the educational tools and assess long term changes in behavior related to food safety.
Cooperators
Research
To save time in developing their food safety plans, raw milk cheesemakers use available templates and resources that may be designed for larger, pasteurized milk cheese facilities and may not best represent their own situations. The educational component of this project will create risk assessment tools that better address the unique needs of very small raw milk cheese plants. The research hypothesis is that using tools designed specifically for raw milk cheese plants will increase the cheesemaker's engagement and understanding during the risk assessment process and preventive controls development, resulting in long term changes in sanitation practices and reduced microbial counts in the environment.
Five cheesemakers in Pennsylvania evaluated their food safety systems and wrote supporting documents, including Good Manufacturing Practices (GMPs) and sanitation standard operating procedures. Two cheesemakers conducted a science-based risk assessment using the guides developed in the educational component of this proposal.
The extent of engagement and understanding of the risk assessment process, need for sanitation and process controls, and changes in sanitation practices were measured using pre- and post-study questionnaires. The questionnaires were conducted in-person and by mail. The impact of changes in sanitation practices was measured by monitoring the cheesemaking and aging environments for microbial indicators 3 times in the 12 months following the risk-assessment.
To get a baseline of sanitation practices, the researchers made an initial visit to each facility on a cheesemaking day to swab the environment for microbial indicators, and observe current cheesemaking and sanitation practices. The cheesemaking facility was swabbed for microbial indicators on a non-cheesemaking day for comparison.
To monitor long term changes in sanitation practices, each cheesemaking facility was measured for microbial indicators on a cheesemaking day at 3 months, 6 months, and 12 months following the risk assessment.
The microbial indicators measured the presence of ATP (3M ATP Clean-Trace Monitoring System), aerobic plate count (3M Petrifilm APC Plates), and listeria species (3M Petrifilm Environmental Listeria Plates), using the methods supplied by 3M (St. Paul, MN).
Each facility was evaluated at 15 sites for ATP and APC, and for 5 sites for listeria species, per visit. The choice of sampling sites depended on the layout of each facility, and will include the cheese room, aging room, milk delivery, and entrance points. The sampling sites were chosen based on common practice for environmental sampling and included sites that were identified as high-risk areas in the risk assessment. The sampling sites at each facility were appropriately documented to ensure that the same sites were sampled at each visit to allow for data comparison across the duration of the project.
Improvements in sanitation practices were determined by the reduction in microbial counts at each facility over the course of the project. Because each cheesemaking facility is different, specific values for microbial results were not compared across facilities, so the percentage change in microbial indicator counts was used to compare results across the different facilities.
The pre-study, self-assessment questionnaire was administered after the baseline sanitation observations and microbial swabs are collected. The post-study questionnaire was administered after the 12 month microbial swabs are collected.
2019
We anticipated that we would see a decrease in the environmental microbial counts over time for the five cheesemaker collaborators who participated in the research portion of this project. Analysis of the longitudinal data for each cheesemaker’s facility did not show this to be true for this study. We observed a range of values in ATP and APC counts and the presence/absence of listeria species within the same facility over time, with no consistent decrease or increase. We believe that the periodic environmental sampling protocol used in this study did not provide enough data to be useful for this group of cheesemakers, some of who were making cheese seasonally or only a few days a week.
However, the environmental data was useful to the cheesemakers by providing then with a better understanding of their facility and their current sanitation practices. The data helped identify areas that should be targeted for improved sanitation across all cheesemaking facilities, with the top areas being: drains, drain covers, floors, and squeegees. Several of the cheesemakers were surprised at the results of high counts over the course of the study and the fact that their current sanitation practices were not as effective as they thought. The data they received influenced three cheesemakers to make modification to their facilities to improve food safety such as reflooring the facility, or adding coving to provide better drainage in a corner, as shown in the pictures below.
A review of the sanitation records through the course of the study and the results of the post-study showed that cheesemakers did improve their sanitation practices. All cheesemakers had written, or were in the process of writing, a master sanitation schedule that included drain cleaning, and SSOPs to support regular and periodic cleaning.
The final documents were not available to the cheesemakers until after the research portion of the study was complete. The draft documents that the cheesemakers worked with did provide guidance in developing GMPs, PPs, and other elements in a food safety plan. By the end of the study several of the cheesemakers implemented preventive maintenance programs, supplier verification programs, recall plans, and visitor policies. At the start of the project one cheesemaker had an basic food safety and by the end of the study this food safety plan was more complete and a second cheesemaker had conducted their hazard analysis.
The final documents are now available free of charge on the Penn State Extension website (https://extension.psu.edu/food-safety-plans-for-small-scale-cheesemakers): The Penn State Extension Guide to Implementing a Food Safety System in Small-Scale and Raw Milk Cheese Plants and The Penn State Extension Food Safety Plan for Raw Milk Gouda Cheese Teaching Example.
2018
The project outlined 5 visits to each cheesemaker and in 2017 we expanded that to 7 visits to provide more information. Due to scheduling issues, we were able to complete 6 visits to each cheesemaker in the 12 month study period. The cheesemakers received the results of the environmental swabbing within 1 week, with a report indicating areas of improvement or areas of concern since the last visit and suggestions for improvement. These reports were well-received and helpful to the cheesemakers to focus their sanitation efforts
The data collection period has ended and data evaluation is in progress. As identified in the first several visits, problem areas continued to be drains, drain covers, floors, and squeegees. Preliminary review of the microbial data indicates that the counts did go down but these areas remain places that require more attention for regular sanitation.
The review of sanitation records at each visit showed that the cheesemakers had not implemented sanitation controls linked to their food safety plans, because 4 out of 5 cheesemakers did not complete their food safety plans to identify the sanitation controls needed. However, these cheesemaker did focus on improving their GMPs and we believe this was responsible for the reduced microbial counts in the cheesemaking environment. The identification of the lack of GMPs at the start of the study and the improvement in GMPs over the course of the study is a very positive outcome.
2017
Pre-study and post-study surveys are being used to determine the extent of engagement and understanding of the risk assessment process, need for sanitation and process controls, and changes in sanitation practices. The surveys were developed and approved by the Penn State Institutional Review Board. All five cheesemakers participated in the pre-study surveys, which were completed on the first visit to the cheesemaker’s facility. Selected comments provided by the cheesemakers in the pre-study survey were an interesting reflection of the current mindset of small-scale Pennsylvania cheesemakers:
- “The hardest part is to know where the hazards are.”
- “My biggest concern now is to know how to tackle this thing [preparing for FSMA]. It will need to be organized in a way that it takes very little time to implement. Basically a sheet with check lists [would be helpful].
- Survey question: Please list any pre-requisite programs your company has in place: Answer: “Not sure what kind of program this is”
- “It’s almost as though the regulations are designed to shut us all down.”
The project outlines 5 visits to collect environmental data, on both cheesemaking and non-cheesemaking days. During the refining of the experimental design we added 2 more visits over the 12 months to provide more information to the cheesemakers. The revised visit schedule is:
- Visit #1: Initial visit to introduce the project, determine swabbing site, and swab environment. This is on a non-cheesemaking day to allow time to interact with the cheesemakers.
- Visit #2: First visit on a cheesemaking day, within 2 weeks of the first visit. This visit was designed to be close to the first visit so that there was minimal time for cheesemakers to adjust sanitation practices from the first visit.
- Visit #3: Second visit on a non-cheesemaking day, approximately 1 month after the visit. This focus of this visit is the consultation with the cheesemaker on conducting their hazard analysis, as reported under Milestone 1, and to collect environmental swabs. The expectation is that conducting the hazard analysis will identify new sanitation controls or practices that are needed and will be implemented in the near future.
- Visit #4: Second visit on a cheesemaking day, 3 months from the visit to conduct the hazard analysis (3rd visit).
- Visit #5: Third visit on a cheesemaking day, 6 months from the visit to conduct the hazard analysis.
- Visit #6: Fourth visit on a cheesemaking day, 9 months from the visit to conduct the hazard analysis.
- Visit #7: Fifth visit on a cheesemaking day, 12 months from the visit to conduct the hazard analysis.
The first visits were completed at all 5 facilities. An assessment was made at each facility to choose the swabbing sites that will be followed throughout the project. Specific sites varied a little between facilities due to the different layouts of each cheesemaking operation, but there are site common to each facility. Maps were made of each facility with the sampling sites identified. Each facility has as least 15 total sampling sites, although not all are sampled for each indicator (APC, coliforms/E. coli, Listeria spp., ATP). Sampling sites include
- Cheese make room (whisk handle, mixing paddle, cheese harp, cheese draining mat, cheese press board, pH meter handles, underside of vat, floors, drains)
- Aging room (bottom of cooling condenser, brine, aging shelves, floor/wall juncture, drains)
- Receiving areas (milk tank outlet valves, swing lines, clean crate ready for cheese, floor under bulk tank)
- Packaging areas (cheese knife, cutting board, vacuum packager, scale buttons, surface of boxes)
- Cleaning areas (squeegees and floor brushes, hose handles and nozzles, sinks, hand soap)
- Transition areas (underneath foot baths, shoes & shoe storage, underneath doors, plastic sheeting
The second and third visits were conducted at each facility. The data from these three visits serve as a baseline for their environmental micro levels for each facility that can be used to monitor improvements over the next 12 months.
The swab results are shared with each cheesemaker as soon as they are available, usually 3-5 days after after the visit. The PI’s review the swab results with each cheesemaker and point out areas that are high in counts and areas of improvement since the last evaluation. Cheesemakers only receive the results for their own facility to maintain confidentiality.
We found that the cheesemakers really responded to receiving their micro results - they were interested to know the numbers, what the numbers meant, and where to focus their sanitation efforts. This is a powerful motivating tool to show cheesemakers the efficacy of their current sanitation practices. Several of the cheesemakers that thought the were doing adequate sanitation were surprised at the results that showed otherwise in some areas of their facilities. Some cheesemakers found that practices in their facility during long periods of no cheese making were causing increased counts compared with when they were making cheese regularly. Sites with high counts were drains, undersides of draining tables, and squeegees.
In addition to collecting environmental swabs, current sanitation records were reviewed at each visit from the perspective of what records exist and how often data is documented. Most of the cheesemakers did not have sanitation records and only recorded the sanitizer levels on their batch sheets. Only 1 cheesemaker out of 5 had written sanitation standard operating procedures (SSOPs).
We hypothesized that the use of tools developed specifically for small-scale cheesemakers would help them create their food safety plans and save time doing so. The cheesemakers in this study helped us to develop and refine a guide to implementing food safety systems and a food safety plan template, and these cheesemakers were given drafts of the documents to use during the course of this study. We discovered during the project that we had greatly underestimated the level of food safety fundamentals there were in place for these cheesemakers, which appeared to be similar for our workshop participants for the education part of this project. The tools were revised to include much more background on Good Manufacturing Practices (GMPs), prerequisite programs (PPs), and basic sanitation, in addition to the anticipated information on conducting a hazard analysis.
Long term decreases in microbial counts in these facilities was not observed during this study. However, cheesemakers learned how to use the environmental monitoring feedback they received to make changes to their facility and sanitation practices.
Education
Development of risk assessment tools tailored to very small-scale cheesemakers simplified the process, and reduced the time needed to conduct their science-based risk assessments and develop preventive controls to comply with new federal regulations. Two reference guides and one workshop were developed. The Raw Milk Gouda Cheese Food Safety Plan Teaching Example contains an example hazard analysis that cheesemakers can use to conduct their hazard analysis and justify the handling of hazards for their science-based risk assessments. The Guide for Implementing a Food Safety System in Small-Scale and Raw Milk Cheese Plants is a step-by-step, how-to guide, including information on Good Manufacturing Practices (GMPs) and sanitation practices. Five raw milk cheesemakers, included two Amish, assisted in the development of the guides to ensure that materials were presented in a way that is helpful and complete. The draft guides were be pilot-tested by the collaborators and then refined before it was made available to the cheesemaking public.
A Workshop on Conducting a Science-based Risk Assessment for Small-Scale Cheesemakers was developed to provide information and hands-on practice on conducting science-based risk assessments and developing preventive controls. The workshop curriculum included lectures on raw milk cheese hazards, writing product descriptions and flow charts, conducting a science-based risk assessments, and developing preventive controls. Participants practiced these skills in break-out sessions. Cheesemakers in the Northeast and Mid-Atlantic regions were invited to the workshops by announcements in Penn State Extension Dairy Foods newsletter, direct mailings, and through the Pennsylvania and other regional cheese guilds. Three workshops were held in different locations in Pennsylvania, and attended by 37 people from Pennsylvania, Vermont, New Jersey, New York, Maryland, and West Virginia.
Milestones
1. Five raw milk cheesemakers collaborators receive individual advice (consultation) on conducting risk assessments, raw milk cheese hazards, and developing risk-based preventive controls as they assist in developing educational and analytical tools for the larger cheesemaker audience.
5
5
April 30, 2017
Completed
October 31, 2017
The completion date of this milestone was delayed due to unexpected time to receive IRB approval for the use of human subjects for the surveys conducted for the whole project, which set the project initiation date back several months. Additional delays in implementing the project were encountered in coordinating the schedules of the PIs with the cheesemakers around their availability. The individual consultations with the cheesemakers were conducted in September and October, and the milestone was completed by October 31, 2017.
The draft raw milk hazard guide and supplemental questions to guide the cheesemakers in conducting their hazard analysis were completed prior to the the meetings with the cheesemakers. We anticipated that the meetings with the cheesemakers to provide advice on conducting their hazard analyses and risk assessments in order to identify the preventive controls that are needed for their food safety plan would take 2-3 hours for consultation and then we would remain on-site for 1-2 hours to answer questions and receive feedback as they worked on their hazard analyses using the draft guidance documents.
The meetings with most of the cheesemakers took 4-6 hours, and the time was spent in more basic explanation of FSMA, good manufacturing practices (GMPs), pre-requisite programs (PPs) for the whole facility and how the food safety plan focuses on the risks associated with manufacturing an individual food product. This was unexpected based on the knowledge that these cheesemakers had taken previous one day workshops on introduction to FSMA, GMPs and PPs, and several had multiple-day HACCP courses. This finding reiterated our concept that food safety programs are complex and difficult for small-scale cheesemakers to implement easily. Because we spent the time on the background material, we did not have time for them to begin the hazard analysis with us on-site to answer questions. We left the guidance materials and the hazard analysis as “homework” and asked for their feedback on the guidance materials within a two weeks.
All of the cheesemakers provided input on the draft materials that will help make the guidance documents more complete and useful to the small-scale and raw milk cheesemakers. Some comments included steps we didn’t think about in the manufacture of cheese that may introduce risk, such as hand carrying cheese wheels from the cheese make room to the aging room by hand where they could contact clothing or in tubs that may be stored on the floor and not completely sanitized prior to use. Other comments indicated that we needed:
- A better description of GMPs and PPs that need to be in place prior to doing the hazard analysis. We assumed that cheesemakers were already familiar with the preliminary steps prior to conducting the hazard analysis so this was not included in the scope of this project. However, this is not the case and is necessary to bring people up to speed so they are able to conduct an effective hazard analysis and risk assessment.
- A better checklist to prepare for conducting the hazard analysis, including a list of GMP and PP topics
- A glossary of terms used in FSMA documents and food safety plans
2. Five raw milk cheesemakers pilot test risk assessment guidelines, finalize their risk assessments and then develop and implement sanitation controls in their facilities.
5
5
January 31, 2018
Completed
March 31, 2019
2019
As noted in 2018, most of the cheesemakers in this part of the study did not have documented GMPs and PPs in place prior to the start of the project and therefore their focus was on these elements of their food safety plan rather than the hazard analysis and preventive controls. At the end of the study these new documents were in place:
- Employee training program – 1 cheesemaker
- Written SOPs – 1 cheesemaker
- Written SSOPs – 4 cheesemakers
- Master sanitation schedule – 3 cheesemakers
- Preventive maintenance program – 1 cheesemaker
- Hygienic zone map – 1 cheesemaker
- Hazard analysis – 2 cheesemakers
- Supplier verification program – 2 cheesemakers
- Recall plan – 1 cheesemaker
- Visitor logs – 2 cheesemakers
2018
As discovered and reported in 2017, many cheesemakers did not have documented GMPs and PPs in place prior to the start of this project. So while the cheesemakers continued to work on elements of the food safety plan, the focus was largely on documenting and implementing GMPs and PPs, and not so much on conducting the hazard analysis. One cheesemaker conducted a hazard analysis, three are still in progress, and one cheesemaker has not yet begun the hazard analysis.
Each dairy was visited six times and sanitation records were reviewed at each visit. Although the hazard analyses were not completed and specific Sanitation Controls identified in the hazard analyses were not written, cheesemakers continued to work on improving sanitation practices through writing, documenting and improved monitoring of GMPs and PPs. The visits were completed in Fall 2018 and the post-study survey is being mailed January 2019. The responses to the survey will identify specific sanitation controls implemented during the 12 month study period.
2017
The completion date of this milestone was delayed due to unexpected time to receive IRB approval for the use of human subjects for the surveys conducted for the whole project, which set the project initiation date back several months. Additional delays in implementing the project were encountered in coordinating the schedules of the PIs with the cheesemakers around their availability.
The cheesemakers have pilot tested the first draft of the risk assessment guideline documents, which was completed by November 2017. We are currently incorporating these changes and developing new documents identified by their feedback, as addressed in Milestone 1.
The time for the cheesemakers to finalize their risk assessments and then develop and implement sanitation controls in their facilities will be longer than expected. We anticipated that the cheesemakers had most of their GMPs and PPs in place and we could move ahead with the hazard analyses, and this was not the case. Consequently the cheesemakers are working on multiple aspects of food safety programs simultaneously and this will delay the completion of the final risk assessments. We have also discovered that these cheesemakers have difficulty finding time to be able to commit to a target date on the food safety program development because of the demand of their commercial production and maintaining their business and livelihood. We hope that this milestone will complete in Spring 2018.
In conjunction with Milestone 8, the cheesemakers are receiving the data from the environmental monitoring portion of this project and that is providing direction on where to focus their first efforts in developing and implementing sanitation controls.
3. Three hundred cheesemakers learn that the Raw Milk Cheese Hazard Guide and Guidelines for Conducting Science-based Risk Assessments in Raw Milk Cheese Plants are available from Penn State Extension through direct mailing and advertising by trade organizations.
300
1000
December 31, 2019
Completed
December 15, 2019
2019
The final documents were not completed and available on the Penn State Extension website until 10/1/19.The documents available free of charge on the Penn State Extension website, https://extension.psu.edu/food-safety-plans-for-small-scale-cheesemakers:
- Guide for Implementing a Food Safety System in Small-Scale and Raw Milk Cheese Plants
- Food Safety Plan for Raw Milk Gouda Cheese Teaching Example
A direct mailing is scheduled as part of the Penn State Extension Dairy Foods Newsletter in January 2020.
A press release titled “Extension offers tools to small-scale, raw-milk cheesemakers for safer products _ Penn State University” was issued from Penn State on December 11, 2019 (https://news.psu.edu/story/601466/2019/12/10/impact/extension-offers-tools-small-scale-raw-milk-cheesemakers-safer?utm_source=newswire&utm_medium=email&utm_term=601508_HTML&utm_content=12-10-2019-21-56&utm_campaign=Penn%20State%20Today). The number of recipients of this press release is unknown.
The Penn State press release was further published by Morning Ag Clips.com on December 12, 2019. The number of recipients of the Morning Ag Clips mailings is unknown.
The Penn State press release was further published by the American Cheese Society’s Weekly Roundup on December 13, 2019. This electronic newsletter is sent to their 2,365 member list.
2018
This milestone has been pushed back because significant changes were made to the Raw Milk Cheese Hazard Guide and Guidelines for Conducting Science-based Risk Assessments in Raw Milk Cheese Plants based on feedback from the workshops.
As observed with our cheesemaker collaborators, many of the cheesemakers attending the workshops did not have GMPs or PPs in place, and were unclear on the differences between a food safety system, which covers all practices in processing plant and a food safety plan that covers specific products. Thus, we thought it would be helpful to expand this document to be a basic instructional guide. The Penn State Guide to Implementing a Food Safety System in Small-scale and Raw Milk Cheese Plants covers these topics: introduction to FSMA, introduction to food safety systems, GMPs, PP, types of food hazards, hazards in cheesemaking, conducting the hazard analysis, writing preventive controls, recall plans, and resources.
The Raw Milk Hazard Guide was envisioned as a guidance document to support the hazard analysis decision and justification process. After the workshops, we felt that a more detailed document would be better and thus created the Penn State Food Safety Plan for Raw Milk Gouda Cheese, Selected Sections of a Food Safety Plan Teaching Example. This document is written in the form of a full food safety plan, using the template supplied by the Food Safety and Preventive Controls Alliance for developing teaching examples.
We decided that it would be good to have the documents peer reviewed by food safety experts, particularly those familiar with small-scale cheesemakers, prior to publication by Penn State Extension. The documents were sent out for peer review in the fall, and the reviews returned in December. We anticipate the documents to be finalized in early 2019, and once available in the Penn State Extension website will notify cheesemakers of their availability.
2017
This milestone has been delayed for the same reasons described in Milestone 1 above.
The feedback we received on the draft documents from the cheesemakers is leading to the development of additional documents (e.g., glossary, checklists) than were anticipated. All documents will be completed in time for the workshops in Milestone 5. We have decided to wait until all workshops are completed prior to releasing the final documents in case we determine other refinements are needed based on comments from workshop participants. We expect the documents to be finalized and available on the Penn State Extension website by late spring 2018.
4. Three hundred cheesemakers learn about workshops on Conducting Science-based Risk Assessments in Raw Milk Cheese Plants through direct mailings and advertising by trade organizations.
300
30
300
30
December 31, 2017
Completed
December 31, 2017
Two workshops are scheduled for 2018, one on January 31 in Lancaster, PA and one on February 28 in Chambersburg, PA.
An invitation to the workshop was sent in December 2017 by direct mailing (USPS) to small-scale cheesemakers in Pennsylvania, and by email to cheese guilds in Pennsylvania, New York, Vermont, Maine, Maryland, and Virginia. A copy of the invitation was included in the December issue of the Penn State Extension Dairy Foods Newsletter that was mailed to 553 people, including the Pennsylvania cheesemakers, state regulators and Penn State Extension Food Safety and Quality Team and Dairy Team.
The workshop is titled “A Workshop for Small-Scale Cheesemakers, Conducting Hazard Analyses & Science-Based Risk Assessments and Developing Preventive Controls.” We have replace the words “Raw Milk Cheesemakers” with “Small-Scale Cheesemakers” that is used in the grant proposal, because of the realization that many small-scale cheesemakers may be making pasteurized cheeses are in need of these materials and we didn’t want to exclude them by focusing on people that make only raw milk cheese.
5. Fifty cheesemakers attend workshops on Conducting Science-based Risk Assessments in Raw Milk Cheese Plants that provide information needed to meet FSMA regulations. They practice conducting a risk assessment and writing preventive controls in breakout sessions.
50
43
December 31, 2018
Completed
March 31, 2019
2019
The project workshops were completed in 2018, with a total of 37 people from 25 creameries attending. Because we had extra materials that we had prepared, we held an additional workshop in March 2019. This workshop was outside of the scope of the IRB approved survey data collection period and therefore no surveys were administered, but it was felt that dissemination of the information was important enough to warrant the additional workshop without collecting research data. Six cheesemakers from five creameries attended. As with the other workshops, the need for fundamental GMP and sanitation information was greatly needed and we spent more time on these subjects than on practicing writing hazard analyses and preventive controls.
2018
Three workshops were conducted in 2018, in Lancaster (January), Chambersburg (February), and University Park (April), PA. A total of 37 people from 25 creameries attended the workshop. Several people that registered cancelled at the last minute or were no-shows. The intention of the workshops was to focus on the hazard analysis and writing preventive controls, but we discovered many cheesemakers were confused about what FSMA and how GMPs, PPs, food safety plans, and food safety systems all fit together. Consequently, we spent more lecture and discussion time on these areas, and not as much time on the practical sessions of conducting hazard analyses and writing preventive controls. The was an important learning from the workshops, and we used this information to modify the documents we were creating to include more basic information of food safety systems as a whole.
We printed workshop materials for 50 participants at one time for cost saving purposes. We gave each creamery a set of materials rather than each person, so we have 25 sets of materials remaining. We have scheduled another workshop for March 2019 at University Park, with support from the Pennsylvania Cheese Guild. Cheesemakers that are exempt from some of the FSMA requirements are beginning to realize they will still need to do a hazard analysis, so are cheesemakers that need this information. This workshop will not be part of the research project in that we will not be conducting surveys of the cheesemakers. Any additional expenses for this workshop, such as lunch, will be covered through a modest registration fee ($15-20). Another learning from the initial workshops was that when a workshop has no fees people don’t have as much commitment to attend, so a modest fee will be charged for this workshop.
2017
Two workshops are scheduled for 2018, one on January 31 in Lancaster, PA and one on February 28 in Chambersburg, PA. Invitations were mailed in December 2017, and registration is ongoing.
Materials for the workshop are in development. The agenda for the workshops is:
- Introduction to the Food Safety Modernization Act
- Intro to GMPs, PPs, and Food Safety Plans
- Dairy Hazards in Cheesemaking
- Understanding Process, Allergen, Sanitation, and Supply-Chain Preventive Controls
- Conducting a Hazard Analysis, Assessing Risk and Determining Appropriate Preventive Controls
- Practice Session—Doing a Hazard Analysis
- Writing Preventive Controls
- Bring It All Together—Where Do I Start?
The proposal indicates that 3 workshops will be held, and we currently have 2 scheduled. Based on registrations at the first 2 workshops, the date and location of the 3rd workshop will be scheduled. The workshops are being held early in 2018 to allow cheesemakers ample time to work on their food safety plans prior to the FSMA deadline date of September 2018.
6. Fifty cheesemakers conduct risk assessments in their own facility, and write and implement preventive sanitation controls, following the workshops outlined in Milestone 5.
50
37
December 31, 2018
Completed
April 30, 2019
2019
Despite a written commitment from the 37 people attending the workshops during the research data collection period, only 18 people completed the post-workshop surveys. Two people that attended the workshop had left their positions with the creamery. An additional 6 people attended a workshop in March 2019, which was after the data collection period and therefore surveys were not conducted for this workshop.
Data from the returned survey showed that cheesemakers did make progress on their hazard analysis, HACCP and food safety plans, and sanitation and process preventive controls, but not at the level we expected. From the 18 returned surveys, one cheesemaker reported they had conducted a hazard analysis prior to the workshop and 7 cheesemakers reported their hazard analysis was in progress. In the post-workshop survey, 4 people had conducted the hazard analysis, 11 were in progress. Two cheesemakers reported that they had implemented a HACCP plan prior to the workshop and 3 cheesemakers said they were in progress with a HACCP plan and 6 cheesemakers were in progress with a food safety plan. In the follow-up survey these numbers increased to 4 cheesemakers that had conducted the hazard analysis, and 11 were in progress, and 2 reported having a HACCP plan with 9 in progress, and 2 cheesemakers reported a food safety plan and 11 were in progress. These numbers were reasonable considering we discovered that many of the cheesemakers attending the workshops were lacking in documented GMPs and basic sanitation procedures. Following the workshop there was an increase in the number of cheesemakers that had or were in the process of writing GMPs, SSOPs, and master sanitation schedules.
The fact that we did not see more cheesemakers conduct their hazard analyses, implement food safety plans, and improve their basic GMP and sanitation practices is likely due to the difficulty in finding the time to write these documents. Half of the cheesemakers comments in the survey that time was a major challenge. At the beginning of the workshop 14 cheesemakers reported they agreed or strongly agreed that they were overwhelmed by food safety requirements, and 12 reported the same in the post-workshop survey. Of the 18 returned post-workshop surveys, 16 people reported they were more confident as a result of the workshop and the materials made conducted a hazard analysis easier, and 12 cheesemakers reported the workshop reduced stress associated with writing food safety plans.
2018
Milestones 6 and 7 have been combined. Milestone 6 stated that cheesemakers will conduct risk assessments and write preventive controls, and Milestone 7 is the documentation of this activity through a verification survey. The only way report the progress of Milestone 6 is by Milestone 7, and this was an error in the grant proposal.
Three-month follow-up surveys were sent to cheesemakers following the workshops held in January, February, and April. We had about a 60% response rate, despite a commitment from the participants to complete the surveys. A second survey was sent to the cheesemakers to try to capture this data, but not until December, due to time constraints. Data analysis of the survey data will begin in February 2019.
7. Fifty cheesemakers document the creation and implementation of food safety plans by returning a completed verification survey, following the workshops outlined in Milestone 5.
50
18
March 31, 2019
Completed
April 30, 2019
2019
See results in Milestone 6.
2018
Milestone 7 has been combined with Milestone 6, see explanation above.
2017
Three surveys for the workshop were developed and approved by the Penn State Institutional Review Board: pre-workshop, post-workshop (same day), and 3 month follow-up survey.
8. Five cheesemakers learn the effectiveness of their environmental sanitation controls by reviewing the microbial test results collected for 12 months.
5
5
January 31, 2019
Completed
March 01, 2019
2019
Cheesemakers received the results of the their visits during the course of the study period and questions about the results and any recommendations provided were answered during this time. A final letter was mailed to each cheesemaker on March 1, 2019 that summarized the dates of the visits, and a summary of the Listeria species detected in the environment and areas of high counts over the course of the study. Improvements made to the facilities and in sanitation and food safety documentation was summarized, as well as suggested areas for improvement in documents as required by FSMA.
2018
A total of six visits to each cheesemaker were made over the 12 month period from August/September 2017 to 2018. We anticipated that seven visits would be made, but due to scheduling issues we were able to coordinate six visits with each cheese makers. Cheesemakers received a short report with their micro results indicating areas of progress or concern from the previous visit, and suggestions for improvement. The PIs were available to answer questions as they arose. Floors, drains, drain covers, and squeegees were areas of high concern for Listeria spp. and E. coli contamination in multiple facilities.
Improvements in microbial environmental monitoring indicators over the 12 month period were usually the result of improvements in GMPs rather than from sanitation controls because most of the cheesemakers did not conduct the hazard analysis and implement sanitation controls during the study period.
2017
A total of seven visits to collect environmental swabs are planned for each facility over 12 months. The environmental swab protocol is described in the research section of this report. In summary, each facility is sampled at a minimum of 15 sites in the cheese make room, aging room, receiving areas, packaging areas, cleaning areas and transition areas.
The first three visits were conducted at each facility.
The swab results are shared with each cheesemaker as soon as they are available, usually 3-5 days after after the visit. The PI’s review the swab results with each cheesemaker and point out areas that are high in counts and areas of improvement since the last evaluation. Cheesemakers only receive the results for their own facility to maintain confidentiality.
9. Five cheesemakers document time savings and food safety knowledge gained by participating in the development and pilot testing of risk assessment guides and environmental sampling of their facilities by completing pre-study and post-study surveys.
5
5
February 28, 2019
Completed
April 30, 2019
2019
The results from the post-study survey of these cheesemakers indicated time saved on food safety plans as a resulted of participating in this project ranged from 6-10 hours (1 cheesemaker), 11-20 hours (1 cheesemaker) to over 30 hours (1 cheesemaker).
At the beginning of the study 1 cheesemaker identified themselves as having basic food safety knowledge and 4 cheesemakers identified as having intermediate knowledge. By the end of the study, 2 cheesemakers identified at the intermediate level and 2 at the advanced level; 1 cheesemaker did not answer this question.
As a results of participating in this study, 3 of the 5 cheesemakers agreed or strongly agreed that they were more comfortable with food safety plans, and 4 agreed that the materials made doing the hazard analysis easier and reduced the stress associated with these tasks.
2018
The post-study survey was intended to be mailed mid-November, but due to time constraints the mailing date was moved to January 2019 so that the surveys would not be overlooked in the mail or put aside by the cheesemakers during the holidays.
2017
The pre-study and post-study surverys were developed and approved by the Penn State Instidutional Review Board.
All 5 cheesemakers participated in the pre-study survey. This data will be compared to the post-study survey at the end of the 12 month research period.
Milestone Activities and Participation Summary
Educational activities:
Participation Summary:
Learning Outcomes
A pre- and post-study (12 months) survey was conducted with the 5 cheesemaker collaborators and a pre- and post-workshop (3 months) survey was conducted with 37 workshop participants, and 18 workshop participants responded to the survey. The survey assessed company demographics, and knowledge, attitudes, and practices on food safety topics including Good Manufacturing Practices (GMPs), sanitation and sanitation operating procedures (SSOPs), hazard analysis, HACCP and food safety plans, and process, sanitation, and allergen preventive controls, and employee training.
Performance Target Outcomes
Target #1
50
In the course of developing their FSMA-required Food Safety Plan, they will make 2 process controls and 3 sanitation controls.
23
Documented food safety related procedures.
The cheesemaker participants made different changes based on their level of GMP and food safety plan documentation at the start of the study. Changes included the addition of written GMPs, employee training records, sanitation operating procedures (SSOPs), master sanitation schedules, hazard analyses, HACCP plans, food safety plans, sanitation, allergen and process preventive controls, supplier verification plans, and recall plans.
The achievement of the performance target was verified by a pre- and post-study (12 months) survey conducted with the 5 cheesemaker collaborators and a pre- and post-workshop (3 months) survey conducted with 37 workshop participants. There were 6 cheesemakers that participated in a workshop that was held outside of the data collection period that were not included in the surveys. Eighteen of 37 workshops participants returned their post-study surveys after 2 attempts to collect the data. We expected that the 55 participants would conduct a hazard analysis and implement 2 process and 3 sanitation controls, and save 20 hours in the process of doing this as a result of participating in the research project and workshops.
Despite the fact that the federal FSMA rules require all food processors to have GMPs, employee training, and other basic food safety elements in place, we found from our study that this is not always the case. The cheesemakers we worked with throughout this study are exempt from the full food safety plans per FSMA regulations, and many of them did not realize that they still needed to have basic procedures in place for GMPs and other processing operations, therefore our focused changed to address this knowledge gap. Cheesemakers commented that they often do not get sufficient information from their regulators, who, technically, are in their facilities to regulate not educate. Many cheesemakers just don’t have the time to seek out regulatory requirements, thoroughly understand them, document and implement them. One comment from the workshops stood out as to how much we still need to impress upon this segment of food processors that food safety is a serious issue. The comment was “I don’t have time to record all of the incoming ingredient lot numbers and track make data and where my cheese goes for sale.” When this person was asked what they would do in case of a recall or someone getting sick or dying from their cheese, they had a blank look and no answer. It is paramount to keep this motivation and access to resources moving forward.
From the workshop surveys we found that the tools we developed were helping people to understand food safety concepts, conduct their hazard analyses and implement practices and controls to improve the safety of their cheeses. Cheesemakers reported that access to the workshops and resource materials helped them save time and reduced stress associated with developing their food safety documents. The amount of time save ranged from 6 to over 30 hours for individual cheesemakers, the lack of consensus on time saved for the hazard analysis is consistent with the fact that everyone is a different level of developing their food safety systems.
Additional Project Outcomes
During the course of these study we realized there was lack of general understanding of the foundations of food safety – GMPs and sanitation, which led to the development of a more detailed guide to implementing food safety systems and example food safety plan than was anticipated. Another key finding from this study was that the small-scale cheesemakers did not have a lot of background in environmental testing and sanitation practices. The researchers expected that processing areas such as drains and floors would be problem areas, but the cheesemakers did not and thought their current practices were sufficiently cleaning their facilities. The environmental testing we conducted further confirmed these were problem areas and the feedback to the cheesemakers helped them improve practices. Because this was not inherent information for the cheesemakers, we are in the process of publishing an Extension Fact Sheet on the sanitation practices and target areas.
Interactions with cheesemakers throughout this project made it clear that there is a great need for basic food safety training for small-scale cheesemakers. At Penn State, there has been a one-day workshop titled Dairy Basics that has changed curriculum on and off for the last 10 years to meet the needs of the intended audience. Based on the workshop learnings in this project, the Dairy Basics workshop was modified to be a one-day workshop called Dairy Basics: Fundamentals of Quality and Safety that includes information about milk composition, milk microbiology, GMPs, sanitation principles, and writing SSOPs. A second one-day workshop given in tandem with Dairy Basics was created from the project workshop, but expanded for all small-scale dairy processors (cheese, milk, cultured products, ice cream) to meet the growing need for this information. The workshop is now called Hazard Analysis and Preventive Controls for Small-Scale Dairy Processors and includes lectures on FSMA, a review of GMPs, hazards in dairy foods, understanding preventive controls, conducting a hazard analysis and determining preventive controls, writing preventive controls and writing recall plans. Both workshops include time for practical sessions for writing GMPs, SSOP, hazard analyses and preventive controls as appropriate to each workshop. This set of two one-day workshops will be given twice a year at Penn State and was first offered in October 2019, with the next ones scheduled for April 2020.
A workshop and two guidance documents were developed specifically for small-scale and raw milk cheesemakers to help them conduct hazard analyses, develop preventive controls and implement a food safety system. Three Amish cheesemakers who had limited background in the current food safety regulatory requirements participated in the March 2019 workshop . As a results of attending the workshop they formed a working group of seven Amish processors to write elements of their food safety plans, including Good Manufacturing Practices (GMPs), sanitation standard operating procedures (SSOPs), hazard analyses, and controls to reduce hazards in their facilities. This group consists of cheesemakers, milk, and yogurt processors, and is led by an “English” (non-Amish) Preventive Controls Qualified Individual (PCQI) who took the Preventive Controls for Human Food – Dairy Foods Processing course at Penn State in 2018. The group has assignments of documents to write and at the meetings the PCQI leads discussions and provides input. The workshop and guidance document developed in this project are helping them write their procedures and documents to comply with FSMA food safety regulations. The group has taken this task seriously and formalized their activities by including a mission statement and taking minutes of their meetings. When they have completed their food safety plans and supporting documents, they intent to expand their group model to help others in the Amish community develop their own documents to meet these requirements.
Direct participation with the cheesemakers in development of the guide to implementing food safety systems and the raw milk gouda food safety plan teaching example was invaluable. Through discussions with them about their practices, questions they asked of us, and by direct observation of their practices we discovered things about cheesemaking at this scale that we never would have thought about. We did not realize the lack of fundamental knowledge of food safety practices so we changed our focus a bit to cover this gap. Another notable observation was in the movement of cheese wheels from the make room to the aging room through environmentally uncontrolled space. This was something we never even thought might be occurring – in one instance going through a hallway in a shared farm-based facility and in another instance going outside to a different building, while hand-carrying cheese. This observation led us to add information to both of our documents that will cue cheesemakers to identify these practices, determine the risk they might pose, and develop ways to reduce this food safety risk. Thus, working directly with the target audience to adequately understand their practices is highly valuable and recommended when developing m such as these.