Wash your hands before entering production! Wear your hair net properly! We just had training on Good Manufacturing Practices (GMPs) last week! Wouldn’t it be nice to walk around a food manufacturing facility and not see GMP violations anywhere? In this fantasy world, all employees would inherently know how to manage their locker; they would know to not bring candy and gum on the production floor. But many of us are not able to live in this dream world and so we must train, train, train. But how do we know that the training is effective? Many food manufacturing facilities spend a tremendous amount of time developing and delivering training, but if competencies are not gained, how do we know that this money was well spent? If we do not evaluate the training, we won’t know if there was an issue with the material, the person giving the training or the motivation of the employees. Donald Kirkpatrick developed a very simple four-step model for evaluating training and learning experiences. Jack Phillips later modified the fourth step and added a fifth. Each step builds and depends upon the previous step, like a pyramid or a house structure.

Level 1 – Reaction

Level 2 – Learning

Level 3 – Application

Level 4 – Business Impact

Level 5 – ROI (return on investment)

Using the Kirkpatrick-Phillips model of evaluating training, let’s look at the steps in the context of GMP training. For our example, we will use a formal annual training event, but the steps can be modified for those informal or quick-learning experiences as well.

Level one or the “smile sheet” is the quick reaction to the training. On a scale of one to four, with one being completely unsatisfied and four being completely satisfied, answer the following questions: Was the training room comfortable and were the participants able to understand and hear the instructor(s)? Were the snacks acceptable? Was the training effective and do you feel that you can use this knowledge in your day-to-day activities? However, be careful to only ask questions that you have an impact to change. If there is no other location to have the training in and the employees don’t like the chairs, don’t ask that question. Employees will soon learn that even if they give their opinion, the training coordinators aren’t going to respond. This creates disillusionment with training, and the trainees are less likely to buy into the next session of training.

Level two asks whether learning took place during the class. A test or skills assessment of some sort is given at the end of the course. For example, if given the following color code, what color would you use to clean the drains? Product contact equipment? The participant answers the questions, and the facility keeps the answers on record as proof of the training. Level two is completed immediately after the class or before the participant leaves the training area. It is meant to quantitatively measure that learning occurred and skills were gained.  

Manufacturing facilities are typically good at completing levels one and two, either formally or informally, but that does not fully prove whether the employee internalized the training and is going to use it in his or her job (that comes with level three). Level three takes place after the class is over 30 to 90 days later. This level looks at the employee to determine whether he or she is using the GMPs that were taught. The food safety or quality management team conducts routine internal audits. These audits frequently contain GMP elements, such as are all employees wearing their hair nets appropriately? Yes or no, or no comment? Is the chlorine concentration adequate in the dip stations? These items are evaluated on a scheduled basis, and corrective actions are triggered if something is not satisfactory. However, we frequently do not draw the results from these audits back to the training effectiveness. A way to do this would be to compare how many nonconformities occurred in audits prior to training and how many nonconformities occurred after training.
The focus of level four is asking questions about the impact on the organization from the GMP training. For example, is there less downgrade from elevated microbiological contamination? Is sanitation time decreased? Is there a decrease in customer complaints? These are all questions that can be asked. Some of the positive results can be attributed to employees following their GMP training.

The last level is money, or specifically, return on investment (ROI). In this level of the evaluation process, we analyze the monetary benefits or costs of training. A quick equation to use is ROI=Program Benefits−Program Costs−Program Costs x 100.

Some program benefits could be savings from decreased downgrade or rework, increased product from spending less time on sanitation, etc. The program costs would be the money paid to the employees during the training, the cost of the trainer, the food provided, etc. Combining all of these elements together determines the actual costs and benefits of the training.  

We must have training, and it is very important that the training is evaluated for effectiveness to ensure that the company is getting an ROI of that time and effort. As we are constantly looking at ways to cut costs, it is very important to evaluate training and improve it as necessary. In food manufacturing, there is always room for continuous improvement, even in our training. Good luck and make sure you are wearing your beard net correctly! 

Janna Hamlett is an instructor in food science/quality assurance at the College of Southern Idaho.