In a financial environment where training organizations are being asked more than ever to prove their worth, effective measurement is probably in the front of everyone’s minds. There are many ways to measure training, from the basic to the advanced. What are some of the analytics you can use to measure training, and more importantly, to measure its effectiveness?
On the basic level, training managers can determine how effective a particular program is by the numbers. How many people have attended a training program, whether in the classroom or online? A better measurement is to know the total number in the target population and compare the actual attendee numbers to that. Many times, an organization’s mandatory training, such as compliance-related courses, can be measured in this manner. Where the goal is 100% attendance, it’s easy to determine success. But is this measurement effective in a “bottom-line” environment? There are two ways to look at this measurement. From the training manager’s perspective, you can determine if a course was engaging and informative enough to keep people coming in. But from an overall view, is this information concrete enough to hand to senior executives? Probably not. But it’s a good start.
What about evaluation scores? There are two types of evaluation scores: participant evaluations of the course and their actual scores on tests and exams. You can look at both to get an idea of effectiveness. First impression course evaluations are sometimes very informative. They will give you an idea of how the course was perceived, whether the information and materials are viewed as useful, and if the instructor was knowledgeable and agreeable. Participant scores are also effective. For example, you can analyze a written test to determine if participants miss questions or activities in certain areas. Or, you can even determine if one question is missed more than others. This type of information will tell you if the course information is difficult to understand, if the instructor is knowledgeable in that subject matter, or if the question itself is misleading. Again, these are great analytics to use on an internal basis.
When it comes to information that is useful to the organization as a whole, you can utilize post-event evaluations. For example, you can survey the audience at a particular point in time after the event to determine if they used the information and were able to apply its concepts to their jobs. Possibly more effective is the surveying of the audience’s managers and supervisors after the event. This particular population has the benefit of observation, so they can see if a participant’s skills, attitudes, or behaviors have changed. The most important, and useful, information that may come from this type of analytic is a numbers based result: did errors decrease? Did customer satisfaction increase? Once you start reporting these results, they become useful to the organization and its management.
Along those lines, and in these economic times, direct financial and number impact of training is the most effective and most useful analytical information. This type of data can be reported to management as a concrete fact. For example, can you determine that training has had an impact on products sold? You may not necessarily be able to pinpoint the exact impact of training, but you can certainly say that the increase in products sold is a direct result of the quality training that you provide. Other ways to look for direct impacts on the business are through increases in income, efficiency, and customer satisfaction. You can look for decreases in errors, complaints, or turnover as examples of the effectiveness of training programs. The problem with this analytical approach is that management can point to any number of factors. For example, increases in products sold could be linked back to the introduction of a cutting-edge product that no competitor offers. How can the training department guard against this type of derailment? Be sure to analyze the situation before you create and deliver a training program. If you can prove that the audience was making a certain number of mistakes, selling a certain number of products, or turning over to a certain percentage, you can prove that training had an impact.
When you start talking about direct number impact of training, the next analytic that comes up is ROI, or Return on Investment. This is the actual, physical calculation of income gained from training. One of the most common training ROI calculations is to take the total benefit, subtract the cost, and divide by the total cost. The resulting number is multiplied by one hundred to determine ROI. The problem, though, is putting a dollar value on the total benefit factor. If you can do this with some certainty, try an ROI calculation. Otherwise, try to determine how training affected measurable factors such as products, income, efficiency, and turnover.
This is a broad overview of training analytics. Remember that your measure of training effectiveness should be performed to give you an “internal” idea of effectiveness first. Then, you should try to translate your training into measurable analytics that have a concrete meaning to the organization and its management.
© Copyright Bryant Nielson. All Rights Reserved.
