While I was working toward my ATD Master Trainer certificate, my employer at the time asked me to develop a plan to evaluate the impact of our training program on the business. Feeling overwhelmed by the request since I had no experience in that area at the time, I decided to participate in the online Evaluating Learning Impact certificate course (utilizing the free course that came with the Master Trainer certificate course).
With the help of the instructor and my fellow participants, I learned what was needed to develop an evaluation plan. Most of the information made sense, but I found myself struggling to fully grasp what was needed to achieve good measurement at Levels 4 (Impact) and 5 (ROI) for our program. I became discouraged because I wasn’t sure what data was needed or even where to get it. As a result, we did not roll out the plan I developed.
That plan, however, was instrumental in helping me land my current position.
During the interview process, I asked the manager what goals she had for the position that went beyond the job description. One of those goals was to evaluate the impact of the training program. She noted that, historically, they had participants complete feedback forms at the end of course sessions and followed up with surveys several weeks later, but they hoped to get more information about the real impact they were having, given the investment the county was making in the training program. I was able to discuss what I learned in the online course and explored some ideas I had for evaluating the impact of the county’s training program.After beginning the new position, I was able to attend a brief seminar and half-day workshop on ROI and training impact presented by Dr. Jack Phillips, chairman of the ROI Institute, offered by the Kentuckiana Chapter of ATD. While the sessions gave an overview of all levels of training evaluation, they focused greatly on how to measure impact and how to determine the ROI of specific courses.
Following the workshop, for the first time I felt I understood what was needed to develop a full training evaluation plan—specifically at Levels 4 and 5—and crafted a plan for our training program. The first part of the plan included updating the questions asked (as well as some additional response options) in the follow-up surveys to gain a greater understanding of how participants are applying what they learned in class, the frequency of their use of the skills and knowledge learned, and any enablers or obstacles to helping them apply what they learned. The second part of the plan includes a change to what and how data and information is reported to provide specifically requested data, as well as newly gained information about impact.
While we beta-tested the “frequency of use” question on the surveys for one course in the second half of last year, we began using the new questions on all follow-up surveys for courses we have offered in 2019. We have already begun to get some significant feedback on how participants are applying what they have learned in the courses we've offered, as well as what is helping them apply the skills and knowledge and what sometimes gets in their way of applying them. I am excited to see the continued results and explore what we learn from them throughout the year ahead.
We will be sitting down with department leaders in the coming months to get their direct feedback about how the training courses are impacting their departments. In addition, we intend to review the data we have already gained regarding last year’s course offerings, the additional information we are gaining in the current year, as well as what we hope to look at in the coming years, and determine if there is specific data or information that they would like to see included in future reports.
Clearly, I have gone from feeling overwhelmed in regard to developing a training impact evaluation plan to being excited about what we will learn about the impact our training program is making throughout the various departments in the county.
It has also rebuilt my self-confidence in not only the area of training evaluation, but also in taking on bigger challenges and projects of this nature in the future.Thank you to ATD and to my colleagues in the Philadelphia, Cincinnati, Nashville, and Kentuckiana chapters (all of which I have been a member at different points over the past 10 years)! I owe a great debt of gratitude for the course offerings, webinars, chapter meetings, and other resources that have helped me grow in my career, improving my abilities as a training facilitator, but also growing me into a more well-rounded training professional.