James Kirkpatrick said the rationale behind changing the model developed by his father, the late Donald Kirkpatrick, is to look at desired outcomes early on in the training process. Through the 60 years it has been in existence, the model has, according to Wendy Kirkpatrick, been misrepresented. Trainers have tried to evaluate after the fact. Instead, evaluation should "be about shaping, maximizing, and influencing learning."
The New World Kirkpatrick Model combines new research and tried-and-true work from Donald Kirkpatrick. It marries people with metrics to get to business results, and five case studies help expound on how talent development professionals, including those in the public sector, can use the new model.
In one example, the Maryland Transit Administration undertook an extensive re-evaluation of its training after determining that the administration was facing increased accidents with new bus operator hires, even though it didn't appear that the training was different from before.
The MTA understood that training was not sufficient in reaching its mission. Having developed goals at the outset, the administration looked at four required drivers to maximize on-the-job application and results:
- Monitoring. This step was done through efficiency ride checks, a probationary period, and a performance dashboard.
- Reinforcing. Three ceremonies—rite of passage, passing the baton, and graduation—are part of this step, along with an emphasis that new hires are not alone. They received a rule book and job aids, along with other resources.
- Encouraging. Additional support is available from mentors, supervisor coaches, and opportunities for bus operators to reach out to instructors.
- Rewarding. Recognition on the MTA Facebook page, along with possibility of becoming an instructor or mentor, are part of this effort.
The effort also included the administration of a final exam and post-course data collection.
To test the learning, bus operator candidates' behind-the-wheel performance was measured during the training. After graduation, operators were monitored on their knowledge and skills by ride-along instructors. At Level 4, results, data about frequency of accidents, attendance records, and other information was entered in a learning management system. Bus operators involved in a preventable accident must attend training shortly after the accident.
This outcome not only relates to bus operators, but is about the organization as a whole, a point the Kirkpatricks make in their new book: "By definition, [Level 4] is some combination of the organizational purpose and mission, combined with the financial reality of sustained existence and success. In the for-profit company, it means profitably delivering the product or service to the marketplace. In a not-for-profit, government, or military organization, it means accomplishing the mission while responsibly using the resources available."
