A New Evaluation Model for the New Way We Work

Content

The recently released Kirkpatrick's Four Levels of Training Evaluation , from ATD Press, presents a new way for trainers to look at the well-known four levels of training evaluation. Rather than beginning with Level 1, reaction, and moving through to Level 4, results, the four levels are listed as results, behavior, learning, and reaction.

The recently released Kirkpatrick's Four Levels of Training Evaluation, from ATD Press, presents a new way for trainers to look at the well-known four levels of training evaluation. Rather than beginning with Level 1, reaction, and moving through to Level 4, results, the four levels are listed as results, behavior, learning, and reaction.

Content

James Kirkpatrick said the rationale behind changing the model developed by his father, the late Donald Kirkpatrick, is to look at desired outcomes early on in the training process. Through the 60 years it has been in existence, the model has, according to Wendy Kirkpatrick, been misrepresented. Trainers have tried to evaluate after the fact. Instead, evaluation should "be about shaping, maximizing, and influencing learning."

James Kirkpatrick said the rationale behind changing the model developed by his father, the late Donald Kirkpatrick, is to look at desired outcomes early on in the training process. Through the 60 years it has been in existence, the model has, according to Wendy Kirkpatrick, been misrepresented. Trainers have tried to evaluate after the fact. Instead, evaluation should "be about shaping, maximizing, and influencing learning."

Content

The New World Kirkpatrick Model combines new research and tried-and-true work from Donald Kirkpatrick. It marries people with metrics to get to business results, and five case studies help expound on how talent development professionals, including those in the public sector, can use the new model.

The New World Kirkpatrick Model combines new research and tried-and-true work from Donald Kirkpatrick. It marries people with metrics to get to business results, and five case studies help expound on how talent development professionals, including those in the public sector, can use the new model.

Content

In one example, the Maryland Transit Administration undertook an extensive re-evaluation of its training after determining that the administration was facing increased accidents with new bus operator hires, even though it didn't appear that the training was different from before.

In one example, the Maryland Transit Administration undertook an extensive re-evaluation of its training after determining that the administration was facing increased accidents with new bus operator hires, even though it didn't appear that the training was different from before.

Content

The MTA understood that training was not sufficient in reaching its mission. Having developed goals at the outset, the administration looked at four required drivers to maximize on-the-job application and results:

The MTA understood that training was not sufficient in reaching its mission. Having developed goals at the outset, the administration looked at four required drivers to maximize on-the-job application and results:

• Content

  Monitoring. This step was done through efficiency ride checks, a probationary period, and a performance dashboard.

  Monitoring. This step was done through efficiency ride checks, a probationary period, and a performance dashboard.

• Content

  Reinforcing. Three ceremonies—rite of passage, passing the baton, and graduation—are part of this step, along with an emphasis that new hires are not alone. They received a rule book and job aids, along with other resources.

  Reinforcing. Three ceremonies—rite of passage, passing the baton, and graduation—are part of this step, along with an emphasis that new hires are not alone. They received a rule book and job aids, along with other resources.

• Content

  Encouraging. Additional support is available from mentors, supervisor coaches, and opportunities for bus operators to reach out to instructors.

  Encouraging. Additional support is available from mentors, supervisor coaches, and opportunities for bus operators to reach out to instructors.

• Content

  Rewarding. Recognition on the MTA Facebook page, along with possibility of becoming an instructor or mentor, are part of this effort.

  Rewarding. Recognition on the MTA Facebook page, along with possibility of becoming an instructor or mentor, are part of this effort.

Content

The effort also included the administration of a final exam and post-course data collection.

The effort also included the administration of a final exam and post-course data collection.

Content

To test the learning, bus operator candidates' behind-the-wheel performance was measured during the training. After graduation, operators were monitored on their knowledge and skills by ride-along instructors. At Level 4, results, data about frequency of accidents, attendance records, and other information was entered in a learning management system. Bus operators involved in a preventable accident must attend training shortly after the accident.

To test the learning, bus operator candidates' behind-the-wheel performance was measured during the training. After graduation, operators were monitored on their knowledge and skills by ride-along instructors. At Level 4, results, data about frequency of accidents, attendance records, and other information was entered in a learning management system. Bus operators involved in a preventable accident must attend training shortly after the accident.

Content

This outcome not only relates to bus operators, but is about the organization as a whole, a point the Kirkpatricks make in their new book: "By definition, \[Level 4\] is some combination of the organizational purpose and mission, combined with the financial reality of sustained existence and success. In the for-profit company, it means profitably delivering the product or service to the marketplace. In a not-for-profit, government, or military organization, it means accomplishing the mission while responsibly using the resources available."

This outcome not only relates to bus operators, but is about the organization as a whole, a point the Kirkpatricks make in their new book: "By definition, \[Level 4\] is some combination of the organizational purpose and mission, combined with the financial reality of sustained existence and success. In the for-profit company, it means profitably delivering the product or service to the marketplace. In a not-for-profit, government, or military organization, it means accomplishing the mission while responsibly using the resources available."