Making Training Measurement and Evaluation Work

Every organization invests in programs, projects, and initiatives—training sessions, leadership development, change projects, employee engagement, and more. How do we know these efforts are achieving the desired results?

That’s where measurement and evaluation come in.

Benefits of Measurement and Evaluation

Done well, an evaluation doesn’t just show whether something was successful; it helps to make it successful. It offers feedback, guides improvements, delivers results that matter, and helps leaders make better decisions about where to spend time, energy, and resources.

Good evaluation ensures stakeholders that talent development professionals:

• Hold themselves accountable for the resources bestowed upon them.

• Use resources efficiently.

• Focus on program improvement, which leads to outcome improvement.

• Create and deliver value that matters most.

Yet, many talent development practitioners are satisfied with out-of-the-box end-of-course participant feedback surveys and quizzes that deliver little evidence that participants can apply what they learn. The question is “why?” Why does complacency prevail over vigilance, even after decades of research and practice demonstrating that good evaluation exists and the investment in it pays off?

Reasons Given for Not Measuring and Evaluating Training

Reasons talent development professionals give for not doing more than they could (or should) center around:

• Fear of the outcome

• Difficulty of the process

• Cost constraints

• Time constraints

Some talent development professionals are just waiting for senior leaders to ask for more.

Feel the Fear and Do It Anyway*

While good evaluation may reveal flaws in training design, delivery, and implementation, and it may be difficult, costly, and timely if you don’t know how to do it, what we know with relative certainty is: When the boss asks for meaningful results today, they will want to see those results tomorrow. So, while the “stay away” rhetoric may encourage delaying the pursuit of sound measurement and evaluation, the time to make measurement and evaluation work is now.

Making Measurement and Evaluation Work

So, what does it take to make measurement and evaluation work? There are five components that come together to create sound measurement and evaluation:

1. Framework of measures

2. Process model

3. Standards or guiding principles

4. Case application and practice

5. Implementation strategy

The framework is a categorization scheme that ensures measures that matter to all stakeholders are defined. Phillips' five-level evaluation framework is the most widely used; it includes measures important to designers, developers, and facilitators as well as the immediate supervisors, senior leaders, and board directors. Measures of participant reaction, learning, application, impact, and ROI are grounded in psychology, sociology, and economics, and when reported together, they allow for a holistic understanding of talent development value. In fact, this framework is applied beyond talent development, making it scalable across the organization.

Process models are important in that they show the steps required to ensure consistent implementation. W. Edwards Deming, recognized as the father of Total Quality Management, is known to have said, “If you can’t explain what you do as a process, you don’t know what you are doing.” Process models ensure talent development professionals can explain what they need to do, what they are doing, and what they did to get the results they report.

As Michael A. Lawson describes in ATD’s Handbook for Measuring and Evaluating Training, 2nd edition, professional standards provide a set of guideposts and guardrails. They ensure consistency in the implementation of a process. The process, coupled with the standards, ensures reliability in the data categorized in the framework.

While a framework, process model, and standards are important, they are only theoretical until they are put into practice. This is the fourth component—case application and practice. Here is where measurement and evaluation do their jobs. But, to make these processes work for the long-term requires a sustainable implementation strategy—the fifth component. This strategy includes a common way of working, a capable team, senior leader buy-in, a scalable approach, and a plan that balances the level of measurement with the importance, cost, and intent of talent development programs.

While senior executives, including CEOs, want to see impact and ROI data, it is not necessary to evaluate all programs at all levels. The table below provides the minimum percentage of programs to be evaluated to each level based on the types of programs offered through the typical large organizations. Programs that are costly, strategic, and operationally aligned are candidates for evaluation to impact and ROI.

Source: ROI Institute, Inc.

Your Measurement and Evaluation Practice

Sound measurement and evaluation need not be difficult, costly, or time-consuming. In fact, a good measurement and evaluation practice should only cost 3–5 percent of the total training budget. For that small amount, talent development professionals can reap all the benefits measurement and evaluation have to offer.

Use this checklist to assess whether your current M&E practice is sound, sustainable, and aligned with business goals. Mark each bulleted item you confidently meet.

Readiness and Mindset

• We recognize that evaluation is essential for proving and improving training value.

• We have moved beyond basic end-of-course surveys and quizzes as our only evaluation tools.

• We are not deterred by fear of negative outcomes—we use evaluation to drive program and process improvement.

• We do not wait for executives to request impact or ROI data—we lead with it.

Evaluation Framework

• We use a structured evaluation framework (e.g., Phillips’s Five Levels).

• We track Level 0 (Input) data for 100 percent of programs (e.g., attendance, cost).

• We measure Level 1 (Reaction & Planned Action) for all programs.

• We assess Level 2 (Learning) for at least 80–90 percent of our programs.

• We measure Level 3 (Application) for 30–40 percent of our programs.

• We assess Level 4 (Business Impact) for 10–20 percent of strategic programs.

• We calculate Level 5 (ROI) for 5–10 percent of high-cost or mission-critical programs.

Process Model

• We follow a documented step-by-step process to plan, collect, analyze, and report evaluation data.

• Everyone on our team understands the process and can explain how it works.

• The process is applied consistently across programs and teams.

Standards or Guiding Principles

• We follow internal or industry evaluation standards (e.g., ROI Institute, AEA, ISO).

• Our evaluation methods are consistent, credible, and defensible.

• We apply conservative assumptions and credible data sources in analysis.

Case Application and Practice

• We regularly apply the framework, process, and standards in real-world projects.

• We share case studies or internal reports that illustrate successful evaluation in action.

• Our team learns from experience and evolves our practice over time.

Implementation Strategy

• We have a sustainable strategy to grow and maintain measurement and evaluation.

• Our approach is embedded in the organization’s way of working.

• Our team is trained and capable of executing sound measurement and evaluation.

• We have senior leadership support for our evaluation efforts.

• We balance the level of measurement with the cost, visibility, and strategic importance of each program.

• We invest 3-5 percent of the training budget in measurement and evaluation activities.

Continuous Improvement and Communication

• We use evaluation results to improve program design and delivery.

• We communicate evaluation results in a compelling and actionable format.

• We report both tangible (e.g., cost savings) and intangible (e.g., morale) benefits.

• We use evaluation insights to guide future investments and decision making.

Did you find there is an opportunity to improve?

Check out ATD’s Handbook for Measuring and Evaluating Training, 2nd edition. This go-to resource offers how-to tips, practical advice, real-world examples, and tools you can use right away.

*Quote by Susan Jeffers; https://susanjeffers.com