ATD developed the talent development framework and puzzle to help practitioners understand the different components of talent development, and how organizations can build their own frameworks to address their unique needs. This blog series explores each component of the talent development framework, why it’s important to the field, and what resources ATD offers to practitioners who want to learn more.
Evaluating Learning Impact
Evaluating learning impact involves the use of learning metrics and analytics to measure the effectiveness of learning solutions. In Utilization Focused Evaluation: The New Century Text, Michael Quinn Patton defines evaluation as “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming.” Evaluating learning impact has long been considered a daunting task. And, as a recent ATD study shows, only 35 percent of talent development professionals “reported that their organizations evaluated the business results of learning programs to any extent.” A different approach is needed.
First talent development professionals who want to evaluate learning impact must know what specifically they want to measure and how they want to measure it. According to learning expert Patti Shank, the biggest mistake many make in evaluating the impact of learning is just testing recall. Just because people have learned something does not necessarily mean that they’ll apply it or understand its importance on the job. Real assessment comes from observation, providing feedback, and evaluating results.
There are several models to evaluate learning, such as Kirkpatrick’s Four Levels and Phillips’ Evaluation Model. Kirkpatrick’s Four Levels is likely the most commonly known evaluation model, with a four-level program that follows a path of reaction, learning, behavior, and results. The fourth step is the most important, because the results show outcomes and overall effect of the learning in practice. Regardless of the evaluation strategy used, when collecting data, it is also important to use a variety of tools such as surveys, interviews, and analytics to get a full picture.
To be competent at evaluating learning impact, the ASTD Competency Study identifies skills that talent development professionals need to exhibit:
• Identify customer expectations.
• Select appropriate strategies, research design, and measures.
• Communicate and gain support for the evaluation plan.
• Manage data collections.
• Analyze and interpret data.
• Apply learning analytics.
• Make recommendations to aid decision making.
Measuring learning impact encourages higher-quality work, resource prioritization and allocation, and accountability. Measurement data are invaluable to organizations that are strategically using talent development efforts to achieve growth and efficiency objectives. Competency in evaluating learning impact is imperative to becoming a real business partner in the organization.
• Use analytics to improve your organization’s evaluation of learning.
• Discover new ways of measuring learning.
• Learn how improving performance management can help learning impact.
• Measure learning using evaluation and not testing.
• Consider a new way to look at Kirkpatrick’s four levels.
• ATD Research: Evaluating Learning: Getting to Measurements That Matter
• Evaluating the Impact of Training by Scott B. Parry
• For more information on training delivery and other major components of talent development, check out the ASTD Handbook, 2nd Edition.
• Take a deeper dive into evaluation with ATD’s Evaluating Learning Impact Certificate Program.
