The ATD Talent Development Framework: Evaluating Learning Impact

Content

ATD developed the talent development framework and puzzle to help practitioners understand the different components of talent development and how organizations can build their own frameworks to address their unique needs. This blog series explores each component of the talent development framework, why it’s important to the field, and what resources ATD offers to practitioners who want to learn more.

ATD developed the talent development framework and puzzle to help practitioners understand the different components of talent development and how organizations can build their own frameworks to address their unique needs. This blog series explores each component of the talent development framework, why it’s important to the field, and what resources ATD offers to practitioners who want to learn more.

Content

Evaluating Learning Impact

Evaluating Learning Impact

Content

Evaluating learning impact involves the use of learning metrics and analytics to measure the effectiveness of learning solutions. In Utilization Focused Evaluation: The New Century Text , Michael Quinn Patton defines evaluation as “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming.” Evaluating learning impact has long been considered a daunting task. And, as a recent ATD study shows, only 35 percent of talent development professionals “reported that their organizations evaluated the business results of learning programs to any extent.” A different approach is needed.

Evaluating learning impact involves the use of learning metrics and analytics to measure the effectiveness of learning solutions. In Utilization Focused Evaluation: The New Century Text, Michael Quinn Patton defines evaluation as “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming.” Evaluating learning impact has long been considered a daunting task. And, as a recent ATD study shows, only 35 percent of talent development professionals “reported that their organizations evaluated the business results of learning programs to any extent.” A different approach is needed. 

Content

First talent development professionals who want to evaluate learning impact must know what specifically they want to measure and how they want to measure it. According to learning expert Patti Shank, the biggest mistake many make in evaluating the impact of learning is just testing recall. Just because people have learned something does not necessarily mean that they’ll apply it or understand its importance on the job. Real assessment comes from observation, providing feedback, and evaluating results.

First talent development professionals who want to evaluate learning impact must know what specifically they want to measure and how they want to measure it. According to learning expert Patti Shank, the biggest mistake many make in evaluating the impact of learning is just testing recall. Just because people have learned something does not necessarily mean that they’ll apply it or understand its importance on the job. Real assessment comes from observation, providing feedback, and evaluating results. 

Content

There are several models to evaluate learning, such as Kirkpatrick’s Four Levels and Phillips’ Evaluation Model. Kirkpatrick’s Four Levels is likely the most commonly known evaluation model, with a four-level program that follows a path of reaction, learning, behavior, and results. The fourth step is the most important, because the results show outcomes and overall effect of the learning in practice. Regardless of the evaluation strategy used, when collecting data, it is also important to use a variety of tools such as surveys, interviews, and analytics to get a full picture.

There are several models to evaluate learning, such as Kirkpatrick’s Four Levels and Phillips’ Evaluation Model. Kirkpatrick’s Four Levels is likely the most commonly known evaluation model, with a four-level program that follows a path of reaction, learning, behavior, and results. The fourth step is the most important, because the results show outcomes and overall effect of the learning in practice. Regardless of the evaluation strategy used, when collecting data, it is also important to use a variety of tools such as surveys, interviews, and analytics to get a full picture. 

Content

To be competent at evaluating learning impact, the ASTD Competency Study (Updated now to ATD Talent Development Capability Model™ ) identifies skills that talent development professionals need to exhibit:

To be competent at evaluating learning impact, the ASTD Competency Study (Updated now to ATD Talent Development Capability Model™) identifies skills that talent development professionals need to exhibit:

Content

• Identify customer expectations.

• Identify customer expectations.

Content

• Select appropriate strategies, research design, and measures.

• Select appropriate strategies, research design, and measures.

Content

• Communicate and gain support for the evaluation plan.

• Communicate and gain support for the evaluation plan.

Content

• Manage data collections.

• Manage data collections.

Content

• Analyze and interpret data.

• Analyze and interpret data.

Content

• Apply learning analytics.

• Apply learning analytics.

Content

• Make recommendations to aid decision making.

• Make recommendations to aid decision making.

Content

Measuring learning impact encourages higher-quality work, resource prioritization and allocation, and accountability. Measurement data are invaluable to organizations that are strategically using talent development efforts to achieve growth and efficiency objectives. Competency in evaluating learning impact is imperative to becoming a real business partner in the organization.

Measuring learning impact encourages higher-quality work, resource prioritization and allocation, and accountability. Measurement data are invaluable to organizations that are strategically using talent development efforts to achieve growth and efficiency objectives. Competency in evaluating learning impact is imperative to becoming a real business partner in the organization.

Content

We’ve assembled some additional resources to explore:

We’ve assembled some additional resources to explore:

Content

• Use analytics to improve your organization’s evaluation of learning.

• Use analytics to improve your organization’s evaluation of learning. 

Content

• Discover new ways of measuring learning.

• Discover new ways of measuring learning. 

Content

• Learn how improving performance management can help learning impact.

• Learn how improving performance management can help learning impact. 

Content

• Measure learning using evaluation and not testing.

• Measure learning using evaluation and not testing. 

Content

• Consider a new way to look at Kirkpatrick’s four levels.

• Consider a new way to look at Kirkpatrick’s four levels. 

Content

• ATD Research: Evaluating Learning: Getting to Measurements That Matter

• ATD Research: Evaluating Learning: Getting to Measurements That Matter

Content

• Evaluating the Impact of Training by Scott B. Parry

•Evaluating the Impact of Training by Scott B. Parry

Content

• For more information on training delivery and other major components of talent development, check out the ASTD Handbook, 2nd Edition (Updated now to the TDBok™ Guide, 2nd Edition ).

• For more information on training delivery and other major components of talent development, check out the ASTD Handbook, 2nd Edition (Updated now to the TDBok™ Guide, 2nd Edition). 

Content

• Take a deeper dive into evaluation with ATD’s Evaluating Learning Impact Certificate Program.

• Take a deeper dive into evaluation with ATD’s Evaluating Learning Impact Certificate Program.