Beyond the Survey: Why L&D Needs More Than Smile Sheets to Show Impact

Content

Training at work is a critical stepping stone for professional development. So, businesses must surely take learning evaluation methods seriously, right?

Training at work is a critical stepping stone for professional development. So, businesses must surely take learning evaluation methods seriously, right?

Content

Employers only tend to evaluate certain parts of the training experience. ATD’s research shows that 93 percent of organizations use participant satisfaction surveys to assess training quality. Unfortunately, far fewer (42 percent) measure actual behavior change data.

Employers only tend to evaluate certain parts of the training experience. ATD’s research shows that 93 percent of organizations use participant satisfaction surveys to assess training quality. Unfortunately, far fewer (42 percent) measure actual behavior change data.

Content

The former definitely has its place. Dissatisfaction is a solid indicator that training doesn’t meet learner needs. Unfortunately, these surveys only tell us how someone felt, not what they learned and applied.

The former definitely has its place. Dissatisfaction is a solid indicator that training doesn’t meet learner needs. Unfortunately, these surveys only tell us how someone felt, not what they learned and applied.

Content

After all, if your data isn’t tied to performance, are you really measuring learning?

After all, if your data isn’t tied to performance, are you really measuring learning?

What Do Surveys Miss?

Content

The satisfaction surveys businesses use (aka “smile sheets”) are a reasonable window into how people feel about training, but fall short as learning evaluation methods for a few reasons:

The satisfaction surveys businesses use (aka “smile sheets”) are a reasonable window into how people feel about training, but fall short as learning evaluation methods for a few reasons:

• Content

  Bias towards positive feedback: Employees might respond positively to training because it was easy, or because they don’t want to make waves, meaning that even satisfaction data isn’t fully reliable.

  Bias towards positive feedback: Employees might respond positively to training because it was easy, or because they don’t want to make waves, meaning that even satisfaction data isn’t fully reliable.

• Content

  Doesn’t measure knowledge or behavior change: Satisfaction surveys don’t capture learning impact, meaning you have no idea just how much knowledge employees retain.

  Doesn’t measure knowledge or behavior change: Satisfaction surveys don’t capture learning impact, meaning you have no idea just how much knowledge employees retain.

• Content

  Often collected too soon: Surveying employees during or just after training means that they may not yet realize whether or not their training is useful.

  Often collected too soon: Surveying employees during or just after training means that they may not yet realize whether or not their training is useful.

What to Measure Instead

Content

ATD’s findings show that only 4 percent of organizations rated themselves extremely proficient at measuring learning impact. Compared to the majority, these businesses generally have a few things in common:

ATD’s findings show that only 4 percent of organizations rated themselves extremely proficient at measuring learning impact. Compared to the majority, these businesses generally have a few things in common:

• Content

  They tend to use four or more data sources.

  They tend to use four or more data sources.

• Content

  They’re more likely to include performance reviews as a data source.

  They’re more likely to include performance reviews as a data source.

• Content

  Proficient businesses inform impact with behavioral and business data (for example, project KPIs or manager feedback).

  Proficient businesses inform impact with behavioral and business data (for example, project KPIs or manager feedback).

Content

By focusing solely on satisfaction and learning objectives, less skilled organizations typically employ only the first stage of the Kirkpatrick model of training evaluation (reaction), while also briefly exploring the second stage (learning) through learning goals.

By focusing solely on satisfaction and learning objectives, less skilled organizations typically employ only the first stage of the Kirkpatrick model of training evaluation (reaction), while also briefly exploring the second stage (learning) through learning goals.

Content

The top 4 percent of businesses, on the other hand, break into stages three and four (behavior and results) by tracking:

The top 4 percent of businesses, on the other hand, break into stages three and four (behavior and results) by tracking:

• Content

  Goal attainment

  Goal attainment

• Content

  Time to achieve competence

  Time to achieve competence

• Content

  Behavior changes over time

  Behavior changes over time

Diversify Your Data

Content

Diversifying your data might sound complicated, but don’t worry. Here are some practical first steps to help you prove learning ROI and effectiveness:

Diversifying your data might sound complicated, but don’t worry. Here are some practical first steps to help you prove learning ROI and effectiveness:

• Content

  Start small: Survey data is still a useful tool. You can pair it with other more business-relevant data points like KPIs or time to productivity to give context to employee responses.

  Start small: Survey data is still a useful tool. You can pair it with other more business-relevant data points like KPIs or time to productivity to give context to employee responses.

• Content

  Talk to managers: Don’t just get feedback from the employees doing the learning, but also their immediate line managers. Employees are great for personal insights, but bosses will give you the big picture on learning impact from their perspective.

  Talk to managers: Don’t just get feedback from the employees doing the learning, but also their immediate line managers. Employees are great for personal insights, but bosses will give you the big picture on learning impact from their perspective.

• Content

  Connect LMS insights: Use tools like Power BI to connect learning data to performance metrics to end the disconnect between training and real work.

  Connect LMS insights: Use tools like Power BI to connect learning data to performance metrics to end the disconnect between training and real work.

• Content

  More data sources: It’s worth repeating that you need several relevant data sources for L&D to get meaningful insights. Don’t be satisfied with less than four that provide a complementary blend of learning and performance data.

  More data sources: It’s worth repeating that you need several relevant data sources for L&D to get meaningful insights. Don’t be satisfied with less than four that provide a complementary blend of learning and performance data.

A Solution to L&D’s Problem

Content

Many organizations have turned to modern LMS platforms like Learn365 to relieve the burden. They have a few key benefits when it comes to streamlining data-heavy L&D management.

Many organizations have turned to modern LMS platforms like Learn365 to relieve the burden. They have a few key benefits when it comes to streamlining data-heavy L&D management.

Content

First, it’s possible to automate post-training assessment with KPIs or virtual grading processes. Second, you can connect training platforms to the tools employees already use (such as by embedding your LMS into Microsoft Teams).

First, it’s possible to automate post-training assessment with KPIs or virtual grading processes. Second, you can connect training platforms to the tools employees already use (such as by embedding your LMS into Microsoft Teams).

Content

Third, feeding learning and performance data into real-time dashboards provides combined metrics with more depth. This also means that stakeholders and L&D leaders don’t have to wait for a fresh report.

Third, feeding learning and performance data into real-time dashboards provides combined metrics with more depth. This also means that stakeholders and L&D leaders don’t have to wait for a fresh report.

Don’t Limit Your L&D Vision

Content

If you only take one lesson from our insights, let it be this: If you’re solely relying on survey data, you’re missing the big picture. Learning programs without business and performance insights are like ships at sea without charts.

If you only take one lesson from our insights, let it be this: If you’re solely relying on survey data, you’re missing the big picture. Learning programs without business and performance insights are like ships at sea without charts.

Content

We recommend evaluating your last three training programs to figure out what you really know about their impact.

We recommend evaluating your last three training programs to figure out what you really know about their impact.

Content

Don’t miss the opportunity to transform your L&D function into a business powerhouse. Download the Zensai + ATD report to guide your internal audit. You’ll see firsthand what leading organizations are doing to connect learning with performance and create lasting success.

Don’t miss the opportunity to transform your L&D function into a business powerhouse. Download the Zensai + ATD report to guide your internal audit. You’ll see firsthand what leading organizations are doing to connect learning with performance and create lasting success.