Advertisement
Advertisement
Big data analytics visualization technology with scientist analyzing information structure on screen with machine learning to extract strategical prediction for business, finance, internet of things
ATD Blog

Ask a Trainer: Is It Possible to Over-Evaluate?

Tuesday, July 14, 2020
Advertisement

In this week’s Ask a Trainer guest post, Will Thalheimer addresses whether trainers evaluate too much or not enough.

Dear Will,

I’m a talent development manager and I’m pretty lucky—I work at an organization that sees the value in training, and I don’t need to constantly defend my department’s right to exist. Given that context, I’ve been wondering how much time I should spend on evaluation. Of course, I want to know whether my team’s trainings are effective, but implementing a robust evaluation process seems like it will take a lot of time and resources. And, honestly, none of my stakeholders are asking for anything beyond smile sheets, which are almost always positive. Should I still be following a rigorous evaluation process, or is there a point where I risk spending more time and money on evaluation than I need to?



Every time we evaluate, we cost our organization resources, time, and energy. So there might be a point where you could theoretically evaluate too much.

However, what I've seen over the last couple of decades is that we don’t evaluate enough, and we do it inadequately. We tend to measure things like butts in seats, how many people completed the course, and we use low-level, poor-performing smile sheets. Occasionally we do a bit more, but often that’s about all we do.

I wouldn't want people to think that they should just ignore evaluation. If you think about evaluation, we need to get feedback to know how well we're doing so that we can make our work better. That's what a professional does, and that's what we should do.

Advertisement

My Learning-Transfer Evaluation Model (LTEM) is eight tiers. The real advantage to the LTEM is that it focuses more on the learning side. Instead of just measuring knowledge, it also looks at decision-making competence and task competence. By focusing on those things, we not only get better evaluation results, but we get better learning designs as well.

You also mentioned smile sheets. There’s been scientific research conducted over many years that has found that the traditional smile sheets are not correlated with learning results. You can get high marks on a smile sheet and have a good course, but it's equally likely that you’ll have high marks and a bad course. If you get low marks, you could have a bad design, but you’re equally likely to have a good design. So, we can't rely on smile sheets.

That being said, are we going to use smile sheets anyway, so can we make them better? I think we can. I’m not anti-smile sheet, but I don’t think smile sheets are the end-all and be-all. Sometimes well-designed smile sheets can be a good starting place for organizations, but they shouldn't end there. Organizations should do more than that.

Advertisement


Learn more about the Learning-Transfer Evaluation Model from Will Thalheimer on the ATD Accidental Trainer podcast. His episode will air on July 15, 2020.



If you have a question for Ask a Trainer, send it to [email protected]. You can find answers to previous questions by visiting the Ask a Trainer hub. Tim will be back next week to tackle a new question.



We welcome your comments and engagement on these posts. All posts are reviewed to ensure appropriateness based on ATD’s requirements for postings in our online communities.

Please note: Content shared in this column is provided by the author and may not reflect the perspectives of ATD.

About the Author

Will Thalheimer, PhD, is a consultant and research translator, providing organizations with learning audits, research benchmarking, workshops, and strategic guidance. Will shares his wisdom through keynotes, research reports, job aids, and blog posts. Compiler of the Decisive Dozen and one of the authors of the Serious eLearning Manifesto, Will blogs atwillatworklearning.com, tweets as @WillWorkLearn, and consults through Work-Learning Research. Will regularly publishes extensive research-to-practice reports—and generously gives them away for free. He is often invited to give keynote presentations at conferences in our field. He is author of Performance-Focused Smile Sheets: A Radical Rethinking of a Dangerous Art Form.

Be the first to comment
Sign In to Post a Comment
Sorry! Something went wrong on our end. Please try again later.