Top
1.800.628.2783
1.800.628.2783
465057533
TD Magazine

More Than a Smile Sheet: Using Level 1 Evaluation Effectively

Monday, August 15, 2016
Advertisement

First identified by Don Kirkpatrick in his seminal works on training evaluation, Level 1 evaluation helps an organization to assess participants' reactions to a learning event, facilitator, setting, materials, and learning activities. Like a canary in the proverbial coal mine, Level 1 is an early indicator of whether the participant valued the experience enough to positively speak about it to others.

using-level-1.png

It can also identify whether it was a negative experience likely to inhibit learning and application back on the job and lead the learner to speak negatively about it to others. Level 1 evaluation, by itself, should not be the only training evaluation tool you use.

What is it?

Level 1 evaluation means measuring the reaction to the learning by evaluating the trainee's perceptions about the learning experience. The intent is to determine if the trainees liked the training and if the training was relevant to their work, to make potential improvements that positively impact learning. There are several areas of consideration: the program content, sequencing, and materials; the logistics of the training and the training environment; the instructor's facilitation, interaction, and organization of the content; and the expectation for applying the training back on the job.

Guidelines

1| Determine how the data will be gathered.

  • Use a questionnaire immediately after the learning event that is either paper or electronic.
  • Use interviews and focus groups as a follow-up to the reaction questionnaires to gather more information.
  • Provide in-session polling with an audience response system tool.
  • Gather in-session data with wall charts, index cards, and other implements.
  • Connect with the learner using social media such as Twitter or Facebook to find out what they are really saying about the training.

    2| Choose your perspective. Choose when you'd like to gather the data. Will you wait until the end of a session or unit? Depending on the length of the program, will you get the reaction at the end of a program or day or module? If you are running a pilot program, what do you want to know during the session so that you can make adjustments?

    3| Align to your desired outcomes. Determine the business outcomes with your stakeholders first and design your Level 3 and 4 evaluations. Determine the learning objectives and the KSAs (key skills and abilities) for the Level 2 evaluation. Write your Level 1 evaluation to assess delivery and the learning experience.

    4| Write questions that ask for quantified responses. Most questions are subjective (including Likert-scale questions such as "strongly agree"). What do you want to know? "Did the training include practice opportunities," with numeric response options tells you something different than asking, "did you like the practice opportunities?" If you use Likert-scale type questions, provide a "comments" section for further information about this answer.

    5| How will you use the evaluation information? If you ask how the learner rates the usefulness of the training on a scale of 1 to 5, what will you do with that information? Think about what you want to know, and ask questions that give you those answers. Pay attention to the answers. Asking the right questions helps you make sure the training is doing what it's supposed to do.

    Advertisement

    Myths

    Myth 1. Level 1 is just a "smile sheet" and offers no relevant or useful information.

    Fact. Level 1 evaluation has its place in helping uncover content sequencing problems, delivery and facilitation problems, and training environment problems, allowing you to quickly rectify them. It also identifies what the learner perceives to be going well.

    Myth 2. If the learners are happy, then the training is a success.

    Fact. Happy learners and unhappy learners may or may not acquire knowledge and skills. You must complete a Level 2 evaluation to identify whether they have learned the content.

    Myth 3. Learners can accurately assess the training's value to their actual job.

    Fact. Learners may not know how they are going to use it and if the job environment is conducive to new knowledge application. This validation should be done at the instructional design phase with subject-matter experts, manager input, and support.

using-level-2.png

Results

Learners provide their subjective perceptions about the pace, the delivery, the organization, and activities of the program. They can be biased by the difficulty of the material, the personality of the trainer, and other factors. The Level 1 evaluation should be one tool in your evaluation arsenal identifying the needed improvements that will strengthen the program. Make sure that the Level 1 reaction form includes

  • reaction to the content
  • reaction to the effectiveness of the trainer or facilitator
  • reaction to the materials, such as handouts, audiovisuals, case studies, and activities.

    Why it works

    Level 1 evaluation, used correctly, has a significant place in understanding the satisfaction of the learner. Immediate feedback helps the facilitator and organization make needed adjustments to the program. Level 1 is so much more than a smile sheet!

About the Author
Geri is president and principal consultant of Geri Lopker & Associates LLC. Her international client list includes corporations, government agencies, nonprofit organizations, and Fortune 100 companies. Geri has been a performance consultant both as an internal area director of operations for a large healthcare agency, and as an external consultant with clients big and small. Geri has more than 20 years of experience working in the areas of systems, finance, change management, leadership, communication, strategic planning, team building, and customer relations. Geri earned a master's degree in human resources and organization development from the University of San Francisco. She earned the Certified Performance Technologist (CPT) certification in 2003.She earned the Certified Professional in Learning and Performance (CPLP) in 2007. She teaches the nationally acclaimed Human Performance Improvement in the Workplace Certificate series. She has also earned the HPI Certificate. She has received the Total Trainer Certificate from OC-ASTD and is senior faculty for the Total Trainer program. Geri is a past president of the International Society for Performance Improvement, Orange County Chapter. She is also a past president of ASTD’s Orange County chapter.
About the Author
Rhonda Askeland is an organization development and training partner, facilitator, coach, instructional designer, and performance consultant with a twenty-year background working with leaders at all levels — C-level, managers, teams, and individual contributors. She has her own consulting business and has worked for training companies Blessing/White and Cahners TRACOM. She has worked with leaders in Fortune 100 companies in diverse industries, as well as government agencies, and a California university system.

Her recent projects include development of a leadership development series in the entertainment industry, co-development and facilitation of an innovative 12-month leadership program for executive managers in the finance industry, and ongoing coaching of sales management on account executives selection process for a credit-reporting agency. In addition to coaching, she created a 90-day onboarding map to follow-up account executive selection to accelerate hire-to-performance.

Her creativity in design and delivery earned her recognition in ASTD's Training magazine for innovative training and experiential learning techniques. Additionally, she was a co-author in two ASTD's T+D Learning Blueprint columns on "Presentation Design" and "More Than a Smile Sheet: Using Level 1 Evaluation Effectively."

Rhonda has a passion for helping organizations achieve results through people. She is president-elect of ASTD Orange County (OC), past president of International Society of Performance Improvement (ISPI)-OC (2009), past board member of ISPI-OC and ISPI-LA, and is currently on the Whittier College Alumni Board.

Rhonda's goal is to improve performance through people.
Be the first to comment
Sign In to Post a Comment
Sorry! Something went wrong on our end. Please try again later.