ATD Blog
Measuring VR Training Success
Mon Jan 25 2021
Content
Proving the efficacy of virtual reality (VR) training remains a top concern for learning and development (L&D) professionals. But showing its value is key to including it in an L&D program on an ongoing basis and continuing to fund it. How can L&D professionals leverage VR training and tout its effectiveness to the C-suite? It’s all about keeping an eye on key measurements.
Proving the efficacy of virtual reality (VR) training remains a top concern for learning and development (L&D) professionals. But showing its value is key to including it in an L&D program on an ongoing basis and continuing to fund it. How can L&D professionals leverage VR training and tout its effectiveness to the C-suite? It’s all about keeping an eye on key measurements.
Tying Training Metrics to Organizational Goals
Content
While the advantages of VR training may seem obvious, your program objectives must be tied to organizational goals to demonstrate value and prove its ability to deliver ROI.
While the advantages of VR training may seem obvious, your program objectives must be tied to organizational goals to demonstrate value and prove its ability to deliver ROI.
Content
The good news is that VR training integrates with learning management systems (LMS) and generates multiple data points, so showing its impact is easy. The key is to align the data to business goals and create a narrative about training that the C-suite will understand.
The good news is that VR training integrates with learning management systems (LMS) and generates multiple data points, so showing its impact is easy. The key is to align the data to business goals and create a narrative about training that the C-suite will understand.
Content
To help L&D professionals do this, we modified Kirkpatrick’s Training Evaluation model to make these four levels more relevant to VR training:
To help L&D professionals do this, we modified Kirkpatrick’s Training Evaluation model to make these four levels more relevant to VR training:
Content
Engagement: How engaging and relevant the trainee found the training program
Engagement: How engaging and relevant the trainee found the training program
Content
Comprehension: How well did the trainee retain and comprehend the training materials
Comprehension: How well did the trainee retain and comprehend the training materials
Content
Speed to Competency: How well was the trainee able to apply the information learned
Speed to Competency: How well was the trainee able to apply the information learned
Content
Outcome: How trainee results contribute to the organization’s bottom line
Outcome: How trainee results contribute to the organization’s bottom line
Content
Let’s take a look at this framework in action.
Let’s take a look at this framework in action.
Content
Engagement
Engagement
Content
Measuring employee engagement is a vital component for determining VR training success because of the correlation between engagement and retention. If the content is interesting and relevant, trainees are more likely to retain the information.
Measuring employee engagement is a vital component for determining VR training success because of the correlation between engagement and retention. If the content is interesting and relevant, trainees are more likely to retain the information.
Content
Due to its immersive nature, VR is more engaging to employees when compared to passive forms of training such as videos or workbooks. VR training leverages gamification to make learning new concepts fun and easier to understand.
Due to its immersive nature, VR is more engaging to employees when compared to passive forms of training such as videos or workbooks. VR training leverages gamification to make learning new concepts fun and easier to understand.
Content
When trying to quantify the success of your VR training program’s engagement, use a post-training survey to collect responses. Ask employees questions like:
When trying to quantify the success of your VR training program’s engagement, use a post-training survey to collect responses. Ask employees questions like:
Content
How easy was it to use?
How easy was it to use?
Content
How helpful were the VR training materials?
How helpful were the VR training materials?
Content
How impactful was VR training?
How impactful was VR training?
Content
Comprehension
Comprehension
Content
One of the biggest challenges of any training program is determining if trainees are learning. VR training makes that easier; it generates data that shows how engaged the user is by documenting:
One of the biggest challenges of any training program is determining if trainees are learning. VR training makes that easier; it generates data that shows how engaged the user is by documenting:
Content
Training frequency and completion
Training frequency and completion
Content
Interactions in the training module itself (movement, clicks, and focus)
Interactions in the training module itself (movement, clicks, and focus)
Content
Proficiency in performing tasks in the simulated environment
Proficiency in performing tasks in the simulated environment
Content
L&D professionals also can include comprehension exams in the learning modules that require trainees to pass with satisfactory scores before moving on to the next section. If a large percentage of trainees scores low on that specific module, tweaks can be made to that particular area without having to create a new training program.
L&D professionals also can include comprehension exams in the learning modules that require trainees to pass with satisfactory scores before moving on to the next section. If a large percentage of trainees scores low on that specific module, tweaks can be made to that particular area without having to create a new training program.
Content
Speed to Competency
Speed to Competency
Content
The goal of any training program is to quickly and efficiently teach employees with new skills they need to do the job. To measure speed to competency, it’s important to look at what the organization is trying to achieve and how VR training will help achieve that goal. For manufacturers, the goal may be to reduce the number of manmade errors on the line. For an airline, it may be decreasing the time it takes to turn around planes.
The goal of any training program is to quickly and efficiently teach employees with new skills they need to do the job. To measure speed to competency, it’s important to look at what the organization is trying to achieve and how VR training will help achieve that goal. For manufacturers, the goal may be to reduce the number of manmade errors on the line. For an airline, it may be decreasing the time it takes to turn around planes.
Content
By measuring speed to competency, organizations can see how fast employees master skills and get back on the job. Metrics to measure can include:
By measuring speed to competency, organizations can see how fast employees master skills and get back on the job. Metrics to measure can include:
Content
Duration: The length of time it takes to master a new skill
Duration: The length of time it takes to master a new skill
Content
Retention: Retesting after a period of time to gauge retention
Retention: Retesting after a period of time to gauge retention
Content
Proficiency: Improvement in employee performance
Proficiency: Improvement in employee performance
Content
Outcome
Outcome
Content
There are many benefits to VR training, but the C-suite is looking for how the training program helps improve the bottom line. When determining how successful VR training is, it’s essential to focus on metrics that speak to what the organization is trying to achieve, such as:
There are many benefits to VR training, but the C-suite is looking for how the training program helps improve the bottom line. When determining how successful VR training is, it’s essential to focus on metrics that speak to what the organization is trying to achieve, such as:
Content
Productivity: Did productivity grow due to less downtime for employees and the equipment?
Productivity: Did productivity grow due to less downtime for employees and the equipment?
Content
Savings: Was there a reduction in training-related travel and expenses?
Savings: Was there a reduction in training-related travel and expenses?
Content
Efficiency: Have work-related accidents been reduced?
Efficiency: Have work-related accidents been reduced?
Content
Retention: Are employees staying with the company longer?
Retention: Are employees staying with the company longer?
Data to Support Every Stage of Training
Content
Leveraging data at each stage of VR training demonstrates its value to internal stakeholders. You don’t have to wait for months, or even until training is completed, to start measuring value. You can gauge comprehension and speed to competency during training. You can poll trainees about engagement immediately after. And measuring outcomes—the goal of any training—can start from the day training is completed and employees are on the job.
Leveraging data at each stage of VR training demonstrates its value to internal stakeholders. You don’t have to wait for months, or even until training is completed, to start measuring value. You can gauge comprehension and speed to competency during training. You can poll trainees about engagement immediately after. And measuring outcomes—the goal of any training—can start from the day training is completed and employees are on the job.
Content
Introducing VR training into an enterprise can be challenging, even in organizations with sophisticated L&D programs. Whatever the concerns—integration into existing training, implementation, cost—careful consideration and dissemination of appropriate metrics to show its effectiveness can pave the road to acceptance of new VR solutions.
Introducing VR training into an enterprise can be challenging, even in organizations with sophisticated L&D programs. Whatever the concerns—integration into existing training, implementation, cost—careful consideration and dissemination of appropriate metrics to show its effectiveness can pave the road to acceptance of new VR solutions.