Whether you work in industry or education, your training initiatives should have well-defined goals and produce measurable outcomes. By clearly establishing objectives, collecting data, and analyzing trends, you will obtain a much clearer understanding of how well your training programs are performing. This will allow you to see which programs are working, what you need to improve, and how best to implement that change. Increasingly, learning and development (L&D) organizations are coming to appreciate the power of a rigorous analytics strategy.
Approaching Your Learning Analytics Data
Ultimately, the goal of data analysis is to obtain actionable information. To do this, you need to establish a consistent framework that gives context to your raw learning data. A complete analytics framework has two main components: strategy and techniques. Strategy refers to the high-level questions that will determine the key objectives, metrics, and tools for your analytics program. Techniques are the best practices that will allow you to design and implement an effective program.
In the first part of this two-part blog post, we will discuss the ‘Strategy’ piece.
The three major questions you want to ask are:
1. Why do I need analytics for my training program?
What are you trying to achieve in your training program? While all programs involve some element of knowledge acquisition or behavioural change, specifics can vary quite widely. For instance, your training program could be attempting to:
- increase the success rate of outbound sales calls
- reduce average time spent on customer support calls
- maximize productivity in a manufacturing facility.
These are “big-picture” business goals that are not always straightforward to evaluate. As a result, many organizations lose sight of the original objective and end up trying to measure too many things at once. By committing to a specific goal, you will be able to avoid this trap and focus only on the metrics that matter.
2. What are the key metrics to measure toward my objectives?
The next step is to determine the key metrics that will allow you to measure progress toward your high-level objective. These metrics are known as “key performance indicators” (KPIs).
Some common learning KPIs include:
- Activity Scores
- Course Completion
- Grades
- Courses Purchased
- Time Spent Engaged
Often, to accurately evaluate progress, you will need to combine several of these metrics in some way.
3. How do I implement my analytics strategy?
Once you’ve determined the metrics you need to answer key questions, you have to decide how to implement your learning analytics program. It is very important to have realistic expectations. Many analytics programs fail because they do not consider the organization’s current analytics capabilities.
In particular, there are three common challenges you should watch out for:
- Technology: Do you have the tools required to collect, store, analyze, and present your data?
- Data: Are you able to get access to all the relevant data in your LMS and other business intelligence systems?
- Skills: Do your people have the knowledge, judgment, and experience to analyze your data effectively?
Appropriately calibrating your expectations can mean the difference between a successful analytics program and one that fails to get off the ground. In our next blog post, Capturing Data to Improve the E-Learning Experience with Learning Analytics (Part 2), we’ll discuss the various techniques required for implementing learning analytics.