Top
1.800.628.2783
1.800.628.2783
ATD Links Archive
Issue Map
ATD Links Archive
ATD Links

Moving From Learning Analytics to Performance Analytics

If you want to get executive attention, start using the words “big data” and “analytics.” You can even take data we’ve always been collecting about learning events and refer to it as learning analytics. Yet when we do this, we typically get the same nodding response that our learning data got before we included the words big data and analytics. This is important data—to us. For executives and operations staff, it is not that critical.

Regardless of organizational size, most executives and operational staff want data that shows the results of our talent development actions. They want us to quantify workplace behaviors and performance measures that make a difference to the bottom line. They want to go deeper than learning analytics or best guesses. They want to base their human capital development investment decisions on performance analytics. For example, executives are asking if the same technology that’s used to learn about the daily purchasing behaviors of customers, can be used to learn about employees’ daily work practices and on-the-job performance. 

A 2014 Oxford Economics study reports that only 38 percent of executives believe they have ample workforce data to understand their organization’s talent strengths and vulnerabilities and inform human capital investment decisions. Evaluation experts Jack and Patti Phillips have identified 12 major forces influencing an organization’s ability to generate the kind of data required to help with human capital investment decisions. These include the changing nature of work, workforce demographics, and workplace technologies. 

An increasingly volatile, uncertain, complex, and ambiguous work environment also presents unique challenges for talent development professionals who want to analyze and optimize workforce performance. Change leader and author Holly Burkett suggests that growing and sustaining optimal performance in a world of constant change begins with talent development professionals who adopt future-focused, change-ready mindsets and capabilities. In short, today’s dynamic and competitive market causes rapid changes in the kind of performance an organization needs to survive and stay relevant. Rapidly changing performance demands make it even more important for professionals to effectively use analytics to diagnose and improve performance issues at the time of need.

Insights and skills gained from using big data analytics have been shown to optimize workforce performance at the individual, team, and organization levels. Let’s consider an example from healthcare to show how this works.

Getting specific

Executives at a large healthcare system wanted to significantly improve patient outcome metrics at six of their hospitals. They wanted to focus on improving the daily practices of their 1,157 nurses and their supervisors since they are in daily, continuous care-giving contact with their patients. Where do you start with such vague Level 4 business impact requirements?

First, the study’s leaders decided to measure the strengths and vulnerabilities of each nurse’s behaviors in four critical competency domains—leadership, communication, decision making, and execution—to establish a baseline of strengths and vulnerabilities. This involved capturing data about 51 behaviors each nurses exhibited in their daily practices. They used a cloud-based 360-degree assessment tool that measured 167,229 professional behaviors against five competency levels for each behavior. The big data analytics software generated more than 2.3 million behavioral data points of current workplace practices within 60 days of beginning the data collection process to establish their baseline workforce competence metrics. Guessing was no longer necessary. 

The workplace performance analytics software provided the organization with a real-time visualization of actionable, real-world baselined behavioral data. For the first time, decision-makers could see data visualizations of the strengths and vulnerabilities of each of these six hospitals and drill down to each individual nurse. This allowed them to use insights hidden in the big data to diagnose previously unknown professional role competency gaps for strategic and tactical talent development interventions. 

The big data performance analytics uncovered hidden Level 3 behavioral data they would have never guessed. It quantified competency gaps in the daily behaviors and practices of their patient care workforce. Insights from this data helped explain substandard Level 4 patient outcome data. They started their talent development process with the “as-is” daily practice data and insights that they never had before. The data provided specific insights about what was really occurring at each hospital, department, shift, and caregiver team. Talent development decision makers used these data-driven insights to develop tailored professional role development solutions to meet real needs that mattered. Their decisions were based on the organization’s priorities and ranged from the enterprise to individual levels. No more guessing about what to do or what learning courses to offer.

Implications for talent development

Advertisement

Now comes the accountability and sustainability part of the talent development effort that executives also expect. This talent development team is continuing to collect behavioral data using the cloud-based 360-degree instrument to generate additional performance analytics to make sure the nurses are changing their daily practices and that the healthcare system is supporting this behavior change. This allows them to adjust or stop learning solutions that are not producing the expected results and to focus their attention where the need is greatest. They are using real-time performance analytics to improve consistency of competent care, reduce variation in practices, increase patient satisfaction, reduce errors and costly turnover, and increase retention and employee engagement.

This is data that executives want and expect instead of data telling them which courses were provided to the nurses, their average assessment scores and completion rates, and who has completed which courses. They are also growing their baseline 2.3+ million behavioral data points at each data collection development phase to mature their descriptive analytics capability to predictive and prescriptive analytics.  

As talent development professionals, we finally have big data analytics technology tailored to measure and evaluate the results of what we do—which is to develop and sustain the daily performance of workforce talent. This technology enables us to keep our finger on the pulse of our workforce’s strengths and vulnerabilities so we can help executives make informed decisions about human capital strategies. 

No more guessing. Instead, we’ll be impressing. How? Maturing our big data analytics insight capabilities from Level 2 learning to Level 3 performance will position us to change how executives and operational managers view and value our capabilities to help them make human capital development investment decisions and learn the return-on-investment of those decisions. Isn’t this what we have been wanting? 

References

Burkett, H. 2017. Learning for the Long Run: 7 Practices for Sustaining a Resilient Learning Organization. Alexandria, VA: ATD Press.

Kellerman, B. 2012. The End of Leadership. New York: HarperCollins.

Oxford Economics. 2014. Workforce 2020: The looming talent crises. successfactors.com/en_us/lp/workforce-2020-insights.html

Phillips, J. and Phillips, P. 2015. High-Impact Human Capital Strategy: Addressing 12 Major Challenges Today’s Organizations Face. New York: AMACOM.

© 2017 ATD, Alexandria, VA. All rights reserved.

About the Author
Dr. Tim Brock is an Associate of the ROI Institute, the CEO of The Institute 4 Worthy Performance, and a Practice Leader with The Institute for Performance Improvement.  Dr. Brock provides consulting services for Fortune 500 companies as well as educational sessions and workshops at multiple international conferences, at company-sponsored talent development initiatives, and at public venues. He has over 30 years of learning, performance improvement, and evaluation experience in the military, education, healthcare, and corporate environments for organizations around the globe.  During his Air Force career, Dr. Brock was responsible for assessing, measuring, and evaluating the effectiveness of initial qualification training effectiveness for all five of the USAF’s ICBM fleet. During his career with Lockheed Martin, he was the manager of their Science of Learning and Performance Improvement thought leadership and evaluation team.  Dr. Brock currently helps organizations create internal workforce capabilities to apply needs-driven accountability processes using the ROI Methodology, the principles and practices of Performance Improvement, and human capital analytics technologies.  He holds a PhD in Education with a specialization in Training and Performance Improvement through Capella University.  Dr. Brock’s publications include “Training NASA Astronauts for Deep Space Exploration Mission: A Research Study to Develop and Validate a Competency-Based Training Framework” (2007) and the chapter Simulations Operations, Curriculum Integration, and Performance Improvement in the book “Healthcare Simulation: A Guide for Operations Specialists” (2016).  He also facilitates the online and classroom versions of ATD’s Evaluating Learning Impact and Measuring Return on Investment certificate courses.
Be the first to comment
Sign In to Post a Comment
Sorry! Something went wrong on our end. Please try again later.