ATD Blog
3 Requirements for Meaningful Measurement
Mon May 13 2024
Brought to you by
Content
I’ve spent more than two decades thinking about different ways to measure human and team performance in various contexts. In that time, I learned that successful measurement requires three things at a minimum:
I’ve spent more than two decades thinking about different ways to measure human and team performance in various contexts. In that time, I learned that successful measurement requires three things at a minimum:
Content
1. Understanding the why or the purpose
1. Understanding the why or the purpose
Content
2. Understanding the impact of the measurement activity
2. Understanding the impact of the measurement activity
Content
3. Clearly communicating the purpose and impact
3. Clearly communicating the purpose and impact
Content
It’s often easy to forget that on the other end of that measure are people—the ones measured, the ones measuring, and the ones deciding what to do about the results. Clearly communicating the purpose and impact in a way that matters to your employees will affect their confidence and trust in the process and outcome.
It’s often easy to forget that on the other end of that measure are people—the ones measured, the ones measuring, and the ones deciding what to do about the results. Clearly communicating the purpose and impact in a way that matters to your employees will affect their confidence and trust in the process and outcome.
Purpose
Content
What do you want to know and why?
What do you want to know and why?
Content
Understanding why data is being collected ensures the measurement is purposeful and designed to meet intended outcomes and make a positive impact.
Understanding why data is being collected ensures the measurement is purposeful and designed to meet intended outcomes and make a positive impact.
Content
With access to big data and mining techniques, it’s easy to go on a “fishing expedition;” if you look hard enough, you will find something, and you’re likely to find “evidence” that supports almost anything. Checking your thinking and framing the measurement problem in advance can help you avoid this pitfall and get the most out of available big data sources.
With access to big data and mining techniques, it’s easy to go on a “fishing expedition;” if you look hard enough, you will find something, and you’re likely to find “evidence” that supports almost anything. Checking your thinking and framing the measurement problem in advance can help you avoid this pitfall and get the most out of available big data sources.
Content
Let’s consider employee engagement. Most organizations want to know if employees are engaged in their work, but the “why” may vary. Specifying why you want to know about employee engagement can help to identify the best measures and gain employee trust and confidence. Suppose you want to know about engagement for employee retention and career development. Data from surveys and employee interviews, compared to time-based metrics for example, can better provide a holistic picture of how your employees are faring in their jobs, where they are thriving (and where they aren’t), and how to keep them engaged over time.
Let’s consider employee engagement. Most organizations want to know if employees are engaged in their work, but the “why” may vary. Specifying why you want to know about employee engagement can help to identify the best measures and gain employee trust and confidence. Suppose you want to know about engagement for employee retention and career development. Data from surveys and employee interviews, compared to time-based metrics for example, can better provide a holistic picture of how your employees are faring in their jobs, where they are thriving (and where they aren’t), and how to keep them engaged over time.
Impact
Content
How will the outcomes affect the measured, the measurer, and the consumers of that information?
How will the outcomes affect the measured, the measurer, and the consumers of that information?
Content
The measurement activity doesn’t start with a survey, a button press, a movement, or a question. Nor does it end after data is collected or analyzed. Chances are that the activity itself—whether you do anything with the results—will have some impact. Framing the measurement activity by asking key questions will help you understand the scope of the challenge and its impact:
The measurement activity doesn’t start with a survey, a button press, a movement, or a question. Nor does it end after data is collected or analyzed. Chances are that the activity itself—whether you do anything with the results—will have some impact. Framing the measurement activity by asking key questions will help you understand the scope of the challenge and its impact:
Content
Who are you measuring?
Who are you measuring?
Content
What does it mean to them?
What does it mean to them?
Content
What about the measurer?
What about the measurer?
Content
What will the data used for and what decisions will they affect?
What will the data used for and what decisions will they affect?
Content
Does the impact on stakeholders help achieve or diminish your desired outcomes?
Does the impact on stakeholders help achieve or diminish your desired outcomes?
Content
A recent project required us to measure the development of complex skills (for example, critical thinking, teamwork) in a classroom. We asked, “Are learners demonstrating proficiency in these skills?” The purpose was to provide developmental feedback to learners since the value of these skills is not immediately obvious—they are more like enablers than key performance indicators.
A recent project required us to measure the development of complex skills (for example, critical thinking, teamwork) in a classroom. We asked, “Are learners demonstrating proficiency in these skills?” The purpose was to provide developmental feedback to learners since the value of these skills is not immediately obvious—they are more like enablers than key performance indicators.
Content
In this example, at least three groups are affected:
In this example, at least three groups are affected:
Content
The learners who are measured
The learners who are measured
Content
The instructors who are measuring
The instructors who are measuring
Content
The administrators who decide what resources to allocate and how
The administrators who decide what resources to allocate and how
Content
If the instructors don’t see the value of the activity or believe it undermines their instruction, they may influence the data negatively by implicitly communicating their skepticism and distrust of the process. Learners are likely to be influenced by this skepticism and be less willing to listen to or act on feedback from the assessment.
If the instructors don’t see the value of the activity or believe it undermines their instruction, they may influence the data negatively by implicitly communicating their skepticism and distrust of the process. Learners are likely to be influenced by this skepticism and be less willing to listen to or act on feedback from the assessment.
Content
Administrators could receive an inaccurate picture of the developmental state and trajectory of learners in the classroom, thereby affecting the implementation of instructional approaches and the allocation of classroom resources.
Administrators could receive an inaccurate picture of the developmental state and trajectory of learners in the classroom, thereby affecting the implementation of instructional approaches and the allocation of classroom resources.
Communication
Content
How do you articulate the purpose and impact in ways that matter to stakeholders?
How do you articulate the purpose and impact in ways that matter to stakeholders?
Content
Start by framing the measurement problem for yourself and all stakeholders. Knowing the purpose ahead of time will help you define the scope and goals and identify who is affected and how to communicate with them. Understanding both practical requirements or constraints and the state of the science is key to answering your questions and achieving purposeful measurement.
Start by framing the measurement problem for yourself and all stakeholders. Knowing the purpose ahead of time will help you define the scope and goals and identify who is affected and how to communicate with them. Understanding both practical requirements or constraints and the state of the science is key to answering your questions and achieving purposeful measurement.
Content
It’s also critical to embrace the art of storytelling to convey the message in a way your audience hears, understands, and learns from. Will your message make an impact or not? How will you communicate your purpose and findings meaningfully? Everyone affected by the activity must clearly understand what this means for them and why it matters.
It’s also critical to embrace the art of storytelling to convey the message in a way your audience hears, understands, and learns from. Will your message make an impact or not? How will you communicate your purpose and findings meaningfully? Everyone affected by the activity must clearly understand what this means for them and why it matters.
Conclusion
Content
Meaningful measurement, at a minimum, requires understanding the people who are measured, those who are measuring, and those who will use the results of the measurement activity. When you clearly communicate purpose and impact in ways that matter, you enable stakeholders to build confidence and trust in the process and its outcomes, ultimately helping you activate the strategy that your measurement activity is designed to support and positively affect your organization.
Meaningful measurement, at a minimum, requires understanding the people who are measured, those who are measuring, and those who will use the results of the measurement activity. When you clearly communicate purpose and impact in ways that matter, you enable stakeholders to build confidence and trust in the process and its outcomes, ultimately helping you activate the strategy that your measurement activity is designed to support and positively affect your organization.