Advertisement
Advertisement
Performance Measures The Missing Link in Professional Competency Models.jpg
ATD Blog

3 Requirements for Meaningful Measurement

Monday, May 13, 2024
Advertisement

I’ve spent more than two decades thinking about different ways to measure human and team performance in various contexts. In that time, I learned that successful measurement requires three things at a minimum:

1. Understanding the why or the purpose
2. Understanding the impact of the measurement activity
3. Clearly communicating the purpose and impact

It’s often easy to forget that on the other end of that measure are people—the ones measured, the ones measuring, and the ones deciding what to do about the results. Clearly communicating the purpose and impact in a way that matters to your employees will affect their confidence and trust in the process and outcome.

Purpose

What do you want to know and why?

Understanding why data is being collected ensures the measurement is purposeful and designed to meet intended outcomes and make a positive impact.

With access to big data and mining techniques, it’s easy to go on a “fishing expedition;” if you look hard enough, you will find something, and you’re likely to find “evidence” that supports almost anything. Checking your thinking and framing the measurement problem in advance can help you avoid this pitfall and get the most out of available big data sources.

Let’s consider employee engagement. Most organizations want to know if employees are engaged in their work, but the “why” may vary. Specifying why you want to know about employee engagement can help to identify the best measures and gain employee trust and confidence. Suppose you want to know about engagement for employee retention and career development. Data from surveys and employee interviews, compared to time-based metrics for example, can better provide a holistic picture of how your employees are faring in their jobs, where they are thriving (and where they aren’t), and how to keep them engaged over time.

Advertisement

Impact

How will the outcomes affect the measured, the measurer, and the consumers of that information?

The measurement activity doesn’t start with a survey, a button press, a movement, or a question. Nor does it end after data is collected or analyzed. Chances are that the activity itself—whether you do anything with the results—will have some impact. Framing the measurement activity by asking key questions will help you understand the scope of the challenge and its impact:

  • Who are you measuring?
  • What does it mean to them?
  • What about the measurer?
  • What will the data used for and what decisions will they affect?
  • Does the impact on stakeholders help achieve or diminish your desired outcomes?

A recent project required us to measure the development of complex skills (for example, critical thinking, teamwork) in a classroom. We asked, “Are learners demonstrating proficiency in these skills?” The purpose was to provide developmental feedback to learners since the value of these skills is not immediately obvious—they are more like enablers than key performance indicators.

In this example, at least three groups are affected:

  • The learners who are measured
  • The instructors who are measuring
  • The administrators who decide what resources to allocate and how

If the instructors don’t see the value of the activity or believe it undermines their instruction, they may influence the data negatively by implicitly communicating their skepticism and distrust of the process. Learners are likely to be influenced by this skepticism and be less willing to listen to or act on feedback from the assessment.

Administrators could receive an inaccurate picture of the developmental state and trajectory of learners in the classroom, thereby affecting the implementation of instructional approaches and the allocation of classroom resources.

Advertisement

Communication

How do you articulate the purpose and impact in ways that matter to stakeholders?

Start by framing the measurement problem for yourself and all stakeholders. Knowing the purpose ahead of time will help you define the scope and goals and identify who is affected and how to communicate with them. Understanding both practical requirements or constraints and the state of the science is key to answering your questions and achieving purposeful measurement.

It’s also critical to embrace the art of storytelling to convey the message in a way your audience hears, understands, and learns from. Will your message make an impact or not? How will you communicate your purpose and findings meaningfully? Everyone affected by the activity must clearly understand what this means for them and why it matters.

Conclusion

Meaningful measurement, at a minimum, requires understanding the people who are measured, those who are measuring, and those who will use the results of the measurement activity. When you clearly communicate purpose and impact in ways that matter, you enable stakeholders to build confidence and trust in the process and its outcomes, ultimately helping you activate the strategy that your measurement activity is designed to support and positively affect your organization.

About the Author

Camilla Chavez Knott, PhD, is a principal at TiER1. She leads a portfolio of work that focuses on developing complex cognitive skills in high-risk environments. Camilla’s work uses well-established and novel approaches to create innovative solutions that are rooted in both scientific theory and real-world constraints.

1 Comment
Sign In to Post a Comment
I rarely think about the instructor/supervisor as a stakeholder, but will absolutely be more attentive to the implicit and explicit influence going forward. Thanks for sharing
Sorry! Something went wrong on our end. Please try again later.
Sorry! Something went wrong on our end. Please try again later.