ATD Blog
Quantitative/Qualitative Case Study for Selecting Stars
Thu Oct 10 2013
We want to learn from the best performers, and based on those insights, equip typical performers. The first step is to identify exemplary performers by examining the outputs that provide value to the organization and determining relevant performance metrics. You should be very deliberate in deciding what information to use for selecting exemplary performers.
A case study
A senior VP of operations of a Fortune 500 company considered two key metrics as most important for the geographically dispersed business leaders who were the focus of this study: gross revenue and cost.
The senior VP further weighted these metrics by saying that gross revenue was twice as important as operating cost. He further refined his expectations for these two metrics by adjusting the specific expected results to take into account local business unit market maturity, the competition in that market, and state regulations.
Based on this information, we worked with a variety of stakeholders to produce two spreadsheets based on measures that were already in use by the organization.
Measure: | Explanation: |
Revenue Growth | % of revenue growth year over year**** |
Margin | % increase in operating margin (gross profit) year over year |
Employee Satisfaction | Single metric from employee satisfaction survey |
Customer Satisfaction:
Category A
Category B | Single metric (0-100) from customer surveys, segmented into two customer categories |
Quality Indicator | Rolled up quality metric (note: in this particular industry, quality is more important than price or speed) |
In discussing these metrics with the senior VP, we found (not unexpectedly) that some metrics were of greater importance than others. When discussing the individual business leader results, the refinement resulted in the following guidance: |
Choose the top three performers based primarily on revenue growth and margin
Don’t include anyone with employee satisfaction below 65 percent, nor quality indices below 85 percent.
The table below shows that the criteria clearly led to the selection of Pittsburgh, Kansas City, and New York as the top areas.
Measure: | Pittsburgh | Orlando | Kansas City | New York | Los Angeles |
Revenue Growth | 15% | 8% | 6% | 11% | 2% |
Margin | 43% | 16% | 34% | 22% | 9% |
Employee Satisfaction | 77% | 58% | 72% | 83% | 68% |
Customer Satisfaction Category A | 84% | 78% | 78% | 81% | 73% |
Customer Satisfaction Category B | 89% | 76% | 74% | 79% | 71% |
Quality Indices | 97% | 92% | 91% | 94% | 82% |
At your organization |
Eventually all the work you put into exemplary performer selection comes down to a single question: Whose performance do you want to replicate?
Clearly, the answer should be: individuals who consistently produce valued outputs at or above standard. But how do you accomplish this outcome?
First, don’t rely on gut instinct alone or on measures unrelated to outputs of value. Next, remember that a draft list of exemplary performers is produced in a variety of ways. Sometimes you have to take a quantitative approach, and other times you might have to take a more qualitative route to determine the true high performers. Finally, the data should be understood in the context of the organization being studied.
How would you look for an exemplary performer to fill a newly created job—a job in which important aspects are performed only in emergencies? What if there is no pool of performers to question?
Dialogue with others to see what responses are presented!
For more on how to shift the performance curve, check out Al’s previous blog article in this series.