ATD Blog
Quantitative/Qualitative Case Study for Selecting Stars
Thu Oct 10 2013
Content
We want to learn from the best performers, and based on those insights, equip typical performers. The first step is to identify exemplary performers by examining the outputs that provide value to the organization and determining relevant performance metrics. You should be very deliberate in deciding what information to use for selecting exemplary performers.
We want to learn from the best performers, and based on those insights, equip typical performers. The first step is to identify exemplary performers by examining the outputs that provide value to the organization and determining relevant performance metrics. You should be very deliberate in deciding what information to use for selecting exemplary performers.
Content
A case study
A case study
Content
A senior VP of operations of a Fortune 500 company considered two key metrics as most important for the geographically dispersed business leaders who were the focus of this study: gross revenue and cost.
A senior VP of operations of a Fortune 500 company considered two key metrics as most important for the geographically dispersed business leaders who were the focus of this study: gross revenue and cost.
Content
The senior VP further weighted these metrics by saying that gross revenue was twice as important as operating cost. He further refined his expectations for these two metrics by adjusting the specific expected results to take into account local business unit market maturity, the competition in that market, and state regulations.
The senior VP further weighted these metrics by saying that gross revenue was twice as important as operating cost. He further refined his expectations for these two metrics by adjusting the specific expected results to take into account local business unit market maturity, the competition in that market, and state regulations.
Content
Based on this information, we worked with a variety of stakeholders to produce two spreadsheets based on measures that were already in use by the organization.
Based on this information, we worked with a variety of stakeholders to produce two spreadsheets based on measures that were already in use by the organization.
Content | Content |
Content Measure: Measure: | Content Explanation: Explanation: |
Content Revenue Growth Revenue Growth | Content % of revenue growth year over year**** % of revenue growth year over year**** |
Content Margin Margin | Content % increase in operating margin (gross profit) year over year % increase in operating margin (gross profit) year over year |
Content Employee Satisfaction Employee Satisfaction | Content Single metric from employee satisfaction survey Single metric from employee satisfaction survey |
Content Customer Satisfaction: Customer Satisfaction: Content
Content Category A Category A Content
Content Category B Category B | Content Single metric (0-100) from customer surveys, segmented into two customer categories Single metric (0-100) from customer surveys, segmented into two customer categories |
Content Quality Indicator Quality Indicator | Content Rolled up quality metric (note: in this particular industry, quality is more important than price or speed) Rolled up quality metric (note: in this particular industry, quality is more important than price or speed) |
Content In discussing these metrics with the senior VP, we found (not unexpectedly) that some metrics were of greater importance than others. When discussing the individual business leader results, the refinement resulted in the following guidance: In discussing these metrics with the senior VP, we found (not unexpectedly) that some metrics were of greater importance than others. When discussing the individual business leader results, the refinement resulted in the following guidance: |
Content
Choose the top three performers based primarily on revenue growth and margin
Choose the top three performers based primarily on revenue growth and margin
Content
Don’t include anyone with employee satisfaction below 65 percent, nor quality indices below 85 percent.
Don’t include anyone with employee satisfaction below 65 percent, nor quality indices below 85 percent.
Content
The table below shows that the criteria clearly led to the selection of Pittsburgh, Kansas City, and New York as the top areas.
The table below shows that the criteria clearly led to the selection of Pittsburgh, Kansas City, and New York as the top areas.
Content | Content | Content | Content | Content | Content |
Content Measure: Measure: | Content Pittsburgh Pittsburgh | Content Orlando Orlando | Content Kansas City Kansas City | Content New York New York | Content Los Angeles Los Angeles |
Content Revenue Growth Revenue Growth | Content 15% 15% | Content 8% 8% | Content 6% 6% | Content 11% 11% | Content 2% 2% |
Content Margin Margin | Content 43% 43% | Content 16% 16% | Content 34% 34% | Content 22% 22% | Content 9% 9% |
Content Employee Satisfaction Employee Satisfaction | Content 77% 77% | Content 58% 58% | Content 72% 72% | Content 83% 83% | Content 68% 68% |
Content Customer Satisfaction Category A Customer Satisfaction Category A | Content 84% 84% | Content 78% 78% | Content 78% 78% | Content 81% 81% | Content 73% 73% |
Content Customer Satisfaction Category B Customer Satisfaction Category B | Content 89% 89% | Content 76% 76% | Content 74% 74% | Content 79% 79% | Content 71% 71% |
Content Quality Indices Quality Indices | Content 97% 97% | Content 92% 92% | Content 91% 91% | Content 94% 94% | Content 82% 82% |
Content At your organization At your organization |
Content
Eventually all the work you put into exemplary performer selection comes down to a single question: Whose performance do you want to replicate?
Eventually all the work you put into exemplary performer selection comes down to a single question: Whose performance do you want to replicate?
Content
Clearly, the answer should be: individuals who consistently produce valued outputs at or above standard. But how do you accomplish this outcome?
Clearly, the answer should be: individuals who consistently produce valued outputs at or above standard. But how do you accomplish this outcome?
Content
First, don’t rely on gut instinct alone or on measures unrelated to outputs of value. Next, remember that a draft list of exemplary performers is produced in a variety of ways. Sometimes you have to take a quantitative approach, and other times you might have to take a more qualitative route to determine the true high performers. Finally, the data should be understood in the context of the organization being studied.
First, don’t rely on gut instinct alone or on measures unrelated to outputs of value. Next, remember that a draft list of exemplary performers is produced in a variety of ways. Sometimes you have to take a quantitative approach, and other times you might have to take a more qualitative route to determine the true high performers. Finally, the data should be understood in the context of the organization being studied.
Content
How would you look for an exemplary performer to fill a newly created job—a job in which important aspects are performed only in emergencies? What if there is no pool of performers to question?
How would you look for an exemplary performer to fill a newly created job—a job in which important aspects are performed only in emergencies? What if there is no pool of performers to question?
Content
Dialogue with others to see what responses are presented!
Dialogue with others to see what responses are presented!
Content
For more on how to shift the performance curve, check out Al’s previous blog article in this series .
For more on how to shift the performance curve, check out Al’s previous blog article in this series.