ATD Blog

Creating a Measurement Model

Thu Apr 19 2012

Content

Creating a Measurement Model

Content

In the first two posts in this series, we discussed the value of data and the process for using the theory of change model for learning design. This week, we’ll walk through creating a measurement model for a fictional company, Acme Donuts.

Content

As you work through the learning initiative design process, the key challenge is to balance strategy with tactics: Connecting specific actions that accomplish high level goals. The Theory of Change model, which starts with broad goals and asks you to work backwards to specific actions, forces you to make the connections between each step clear. The narrative step also challenges you to explain the connections: An excellent exercise for testing assumptions and discovering gaps.

Content

I created this sample narrative to outline the many factors relevant to designing a learning program that includes a measurement model from the beginning.

Content

Developing a Program Narrative: Acme Donuts Example

Developing a Program Narrative: Acme Donuts Example

Content
What are your initiative’s goals? To develop functional goals, you need stakeholder buy-in, alignment with organizational goals and specific descriptors. Make sure you’re not developing goals in a vacuum._
What are your initiative’s goals? To develop functional goals, you need stakeholder buy-in, alignment with organizational goals and specific descriptors. Make sure you’re not developing goals in a vacuum._
Content
Acme, Inc. is a manufacturer of donuts. Our goal is to increase profits 15% this fiscal year. To do that, we must meet two key preconditions: 1) Increase the recurring revenue generated from long-term sales agreements; 2) The sales team must develop strong communication, sales and negotiation skills to negotiate and close sales agreements._
Acme, Inc. is a manufacturer of donuts. Our goal is to increase profits 15% this fiscal year. To do that, we must meet two key preconditions: 1) Increase the recurring revenue generated from long-term sales agreements; 2) The sales team must develop strong communication, sales and negotiation skills to negotiate and close sales agreements._
Content
Whether conscious or unconscious, everyone makes assumptions. A key step in developing a functional initiative is ensuring that you understand what variables you will hold neutral as you target changes in your organization._
Whether conscious or unconscious, everyone makes assumptions. A key step in developing a functional initiative is ensuring that you understand what variables you will hold neutral as you target changes in your organization._
Content
We are making two key assumptions: 1) The product is appropriately priced for its quality. 2) There is sufficient demand in the marketplace to enable us to increase sales._
We are making two key assumptions: 1) The product is appropriately priced for its quality. 2) There is sufficient demand in the marketplace to enable us to increase sales._
Content
Work backwards from your goals to the preconditions that must be met. Limit the scope of your discussion to the changes that can be affected through training and development activities. If, for example, Acme’s Donuts just aren’t very good, training sales people more effectively may not do any good at all._
Work backwards from your goals to the preconditions that must be met. Limit the scope of your discussion to the changes that can be affected through training and development activities. If, for example, Acme’s Donuts just aren’t very good, training sales people more effectively may not do any good at all._
Content
Both sales team members and managers have described needing opportunities both to study and practice the techniques of sales and communication and to work together as a team to identify best practices and improve performance over time._
Both sales team members and managers have described needing opportunities both to study and practice the techniques of sales and communication and to work together as a team to identify best practices and improve performance over time._
Content
Outline specific initiatives you will undertake. Ensure that you outline how those initiatives connect with your preconditions and goals. To reach the targeted competencies of communication, sales and negotiation skills, we will embark on a multi-phase initiative:
Outline specific initiatives you will undertake. Ensure that you outline how those initiatives connect with your preconditions and goals.To reach the targeted competencies of communication, sales and negotiation skills, we will embark on a multi-phase initiative:
Content
Participation in online learning courses to expose sales team members to the techniques of diagnosing and creating a sales strategy.
Participation in online learning courses to expose sales team members to the techniques of diagnosing and creating a sales strategy.
Content
Group workshops with their peers to enable them to practice their communication and negotiation skills.
Group workshops with their peers to enable them to practice their communication and negotiation skills.
Content
Connecting each participant to a mentor who will meet regularly with them to enable them to seek feedback and advice. Creating a social network to enable team members to share successes and resources.
Connecting each participant to a mentor who will meet regularly with them to enable them to seek feedback and advice. Creating a social network to enable team members to share successes and resources.
Content
Develop specific metrics based on your goals. Ensure that you have a variety of metrics at each level and step of your initiative. Make sure you can assess specific factors so that you can, in turn, make specific improvements to your learning initiatives.We will measure the success of our initiative on a monthly basis with these metrics:
Develop specific metrics based on your goals. Ensure that you have a variety of metrics at each level and step of your initiative. Make sure you can assess specific factors so that you can, in turn, make specific improvements to your learning initiatives.We will measure the success of our initiative on a monthly basis with these metrics:
Content
Set expectations. Ensure that you and your colleagues understand and agree on goals and reasonable expectations for results.We expect to see some change in the metrics in the first two-three months of the initiative, but we believe that we will see significant improvement within six months as sales team members have an opportunity to implement and improve their new skills in cooperation with their peers. We will use data from assessment and from performance to continually create online courses and group workshops targeting areas of weakness. To the extent possible, we will use internal experts to lead workshops both in person and virtually.
Set expectations. Ensure that you and your colleagues understand and agree on goals and reasonable expectations for results.We expect to see some change in the metrics in the first two-three months of the initiative, but we believe that we will see significant improvement within six months as sales team members have an opportunity to implement and improve their new skills in cooperation with their peers. We will use data from assessment and from performance to continually create online courses and group workshops targeting areas of weakness. To the extent possible, we will use internal experts to lead workshops both in person and virtually.
Content
Sales team’s self-assessment of communication, sales and negotiation skills on a detailed scale
Sales team’s self-assessment of communication, sales and negotiation skills on a detailed scale
Content
Sales manager and VP of Sales assessment of individual and team performance
Sales manager and VP of Sales assessment of individual and team performance
Content
Monthly recurring sales per salesperson and as a group; the rate of change in monthly recurring sales
Monthly recurring sales per salesperson and as a group; the rate of change in monthly recurring sales
Content
Profit growth
Profit growth

Content

In closing, this example is just the beginning. Check out the Theory of Change website, which provides excellent case studies from a variety of social and job training programs.

In closing, this example is just the beginning. Check out the Theory of Change website, which provides excellent case studies from a variety of social and job training programs.

Content