October 2021
Issue Map
Measuring tape in the shape of a question mark
TD Magazine

Stumped on How to Measure DEI Training?

Friday, October 1, 2021

Use this process to structure and evaluate diversity, equity, and inclusion initiatives.

Diversity, equity, and inclusion constitute a process and system of integrating new metrics and practices into how your organization does its work. For simplicity, we'll reference this process as a program or initiative throughout this article, but that does not mean DEI is a one-and-done effort.

Advertisement

"The key is to recognize that DEI is a long-term journey and that we cannot cram all our goals into one program. Then we become more prepared and confident to affect change because we can focus on this program and the impact it will seek to have," says Crystal Kadakia, an organizational change consultant and author.

To position your efforts to deliver predictable, measurable results, your DEI initiative should have aligned goals, desired workplace behaviors, and learning objectives.

Results first

The initial step in designing a DEI program that works is to clearly define the desired program results. Rather than putting the weight of this important task on one individual and to reduce biased designs, create a committee that is representative of the company's departments, levels, and employee backgrounds.

"No one person in an organization should own DEI, and it can't just be punted to human resources, although HR plays a critical role in helping the organization achieve strategic DEI goals," explains DEI consultant and author Maria Morukian. "DEI efforts require active and visible support from leadership, engagement from across the organization, and accountability at all levels."

The committee's first task is to create the vision of what success will look like. "A vision of success for the DEI team moves beyond high-level organizational values to naming specific systemic and cultural changes that newly developed programs will support," says Kadakia.

For example, a general organizational vision statement could read: "We are diverse in our staffing and inclusive in the way we operate through all job functions and levels. We actively seek out diverse approaches and points of view because we believe that drives innovation. We foster both a top-down and grassroots approach to our work to equitably hear voices at all levels to ensure accountability for fostering an equitable culture at all levels of the organization."

Once the vision is clear, it is critical to identify specific, observable, measurable results for DEI.

Setting diversity goals. Diversity is the practice of including people from different backgrounds including race, ethnicity, gender, gender identity, sexual orientation, age, social class, physical ability or attributes, religious or ethical values system, national origin, and political beliefs. Most employers find defining desired results for diversity easy. Examples include:

  • Percentage of diverse candidates who apply, interview, or are hired for a position
  • Percentage of diverse employees at each level
  • Diversity of employees promoted to higher-level or advanced positions
  • Retention of diverse employees
  • Employee engagement among diverse workers
  • Diversity of executive board members

Setting equity goals. Defining equity is important because it is not the same as equality, which means everyone is given the same resources or opportunities. Equity recognizes that each person has different circumstances and allocates the resources and opportunities needed to reach an equal outcome. Equity helps to achieve diversity goals by making it equally likely for minority groups to achieve the same level of success.

Set equity goals by making an honest and comprehensive assessment of the systemic factors contributing to barriers to success for marginalized or underrepresented identities. Examples of equity goals include:

  • Salary parity for diverse employees at the same level
  • Equal opportunities available for all employees
  • Equitable performance management and succession planning practices

Setting inclusion goals. Inclusion is a culture expressed through implicit actions and underlying beliefs that ensure everyone feels a deep sense of belonging, is valued for their unique skills and characteristics, and can contribute fully to organizational life.

In a workplace environment, inclusion also has a more literal meaning of having people at different levels of the company heard as much as those in positions of power. Examples of inclusion goals include:

  • All voices heard equally during committee meetings
  • Frequency and types of opportunities for employees to provide input on decisions that affect them directly
  • Positive employee survey results on questions related to feeling respected and valued for their contributions
  • Developing communication systems that connect frontline employees to top-level leaders
  • Building organizational capability for behaviors that foster psychological safety

Actions speak louder than words

Companies often do a good job of sharing their DEI vision and goals in public statements, internal documents, and training programs. Those are only symbolic actions and are unlikely to create any change unless employers define the critical behaviors individuals must perform on the job and set up a system of support and accountability to ensure those actions occur.

Defining and implementing a Level 3 behavior plan is the biggest requirement for organizational DEI success, and it's traditionally the item companies most frequently overlook in general. What that means is if you want people to do something and have positive, measurable change occur, you need a multifaceted Level 3 plan. As represented in the 70-20-10 model, on-the-job experiences are the biggest source of learning for employees.

DEI includes complex, sensitive, and longstanding cultural issues. And while programs may not have the power to change what every employee thinks or feels, they can define acceptable behavioral standards and consistent implementation of them.

Critical behaviors are the few, key ones that employees will have to consistently perform on the job to bring about the targeted outcomes. Such behaviors convert abstract concepts like DEI into observable and measurable actions that employers can track, coach, and reward.

For example, to make committee meetings equitable by having all voices heard, a critical behavior could be to require the use of a structured meeting agenda that gives each meeting participant the same number of minutes to voice their views on a topic. Another critical behavior could be to have committee decisions made by popular vote. If pressures to align with senior committee members are strong, conduct votes anonymously.

Employees should have their own set of no more than three critical behaviors defined for the DEI initiative or a specific DEI-related project that are specific to their job function (if they differ from job to job). If people cannot remember the behaviors, they will not perform them. That may sound limiting, but the same critical behavior may address more than one DEI goal, and they can differ from department to department or job to job.

After defining the critical behaviors for the people most directly involved in creating the desired DEI results, the next step is to set up required drivers to ensure they happen consistently.

A blended approach yields maximum results

Required drivers are processes and systems that monitor, reinforce, encourage, and reward performance of critical behaviors on the job. If there is a secret weapon to program success, this is it. Even when people know what to do, chances of them doing it are much more likely if others support their efforts and hold them accountable.

Performance support and accountability is where to invest the largest share of resources to maximize program outcomes and success. Each critical behavior or group of behaviors for a given group of employees should have a corresponding set of required drivers. The good news is that required drivers are not mutually exclusive; the same driver could support multiple behaviors, and the same driver may also cover multiple purposes.

Use an instructional design approach that includes a holistic view of learning and performance with a broad view of what constitutes learning and a focus on what happens in the on-the-job environment.

In the example of making committee meetings equitable, required drivers could include:

  • Providing a meeting agenda job aid (reinforcing)
  • Assigning a committee member to work with meeting leaders if they need help (encouraging)
  • Publicizing equitable meetings in company news (rewarding)
  • Requiring meeting minutes for meetings related to company objectives and having a committee member review them for compliance (monitoring)

Notice how one driver can cross over into multiple categories. For example, a meeting agenda job aid that reinforces equitable time for each person to speak also serves as a monitoring tool and encourages people to conduct their meetings in the prescribed way. Publicly recognizing teams that are conducting their meetings to standard rewards them and provides encouragement for others to follow suit. Make sure the required drivers package covers all four categories—monitoring, reinforcing, encouraging, and rewarding—for mission-critical initiatives, but that does not mean you need four separate and exclusive buckets.

DEI initiatives are important and therefore warrant a required drivers package with all four categories. The DEI committee can suggest a universal set of required drivers, and subject matter experts in each area can review them and add or customize items for each group. Setting up a required drivers package is not difficult; however, the bigger challenges are to ensure implementing it is realistic and to assign the indicated roles and responsibilities.

A big reason DEI initiatives will not be successful with one individual at the helm is because the required drivers package needs support in all areas. Companies cannot hold one person responsible for every employee doing the right thing on the job. And by the same token, one person cannot support and hold accountable every employee either.

Accountability for DEI can be challenging. Many people dislike confrontation and tend to avoid giving feedback. DEI often feels amorphous and brings an emotional charge that can seem daunting. Build an environment where feedback is well practiced and perceived as a supporting tool, not a punishment. That directly relates to inclusion goals about making all voices heard and establishing comfortable communication channels.

Now design the training

When you receive a request for a new training program, you may instinctively open your laptop and start outlining and compiling slides. However, resist that temptation until you have clear definitions of the desired outcomes and, more importantly, the critical behaviors and required drivers package.

Defining critical behaviors up front creates the framework for your learning objectives and the instructional design plan. Starting with what you want people to do on the job establishes the outline for the training content.

Advertisement

During training, make behavioral expectations clear and explain how the company will support people afterward and hold them accountable. Design activities around practicing the behaviors and using the job aids.

For a sensitive topic such as DEI, provide an opportunity for discussion and voicing of concerns. If the training solution is self-paced, provide a way for people to get answers to their questions or discuss concerns with someone in real time.

Some employers are likely feeling great pressure to launch a DEI program quickly, but skipping over defining the outcomes and the Level 3 package will not work for this initiative—or any initiative. Invest the time now to make sure the resources you invest create outcomes and true organizational value.

Your finest hour

In the movie Apollo 13, flight director Gene Kranz overhears two NASA directors discussing the low survival chances for the crippled spacecraft. "I know what the problems are, Henry," one of them says. "This could be the worst disaster NASA has ever experienced."

"With all due respect, sir," Kranz intervenes, "I believe this is going to be our finest hour."

The approach you and your organization take to creating true DEI is a blueprint for the success of anything important. There must be positive, inclusive, accountable actions that will bring about improved performance, contribute to your company's mission, and fulfill the initiative's intent.


The Current State of DEI&B Education

Does your organization have formal goals or outcomes for its diversity, equity, inclusion, and belonging efforts, including DEI&B education?

No: 
21%

Yes, tied to measurable key performance indicators:
36%

Yes, but not tied to measurable KPIs:
43%

SOURCE: DEI&B EDUCATION FOR EMPLOYEES, ASSOCIATION FOR TALENT DEVELOPMENT, 2021

Sample DEI Goals and Associated Critical Behaviors

Diversity goal: Diversity of workforce matches area demographics at all levels.

Equity goal: There is salary parity for employees at the same level.

Critical behaviors:

  • Conduct annual audit of employee diversity and salary parity in all pay grades.
  • When multiple job candidates are qualified for a job and the employee cohort within the position's pay grade is not as diverse as the community, hire the minority group candidate to fill the open position.
  • Conduct semiannual employee engagement and equal opportunity surveys (including identifying demographic questions).

Inclusion goal: Committee meetings have equitable contributions from all participants.

Critical behaviors:

  • Mandate a structured meeting agenda with equal time allotted to each participant to speak on the topic.
  • Make committee decisions by popular, anonymous vote.
About the Author

James Kirkpatrick is a thought leader and change driver in training evaluation and the creator of the New World Kirkpatrick Model. Using his 15 years of experience in the corporate world, including eight years as a corporate training manager, he trains and consults for corporate, government, military, and humanitarian organizations around the world. He is passionate about assisting learning professionals in redefining themselves as strategic business partners to become a viable force in the workplace. His latest book, co-authored with Wendy Kirkpatrick, is Kirkpatrick’s Four Levels of Training Evaluation (ATD Press).

About the Author

Wendy Kirkpatrick is a global driving force of the use and implementation of the Kirkpatrick Model, leading companies to measurable success through training and evaluation. She is a recipient of the 2013 Emerging Training Leaders Award from Train­ing magazine. Together Jim and Wendy are co-owners of Kirkpatrick Partners.

2 Comments
Sign In to Post a Comment
Thank you for this article. I especially appreciate your examples and approach to setting and measuring goals.
Thank you, Jane! Crystal Kadakia and Maria Morukian were extremely helpful with their input on the examples.
Sorry! Something went wrong on our end. Please try again later.
Sorry! Something went wrong on our end. Please try again later.
Sorry! Something went wrong on our end. Please try again later.