Decoding and Assessing Assessments

The timing for Sunday morning’s session, “Decoding the Black Boxes of Assessment Tools,” led by Sharon Wingron, CPTD, founder and CEO of DevelopPEOPLE, is by design, as she provided tips for attendees ahead of the EXPO floor opening, arming them with criteria for choosing assessments.

Wingron kicked off her session by posing a quick assessment of the audience, asking them to identify as a square, circle, triangle, or a squiggly line. Afterward, Wingron said she made up the “assessment” to convey the critical point of choosing an assessment tool—make sure it is grounded in science.

A topic front and center of everyone’s minds these days, Wingron queried of attendees, “Is artificial intelligence science based?” An assessment tool isn’t necessarily an excellent one because the vendor is using AI, she warns.

Wingron presented a definition of assessment: “The evaluation or estimation of the nature, quality, or ability of someone or something.” An assessment typically includes a self-evaluation response but may also feature a rater or observer response, and it is psychometrically solid.

Before you begin your vendor search, Wingron urged, understand what it is you want the assessment to do, such as using it for development, emotional intelligence, teambuilding, selection, or training, among the other possibilities.

Wingron built on audience responses to continue to make her case. For example, when asking about when not to use an assessment, one attendee mentioned that an assessment may not translate culturally and may contain biases. To ensure that the assessment is valid in the language in which you plan to administer it, ask potential vendors to validate your choice. Is the tool user-friendly—easy to understand, easy to administer, and a reasonable length so to as avoid cognitive overload and user fatigue? Another instance where you may not want to use an assessment is if you aren’t able to invest the time or resources in the tool long term to make it worth the investment.

Beyond asking a potential vendor about the science behind their assessment tool (consider asking them for research or a technical manual), other considerations and questions include ease of administration, access to results (How quick is turnaround and who can see results? For example, you may not want users to see the full range of the assessment results.), price, the vendor’s previous experience, and quality of customer service.

Additional criteria when choosing an assessment tool include:

• How do you plan to measure success? What is the time frame for those results?

• Is the tool still relevant (for example, is the research on the tool’s validity recent)?

• Do you want to potentially train someone in house on administering the assessment or seek a consultant?

• Will you be able to scale the tool if you are using it for team building or performance across the organization?

• Does the tool align with accessibility and accommodation standards?

Whatever assessment tool you use, use it as part of your suite of information as one data point. Note that, as Wingron stressed, using an assessment for selection and hiring is not necessarily legally defensible.

Read more about ATD25 at conferencedaily.td.org.