Top
1.800.628.2783
1.800.628.2783
ATD Links Archive
Issue Map
ATD Links Archive
howto.fw.png
ATD Links

Selecting and Evaluating SMEs

The selection and evaluation of SMEs are generally parallel activities since they are based on a set of criteria related to several key observable and measurable standards expected in SMEs. To even a greater degree than in some choices made in the design process, making sure you have articulated your expectations, in both securing and evaluating your SME, is a necessity.

SME qualifications are best divided into two general categories: content specific and noncontent specific. This allows a thorough and comprehensive 360-degree scan of both the professional criteria associated with content knowledge and the numerous noncontent skills deemed necessary for SMEs in the design environment.

Content-based criteria

When looking at content knowledge, you need to be more than one-dimensional in your assessment of SMEs. Content knowledge is a complex and often confusing set of skills that rests atop a foundation of five multidimensional aspects of achievement.

You will need to review and determine the level of expertise based on at least these five elements of subject matter knowledge in each SME:

  • relevance of experience
  • depth of experience
  • timeliness of experience
  • location of experience
  • training/teaching experience.

While on the surface these may not seem to be related to an extent that really allows discretionary judgment of an individual SME, drilling down into each of the criteria nets a wealth of information on which to base valid and documented decisions.
Relevance of experience

While this may seem deceivingly obvious on the surface, relevance is the first obstacle that each SME must pass through to be considered qualified. Hopefully you don’t assume that every SME in a content area is qualified for your specific content needs in a project. One size does not fit all, or perhaps even most, of your content knowledge needs; don’t allow yourself to be lazy in this judgment.

Relevance in our SME assessment is the ability of a content expert to share content knowledge in a specific content area with relative ease. To be considered to have relevant experience, a SME must also have a documented body of work in the specific content area you are working in.

For example, you might need a content expert in a specific engineering process, and while an engineer with experience in a related field might be easier to secure, you need to determine if a particular candidate can cover your content without compromising your expectations.

Depth of experience

When looking at depth of experience in a SME, it may be tempting to think that depth and relevance might seem like a difference without a distinction, but this is a costly mistake. Depth in SMEs is fundamentally their ability to drill down to the lowest point of detail necessary for your content requirements.

For example, if you are working on a project related to the printing business and color selection, the candidate who knows that a professional color chart contains 1,114 colors is preferable to one who thinks that the primary colors of red, green, and blue are all you really need to know about color. This same principle applies to all issues associated with depth—how deep can a SME go into the details?

Timeliness of experience

Our third area of assessment in choosing SMEs is sometimes one of the most important to you as an instructional designer because having dated content appear in a new course or program is one of the cardinal sins of training. Ensure that you have the most recent and relevant content by working with SMEs who are current in their knowledge.

The shelf life of your content is something you need to determine quickly. Well-established and stable content that only changes incrementally over time is less affected than state-ofthe- art content that may change on an hourly basis. If your knowledge base is considered “just in time,” then your SMEs need to be operating at that same speed.

Advertisement

Determining the criticality of timeliness for your needs is the first decision to make. Imagine you are working on a project that requires the most recent statistical data such as economic reports relating to unemployment, job creation, and productivity.

Choosing a SME with a PhD who teaches economics at a major university, but has little knowledge of this week’s employment numbers would not be a better option than choosing an economist working with the data as they are released. By the same token, your college professor acting as a SME may be best suited for a course in economic theory.

Location of experience

Thinking about geography when selecting SMEs might seem puzzling, until you consider that even minute changes in location can have a major effect on content. Depending on your project, failing to account for the location of an expert’s experience can either marginally or completely affect your work. Let’s look at several examples.

If you are working on a project to train paramedics that has a regional or national scope, each jurisdiction potentially has different skill requirements, licensure requirements, and clinical requirements for participants. Some may require 180 hours in clinical practice, and some may require 80. This is a huge difference in terms of designing training. Knowledge of and input from these differing standards are critical.

When designing technical and skills training, geographic location can be a critical element. The elements of geography that affect content include weather variables like temperature ranges and humidity; soil and ground condition variables like clay versus sand; and earthquake, hurricane, flood, and other natural disaster potential.

To an outsider looking in, these variables may seem minor at best, or even unimportant, but nothing could be further from the truth. A content expert who has only worked in a warm climate has no relevant information for cold weather climate situations, and building codes are certainly different in quake-prone areas compared to others with little, if any, quake activity.

These are just examples, but depending on your situation, there could be many differences that you need to represent on your committee. The other regional variation that often comes to the surface in SME committees is jargon. This not only applies to tools and equipment, but also comes up in the names of processes and procedures that are exactly the same, but called one thing in California and another in Connecticut.

This may be a major problem down the line if it isn’t addressed early in the process. You can’t have a large percentage of your end-user population trying to figure out your terminology because you didn’t allow for regional variations in your SME group. One indirect consideration here is that a budget may not allow for the cost of bringing in dispersed committee members. You may have to choose members who can represent the locational variations within the committee or use remote conferencing options.

Training/teaching experience

If you are working on a project that has a training deliverable, having a SME with training experience can be a valuable asset to the process. Knowing what works and doesn’t work during implementation can be a critical added dimension.

At times, it is this SME who eventually teaches the content, and having the connection at this stage of the process may lessen the disconnect issues from the design to implementation stages. Someone possessing these skills is your ideal candidate as a hybrid SME. It will be useful to determine if the training and teaching experiences are relevant to the content and eventual implementation choices.

A career as a classroom teacher may not work to your benefit if you are designing online learning using a learning management system (LMS) since the approaches are different. There are also variations in teaching approaches and philosophy, for example, in higher education versus more general technical training. Having the correct fit is important since experienced teachers and trainers may be set in their views on implementation or they may not work well with your content and population.

Noncontent-based general skills criteria

In addition to the SME content-based criteria, you should be familiar with your SME’s general skills competencies. In a world of equals in terms of content expertise, you may find the noncontent skill set useful in assessing and evaluating your SMEs. While there are a number of criteria you could use in this regard, you may find the most important general skills criteria to consider include communications ability, writing ability, and sociability.

Article is excerpted from ASTD Handbook: The Definitive Reference for Training & Development 2nd Edition (ASTD Press, 2014). In today’s parlance, smart board is the new chalkboard and for the profession of training and development, this new, 2nd edition of the ASTD Handbook is the MUST-HAVE resource for every practitioner. The ASTD Handbook (2nd edition), more than a year in the making, maintains the authentic credibility of ASTD’s first Handbook (2008), incorporates the new competencies of the profession, and includes more than 50 chapters authored by the top professionals in the T&D space.

This all-new material is not just the "best of," but it is the BEST there is. Together with the first edition it represents the essence of the training practice with solid how-to content, plus tools, resources, technology, and more.  Spearheaded by expert trainer and world-renowned author, editor, and speaker, Elaine Biech, this is an essential title for your training library. 

About the Author
Chuck Hodell is the author of the bestselling ATD book ISD From the Ground Up and has been involved in the worlds of training and education for more than 30 years. He has written extensively on instructional design and training-related topics for ATD, including several Infolines. He has enjoyed stints as a musician, police officer, telephone company repair technician, trainer, teacher, and academic administrator. Like many talent development professionals, his first exposure to training and instructional design was as a subject matter expert. He currently serves as associated director of the graduate program in instructional systems development at the University of Maryland Baltimore County. He is also the senior program director for instructional design at the Transportation Learning Center and academic adviser to the International Masonry Institute. Hodell has an undergraduate degree from Antioch University and an MA and PhD from the University of Maryland Baltimore County.
Be the first to comment
Sign In to Post a Comment
Sorry! Something went wrong on our end. Please try again later.