How to Explore Experts' Mental Models

Content

Talent development professionals often need to conduct assessments of one kind or another, and seldom move beyond the tried-and-true techniques of surveys, interviews, and focus groups. When trying to probe for deeper information, though, these techniques may not be enough. When something more robust is needed, turning toward techniques used by researchers can be helpful.

Talent development professionals often need to conduct assessments of one kind or another, and seldom move beyond the tried-and-true techniques of surveys, interviews, and focus groups. When trying to probe for deeper information, though, these techniques may not be enough. When something more robust is needed, turning toward techniques used by researchers can be helpful. 

Content

Case in Point

Case in Point 

Content

Robin Grenier and Dana Dudzinska-Przesmitzki have recently published an article in Human Resource Development Review that describes a process for eliciting mental models. In talent development, the techniques they describe are useful in obtaining nuanced detail regarding how people make judgements or decisions—information that subject matter experts have trouble articulating to others. Their technique is easily implemented, and its effectiveness for the purpose is grounded in proven research methods.

Robin Grenier and Dana Dudzinska-Przesmitzki have recently published an article in Human Resource Development Review that describes a process for eliciting mental models. In talent development, the techniques they describe are useful in obtaining nuanced detail regarding how people make judgements or decisions—information that subject matter experts have trouble articulating to others. Their technique is easily implemented, and its effectiveness for the purpose is grounded in proven research methods. 

Content

Grenier and Dudzinska-Przesmitzki recommend that practitioners conduct a three-part elicitation process for each subject matter expert whose mental model you want to document.

Grenier and Dudzinska-Przesmitzki recommend that practitioners conduct a three-part elicitation process for each subject matter expert whose mental model you want to document. 

Content

Step #1: Ask your subject matter expert to list 20 concepts or ideas related to the topic at hand. For example, you might ask what a term means, the factors that enter into consideration in decision making or judgments, or related questions relevant to your need. Perform this step via email, collecting 20 items for each question. This step serves as a mental warm-up exercise, and elicits some of the top-of-mind ideas related to the mental model.

Step #1: Ask your subject matter expert to list 20 concepts or ideas related to the topic at hand. For example, you might ask what a term means, the factors that enter into consideration in decision making or judgments, or related questions relevant to your need. Perform this step via email, collecting 20 items for each question. This step serves as a mental warm-up exercise, and elicits some of the top-of-mind ideas related to the mental model. 

Content

Step #2: Craft and conduct a semi-structured interview on the topic. Ask detailed questions related to your need. You will likely need to probe some of the items that were drawn out in Step #1.

Step #2: Craft and conduct a semi-structured interview on the topic. Ask detailed questions related to your need. You will likely need to probe some of the items that were drawn out in Step #1. 

Content

Step #3: Schedule an additional interview to have your subject matter expert draw the mental model. During this interview or exercise, give your subject matter expert sticky notes or cards on which you have printed the responses from your emailed questions, as well as additional words or phrases that seemed important based on interview responses. Ask the subject matter expert to use the sticky notes to draw a concept map that illustrates the thought process you are trying to document. (The SME doesn’t have to use all of the notes, just the ones that seem to best represent parts of his or her mental model.)

Step #3: Schedule an additional interview to have your subject matter expert draw the mental model. During this interview or exercise, give your subject matter expert sticky notes or cards on which you have printed the responses from your emailed questions, as well as additional words or phrases that seemed important based on interview responses. Ask the subject matter expert to use the sticky notes to draw a concept map that illustrates the thought process you are trying to document. (The SME doesn’t have to use all of the notes, just the ones that seem to best represent parts of his or her mental model.)

Content

To facilitate this process, give the SME a prompt of the kind of decision or judgement you are exploring and ask the SME to “think aloud” the process for considering the situation and coming to conclusion, using the sticky notes as appropriate. The SME should organize selected sticky notes in relationship to one another and write down the connections between and among them (such as this causes that, these concepts are sub-criteria of other criteria, these concepts are steps in a process, and so forth). You may need to assist the SME by asking specific questions as she works (for example, What is the relationship between this and that? Where does this concept fall in your process?). Keep your questions as open as possible, and don’t impose your reasoning on the SME.

To facilitate this process, give the SME a prompt of the kind of decision or judgement you are exploring and ask the SME to “think aloud” the process for considering the situation and coming to conclusion, using the sticky notes as appropriate.  The SME should organize selected sticky notes in relationship to one another and write down the connections between and among them (such as this causes that, these concepts are sub-criteria of other criteria, these concepts are steps in a process, and so forth). You may need to assist the SME by asking specific questions as she works (for example, What is the relationship between this and that? Where does this concept fall in your process?). Keep your questions as open as possible, and don’t impose your reasoning on the SME. 

Content

For example, the figure below represents a training design process as described by an employee. She was asked, how do you create a training event? The dashed lines represent information that came out during the mapping process or interview. The grey areas were developed in step one when asked to list the components of the training design process. The map draws out how the employee thinks about this process, and subtle details could be explored further with additional conversation.

For example, the figure below represents a training design process as described by an employee. She was asked, how do you create a training event? The dashed lines represent information that came out during the mapping process or interview. The grey areas were developed in step one when asked to list the components of the training design process. The map draws out how the employee thinks about this process, and subtle details could be explored further with additional conversation. 

Content

Figure 1: Using a "Thinking-Out-Loud" Concept Map to Share Procedural Knowledge

Figure 1: Using a "Thinking-Out-Loud" Concept Map to Share Procedural Knowledge 

Content

During these elicitation processes, you can ask probing questions about the meaning of concepts, why concepts were selected for inclusion, and the significance of the links described. You will also notice if there are inconsistencies how the subject matter experts describe aspects of the model, and you’ll have opportunity to explore those further.

During these elicitation processes, you can ask probing questions about the meaning of concepts, why concepts were selected for inclusion, and the significance of the links described. You will also notice if there are inconsistencies how the subject matter experts describe aspects of the model, and you’ll have opportunity to explore those further. 

Content

Bottom Line

Bottom Line 

Content

At the conclusion of these procedures, you’ll have a detailed representation of the mental model experts use in the situation you are exploring. The information you gather here gives you rich detail that can be used in analyzing your situation, or crafting development programs intended to pass on expert knowledge.

At the conclusion of these procedures, you’ll have a detailed representation of the mental model experts use in the situation you are exploring. The information you gather here gives you rich detail that can be used in analyzing your situation, or crafting development programs intended to pass on expert knowledge. 

Content

The process is called a Multi-Method Mental Model Elicitation, and it works because it triangulates verbal and graphical representations of a mental model that helps subject matter experts articulate deeper thought processes that tend to be more tacit. With each step of the process, the representation of the mental model becomes more robust as the expert builds on previous responses and adds explanation and detail.

The process is called a Multi-Method Mental Model Elicitation, and it works because it triangulates verbal and graphical representations of a mental model that helps subject matter experts articulate deeper thought processes that tend to be more tacit. With each step of the process, the representation of the mental model becomes more robust as the expert builds on previous responses and adds explanation and detail. 

Content

For more on this process, see “A Conceptual Model for Eliciting Mental Models Using a Composite Methodology,” by Robin S. Grenier and Dana Dudzinska-Przesmitzki. Human Resource Development Review 14(2), June 2015. Published by Sage. You can reach Dr. Robin Grenier at [email protected] .

For more on this process, see “A Conceptual Model for Eliciting Mental Models Using a Composite Methodology,” by Robin S. Grenier and Dana Dudzinska-Przesmitzki. Human Resource Development Review 14(2), June 2015. Published by Sage. You can reach Dr. Robin Grenier at [email protected]. 

Content

Sage is offering access to the article for free for a limited time.

Sage is offering access to the article for free for a limited time. 

Content

Editor’s Note: This is the first post in a series highlighting research from the journals of the Academy of Human Resource Development (AHRD). In partnership with ATD, AHRD is committed to sharing useful research with the practitioner community.

Editor’s Note: This is the first post in a series highlighting research from the journals of the Academy of Human Resource Development (AHRD). In partnership with ATD, AHRD is committed to sharing useful research with the practitioner community.