0000015e-ab3e-db83-a5ff-effe7c2b0000-authorID

laura-quinn.jpg

Laura Quinn

Fall 2017 - CTDO Magazine

Members of the C-suite must work as one to execute the organization’s vision.

Executive Teams at Their Best

At the beginning of the past two years, I’ve written a post on important points I think you should know. In 2021 I mentioned the importance of context, emotion, and learning to learn. In 2022 I discussed some of the newer cognitive science models of cognition—including distributed, social, and embodied cognition—as well as the role of mental models. This year, I’d like to focus on a trend that we should extinguish: the belief that we are formal, logical, reasoning beings.The cognitive revolution transcended the view of behavioral psychology; we couldn’t talk about what went on in the brain. While many valuable insights were generated by that scientific paradigm, including the role of performance objectives, the move to thinking we could isolate necessary components provided further advances. Results that have stood the test of time—the recognition of sensory, working, and long-term memory as well as core cognitive processes like attention, elaboration, and retrieval—were identified.We tend to adopt existing models as frameworks for thinking about the brain, and the cognitive revolution was no exception. For example, consider the computer as a general-purpose information processing system. It made sense to adopt that model to reflect our abilities to process information and complete a variety of tasks. Thus, the dominant framework for thinking about thinking was to view our brains as formal, logical, reasoning machines.Such models, with some assumptions like mapping the world into formal representations, accomplished impressive tasks. Systems built upon that model could break down problems into components and solve them. They could even learn. Artificial intelligence (AI) advanced as a result. So-called “expert systems” were built upon interviews with the best practitioners. Sadly, these models failed in practical situations, leading to an AI winter. This was a disconcerting time, when behavior naturally observed in humans did not reflect these symbolic systems.The Postcognitive RevolutionOne of the most impressive capabilities of our thinking is the ability to organize the world into flexible frameworks. What David Rumelhart called schemas, Marvin Minsky called frames, and Roger Schank called scripts all reflect our ability to infer knowledge about the world. Thus, we had representations such as wheels that scaled up to vehicles but differently for automobiles versus bicycles. Symbolic logic struggles with this, however. Attempts were made to adapt, such as Lofti Zadeh’s Fuzzy Logic and Douglas Hofstedter’s Slipnets, but suffered in practice.An alternative framework emerged that could be characterized as “subsymbolic,” operating below the symbolic level. John Holland, Keith Holyoak, Richard Nisbett, and Paul Thagard created a genetic algorithm approach in Induction, which borrowed mutation from biology, but wasn’t aligned with neuroscience. Rumelhart, along with Jay McClelland, proposed connectionist models instead in their two-volume set Parallel Distributed Processing with a neuroscientific inspiration. These models were found to exhibit rational behavior similar to humans. They represented the same types of behaviors, including errors, that were empirically obtained from psychology research.These neural net models created not only a psychological revolution, capturing elements of cognition, but also a renewed basis for artificial intelligence, underpinning much of today’s work in machine learning.A Cognitive LegacyOne small legacy of the formal, logical, reasoning model of cognition, however, seems to linger. It appears that many of our practices in learning and development (and business in general), still seem to be based on this idea. This is a potential problem.If we’re formal logical beings, there are several things we should be able to assume. For one, if we give people new information, they’ll appropriately adjust their behavior. Moreover, the person with the greatest access to information should be the one to make a decision, and that decision will be the best one possible. We see this manifest in several ways.For one, our training frequently resembles the presentation of new information with an expectation of behavior change. Too often, we see bullet points of information extracted from (and even presented by) experts. After this, there’s an expectation that we’ll do something differently. Similarly with annual or quarterly reviews. People need feedback at the appropriate time and support for change. We don’t just change when we wish to; it takes time and practice.We also see decisions made by individuals without consultation. Folks will decree a new approach because it’s been determined by the person at the appropriate decision point. Yet the evidence suggests that we get better outcomes when we bring more people into the decision-making process, such as in Keith Sawyer’s Group Genius. When we follow practices that help us suitably explore a problem individually, and then collectively brainstorm solutions, test, and adapt, the outcomes are better.A Problematic LegacySadly, there is also evidence that this approach to work isn’t working. We see it in the outcomes of learning investments, and when we evaluate organizational success as an outcome of our approaches, we see that more informed ones work better.In the 2012 Harvard Business School working paper The Great Training Robbery, Michael Beer, Magnus Finnstrom, and Derek Schrader cite metanalyses that suggest only 10 percent of training investment has an impact. There may be multiple causes for this, but it does suggest that we’re wasting money with our approaches. The Serious eLearning Manifesto suggests that much of traditional learning is based on information presentation, so it’s safe to say that we’re not doing as well as we could.Similarly, efforts like Laura Overton’s Toward Maturity and Lauri Bassi’s Human Capital Capability Scorecard suggest that organizations do better when they’re better aligned with how people think, work, and learn. Organizational performance reliably improves with better employee approaches.Overall, the continued use of ineffective practices undermines our organizational success. We need to effectively achieve our organizational goals, which includes ensuring we can optimally execute what we know we need to do and continually innovate when we have to invent or adapt.Moving ForwardWhat should you do differently?Truly understand how your brain actually operates. Know the basics of the human information processing loop and extensions for learning and performance.Avoid the myths that continue to hinder our industry.Conduct deep needs analyses to find important performance gaps and their root causes. There are many causes of problems besides a lack of knowledge or skills. Stop thinking that if you give people knowledge, behavior will change.Leverage external resources. Decision making is best done collectively and iteratively. To do otherwise is to minimize your potential.Move from legacy practices to evidence-based practices. There are vested interests that continue to exist based on perpetuating approaches that aren’t effective. Many products that are quite lucrative—profiling instruments and development programs —aren’t effective.Start measuring impact. This doesn’t mean what people prefer (there’s no correlation with what people think of a program and its impact). Instead, assess the actual change in the organization. You can’t collect this yourself; you must partner with the business units that require change.Align programs to demonstrable gaps, and then measure until they’ve improved.Ultimately, L&D should be the folks that are responsible for not only optimal execution but also continual innovation. When you’re innovating (problem solving, designing, or researching), you don’t know the answer when you start, so inherently it is learning. It’s informal learning, and no one knows the answer (yet)! Still, it’s an opportunity for L&D to attain a more valuable role in the organization.To make this strategic move and start bringing evidence-based practices from formal to informal learning, take ownership of these practices within your teams. Make it safe to fail; share stories of what you tried; and learn together. Create and understand a learning microculture before taking it to the rest of the organization.Last, identify the organizational legacies that are holding you, and your organizations, back. Then, find alternatives, and implement them. When you’re free of your unexamined past, you can move forward to an evidence-based future that means better outcomes. That’s a trend I’d like to see and encourage you to undertake.

A Trend to Supersede

ATD developed the talent development framework and puzzle to help practitioners understand the different components of talent development and how organizations can build their own frameworks to address their unique needs. This blog series explores each component of the talent development framework, why it’s important to the field, and what resources ATD offers to practitioners who want to learn more.Evaluating Learning ImpactEvaluating learning impact involves the use of learning metrics and analytics to measure the effectiveness of learning solutions. In Utilization Focused Evaluation: The New Century Text, Michael Quinn Patton defines evaluation as “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming.” Evaluating learning impact has long been considered a daunting task. And, as a recent ATD study shows, only 35 percent of talent development professionals “reported that their organizations evaluated the business results of learning programs to any extent.” A different approach is needed. First talent development professionals who want to evaluate learning impact must know what specifically they want to measure and how they want to measure it. According to learning expert Patti Shank, the biggest mistake many make in evaluating the impact of learning is just testing recall. Just because people have learned something does not necessarily mean that they’ll apply it or understand its importance on the job. Real assessment comes from observation, providing feedback, and evaluating results. There are several models to evaluate learning, such as Kirkpatrick’s Four Levels and Phillips’ Evaluation Model. Kirkpatrick’s Four Levels is likely the most commonly known evaluation model, with a four-level program that follows a path of reaction, learning, behavior, and results. The fourth step is the most important, because the results show outcomes and overall effect of the learning in practice. Regardless of the evaluation strategy used, when collecting data, it is also important to use a variety of tools such as surveys, interviews, and analytics to get a full picture. To be competent at evaluating learning impact, the ASTD Competency Study (Updated now to ATD Talent Development Capability Model™)  identifies skills that talent development professionals need to exhibit:• Identify customer expectations.• Select appropriate strategies, research design, and measures.• Communicate and gain support for the evaluation plan.• Manage data collections.• Analyze and interpret data.• Apply learning analytics.• Make recommendations to aid decision making.Measuring learning impact encourages higher-quality work, resource prioritization and allocation, and accountability. Measurement data are invaluable to organizations that are strategically using talent development efforts to achieve growth and efficiency objectives. Competency in evaluating learning impact is imperative to becoming a real business partner in the organization.We’ve assembled some additional resources to explore:• Use analytics to improve your organization’s evaluation of learning. • Discover new ways of measuring learning. • Learn how improving performance management can help learning impact. • Measure learning using evaluation and not testing. • Consider a new way to look at Kirkpatrick’s four levels. • ATD Research: Evaluating Learning: Getting to Measurements That Matter•Evaluating the Impact of Training by Scott B. Parry• For more information on training delivery and other major components of talent development, check out the ASTD Handbook, 2nd Edition (Updated now to the TDBok™ Guide, 2nd Edition). • Take a deeper dive into evaluation with ATD’s Evaluating Learning Impact Certificate Program.

Laura Quinn

About