Google Ad #1 - desktop - 728x90

Google Ad #3 - desktop - 728x90

Google Ad #1 - mobile - 300x200

Google Ad #3 - mobile - 300x200

ATD_Half_Page

Googel Ad #1 - Side Bar - Content pages

ATD Footer Logo

R. A. Pearson

Climate Control Critical Factor Of Executive Growth

Marla designs help documents for the software applications her company produces. She sits across from Sean who develops monthly training webinars. Marla would like to “move up” to building instruction like Sean does, but Sean tells Marla that he thinks that the work that Marla does is actually more useful for the company’s users.We could argue the merits of documentation versus training, but there’s always the question of what each user needs at each moment of time. Context is everything.The science of learning is a science that works to understand how we learn and how we can improve learning and instruction. But to be fair, humans have been learning on their own since, well, forever.For instance, if Corinne wants to know how to do something, she often asks the person next to her and then gets on with her job. Yan mentors Morey and they meet most Tuesday afternoons. Marlena uses self-study materials and is going to take the certification exam in less than three weeks.So why do numerous disciplines (neuroscience, psychology, education, computer science, and others) expend enormous energy and resources to figure out how we learn and how to best build instruction? Can’t we just leave people alone?We Learn All the TimeAn interesting study by D.W. Livingstone shows that the majority of workplace learning happens not in formal settings but in informal ones. Numerous studies, including those by Livingstone, Jay Cross, Jane Hart and others, show informal learning as a proportion of all workplace learning at greater than 70 percent.If you haven’t already looked into informal learning, you should, because training professionals need to understand it more. Saul Carliner’s book, Informal Learning Basics, on the subject is a good primer, as are writings by Jay Cross and Jane Hart.You might think that the definitions of learning and instruction are self-explanatory, but actually they aren’t so simple. And they help us see exactly what we are trying to do.Learning, DefinedThere are many definitions of learning. A favorite comes from De Houwer:  “Learning is changes in behavior that result from experience.”Some definitions don’t require changes in behavior, only changes in knowledge. For me, this doesn’t cut it. If you can’t do what you are learning, the learning isn’t quite cooked yet. For example, if you can’t use what you are learning, you are on your way but not there yet.Applied learning requires knowledge and experience with the range of possibilities, of course. As my colleague Catherine Lombardozzi reminded me, people need deep knowledge, not only to follow a script. So if a customer service agent is following a script for handling returns, but she cannot handle the variety of returns, she only knows how to follow a script. She hasn’t really learned the whole returns process.One definition of learning discusses how learning requires the ability to correct errors. Now this is truly deep learning. One of the biggest problems people have is correcting things that go wrong. Misconceptions and mistakes are a big problem in learning. So when defining learning, we may want to determine how far into expertise we are talking about. Are they proficient (competent or skilled) or are they expert?Why Build Instruction?People can and do learn on their own. But we also know that when there are very specific things we want people to be able to do, there are efficient ways to help them. If learning occurs when we interact with our environment, then instruction is the creation of special environments specifically to promote learning.Dr. Richard Mayer, professor of psychology at the University of California, in his book Applying the Science of Learning reminds us we are trying to change behavior as a result of instruction and there are methods to do so that are more effective than others (that’s what the science of learning and instruction are about).This view has very far reaching implications. It means that if we are building “instruction,” we have to impact what people do. That’s a high bar. Although it is may not be practical to expect that everything we do reach that high bar, we ought to reach that bar often.So that leads to two question that we should ask ourselves:How do we infer (assume changes in) learning after instruction?How do we determine if the change has made it from instruction to the job?We will answer these critical questions soon.This is article 1 of Science of Learning 101, and I very much want your comments. Is this helpful to you? Does it answer your questions? Your comments can change the course of this series.ReferencesCross, J. (2007). Informal learning: Rediscovering the Natural Pathways that inspire innovation andPerformance. San Francisco: Pfeiffer.De Houwer, J., Barnes-Holmes, D. & Moors, A (2013). What is learning? On the nature and merits of a functional definition of learning. Psychonomic bulletin & review.Livingstone, D.W. (2001). Adults’ informal learning: Definitions, findings, gaps and future research. NALL Working Paper 21-20010. Centre for the Study of Education and Work, Ontario Institute for the Study of Education, University of Toronto. Retrieved from  Mayer, R. E. (2011). Applying the science of learning. Upper Saddle River, NJ: Pearson.

Science of Learning 101: What’s the Point of Instruction?

The Personal History A Second Report

First Person Shooters represent over a Billion Dollars of R&D

The Worthington Personal History A Report

Personnel Training in a Small World

Sally is the chief learning officer for a large well-known bank. She was recently brought in by her CEO to “professionalize” the Learning and Performance function within the company. Sally inherited a very extensive learning measurement system that rigorously cataloged the results of each training course using Likert-Like questions and numerical averaging.As Sally sifts through the data, she notices something curious. Instructors whose smile sheet ratings were consistently low were still teaching, even after several years of poor results. Sally, of course, is thrilled! She’s been mandated to professionalize the learning function and already she’s found a problem she can fix. She makes a plan. She’ll bring her team together to improve—or fire—the low-performing trainers.Brilliant, right? Maybe! It depends on whether Sally’s assumptions are accurate:Instructors who get the best smile sheet ratings will help their learners achieve the best learning results and the best on-the-job performance. Instructors who get the worst smile sheet ratings will produce learners who achieve the worst learning results and the worst on-the-job performance.Interestingly, Sally’s predecessors never thought to correlate their smile sheet ratings with learning results or on-the-job performance. Sally, like most L&D professionals, doesn’t think of doing this either. She just assumes that smile sheet results are related to learning outcomes. Indeed, as an industry, we tend to assume that smile sheets are telling us something important about the successes and failures of our training efforts. Are we right? Is Sally right? And does it really matter, anyway?In 2013, ATD and i4cp teamed up to examine how organizations measure the effectiveness of the learning programs. The research, reported in The Value of Learning: Gauging the Business Impact of Organizational Learning Programs, found that Level 1 smile sheets were by far the most common method of measuring our learning solutions. One implication from these findings is that we, as workplace learning professionals, get most of our feedback from smile sheet data.Consequently, because feedback enables improvement, the validity of smile sheet data is critical. In other words, if we’re not getting good data from our smile sheets, we’re not going to be able to create maximally effective learning.Research on Smile SheetsSo, what does the research say? Are smile sheets effective in giving good feedback? The short answer is No. Two meta-analyses in the scientific literature covering more than 150 research studies found that smile sheets are virtually uncorrelated with learning results (see Further Reading below). Specifically, both meta-analyses found that smile sheets were correlated with learning results with a correlation of r = .09.Statisticians tell us that correlations below r = .30 should be considered weak correlations. Therefore, a correlation of r = .09 is practically no correlation at all! It would be like correlating smartphone usage with hair color.Bottom line: Sally’s assumptions were faulty. Our assumptions, as an industry, have been faulty for decades. We’re all caught in the delusion that smile sheets tell us something important, when in fact they may regularly mislead us.A Smarter Smile SheetWow! What should we do then? Should we throw out our smile sheets? Should we collect data to be polite and then ignore the data? I don’t think so. After thinking about the smile sheet problem for a decade or so, I’ve come to the conclusion that if L&D professionals redesign smile sheets from the ground up—based on the science of learning and guidelines from the measurement field—we can build effective smile sheets.In Performance-Focused Smile Sheets: A Radical Rethinking of a Dangerous Art Form, (more info available at www.SmileSheets.com), I describe how to build a much more effective smile sheet. There are two critical aspects.First, questions should be based on the science of learning. Because we’re asking that our smile sheets tell us about learning, it makes sense to base our smile sheet designs on learning science. Second, we have to do a better job of designing our smile sheet questions. Most of us use Likert-like scales or numerical items, both of which are fundamentally inappropriate for smile sheet use. We also have to help our learners make good smile sheet decisions. According to research from Brown, Roediger, and McDaniel (2014), as well as Kirschner and van Merriënboer (2013), learners don’t always know their own learning. Because of this, asking them questions about their learning may give us faulty information—unless we design our questions to support their smile-sheet decision making. Too many of our questions ask learners to answer questions they are not prepared to answer.Here are some of the major areas that smile sheet questions should target:ability of learners to successfully apply what they learned to their work learners’ full comprehension of what was presented to them learners’ capability to remember what they’ve learned level of motivation that learners will bring to implement what they’ve learned post-training resource that will guide and enable learners to persevere in putting their learning into practice.Most smile sheets do a poor job of targeting these issues, which is unfortunate because it is through these types of research-inspired factors that learning effectiveness can be discerned.Bottom LineMost of our smile sheets are ineffective and provide poor feedback. Fortunately, we can build better ones. Of course, smile sheets will never give us all the information we need. It’s critical that we also measure learning by focusing on learner comprehension, decision making, and memory, as well as measure application by examining the factors and results produced in the way of on-the-job performance.While smile sheets are never enough, most organizations will continue to use them, so we ought to make them as effective as possible. Scientific research makes it clear that our current smile sheet efforts are ineffective. Fortunately, the science of learning and wisdom from the measurement field give us hope for better smile sheets.Like what you read? Join us at the ATD 2016 International Conference & Exposition in Denver, CO, May 22-25 to hear Will Thalheimer speak.Further ReadingAlliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341-358.Brown, P. C., Roediger, H. L. III, & McDaniel, M. A. (2014). Make it stick: The science of successful learning. Cambridge, MA: Belknap Press of Harvard University Press.Kirschner, P. A., & van Merriënboer, J. J. G. (2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48(3), 169-183.Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., & Zimmerman, R. D. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93, 280-295.

Are Smile Sheets Supported by Research?

R. Karl Hebenstreit is a certified executive coach, leadership and organization development consultant, and international speaker with more than 25 years experience coaching leaders and their teams (from individual contributors to CEOs in myriad industries and sectors) to work better together and consistently exceed their organizations’ goals. He holds a PhD in Organizational Psychology and has authored three books: The How & Why: Taking Care of Business with the Enneagram (now in its third edition), Nina and the Really, Really Tough Decision (available in English, Spanish, French, and Greek), and Explicit Expectations: The Essential Guide & Toolkit of Management Fundamentals. Karl is motivated by seeing his clients achieve their full potential, professionally and personally. 

R. Karl Hebenstreit 

Kate McGrath is an HR Consultant with the U.S. Office of Personnel Management (OPM). As a performance management consultant, Kate has worked on and led a variety of projects including reviewing and consulting on SES, SL/ST, and Non-SES performance plans, delivering performance management training at the Eastern Management Development Center, facilitating focus groups, developing standard assessments, leading the development and training on critical leadership competencies, and facilitating position classification. Kate assisted in the R&D effort to identify and develop telework services, and managed the pilot project to implement and test the telework services. Kate holds a MS in Industrial/Organizational Psychology and a BS in Psychology from Missouri State University.

Katie McGrath

categories.lvl0

Search results for 'R. A. Pearson'

Categories

Refine By