ATD Links Archive
Issue Map
ATD Links Archive
ATD Links

Isolation of Coaching Effects

Isolating the effects of a coaching program on business impact data is one of the most challenging, yet necessary, steps in the ROI Methodology. When addressed credibly, this step links the coaching process directly to business impact.

Other factors are always present. In almost every situation, multiple factors affect business results. During the time that coaching programs are implemented, many other functions within the organization may be attempting to improve the same metrics addressed by coaching program. For example, marketing and sales programs are designed to affect revenue, and, while coaching may help a marketing or sales employee positively affect revenue, the marketing or sales program itself also may do that. In addition to internal factors, external factors may influence business results during the time that coaching is occurring. 

Without isolation, there is only evidence, not proof. Without taking steps to show the contribution of coaching, there is no clear business linkage. Instead, there is only evidence that coaching may have made a difference. When business results improve during coaching, it is possible that other factors may have contributed to that improvement. The proof comes from isolating the effects of the coaching initiative. 

Other factors have protective owners. The owners of the other functions that influence business results are convinced that their processes or programs make the difference. For example, marketing and sales managers probably firmly believe that any increase in sales is entirely due to their efforts, and they often present a compelling case to management to support their claim. Likewise, the information services department likely believes that their technology changes make the primary difference in efficiency gains within an organization. They, like marketing, often present a compelling case to management. Other processes or programs, such as performance improvement, reward systems, and job redesign, all have protective owners, and their arguments are plausible. Therefore, owners of coaching programs are under pressure to build a credible argument for their case to claim value added to the organization. 

To do it right is not easy. The challenge of isolating the effects of a coaching program is critical and can be done, but it is not easy for complex programs, especially when strong-willed owners of other processes are involved. It takes determination to address this situation every time a return-on-investment study is conducted. 

Common myths about isolating the effects of coaching

Several myths about isolating the effects of a coaching program often create concerns, confusion, and frustration with the process. Some professionals further complicate this matter by suggesting that isolation is not necessary. Here are the most common myths related to isolation: 


Programs such as coaching are complementary to other processes. Therefore, you should not attempt to isolate the effects of coaching. Coaching programs are complementary to other factors, all of which drive results. If a project sponsor needs to understand the relative contribution of a coaching initiative, this issue must be addressed. If accomplished properly, it will show how all the complementary factors are working together to drive the improvements. 

Other functions in the organization do not isolate the effects of their programs and projects. While some functions do not grapple with this issue because they try to make a convincing case that the improvement is related to their own processes, others are addressing the issue. A credible approach to address this issue is necessary. Notice the next time you complete a customer satisfaction survey after you make a purchase or open a new account. Do they ask you why you made the purchase? If yes, they are trying to isolate the results of multiple variables. 

If you cannot use a research-based control group, then you should not attempt this step. Although a well-constructed control group is the most credible approach, it will not apply in the vast majority of coaching situations. Consequently, other methods must be used to isolate effects. The problem does not go away just because it is not possible to use the most desired or favorite isolation technique. The challenge is to find other isolation techniques that are effective and that will work, even if they are not as credible as the control group method. 

The stakeholders will understand the linkage, so you do not need to attempt to isolate the effects of coaching on business impact measures. Unfortunately, stakeholders see and understand what is presented to them. Absence of information makes it difficult for them to understand the linkage, particularly when others are claiming full credit for their improvement. 

Estimates of improvement provide no value. The worst-case scenario is to tackle this issue with the use of estimates from those individuals who understand the process the most. Although this is a last choice, it may provide value and be a credible process, particularly when the estimates are adjusted for the error of the estimate. Estimates are used routinely in a variety of functions throughout organizations. 

Ignore the issue. Maybe they won’t think about it. Stakeholders are becoming more sophisticated on this issue, and they are aware of multiple influences. If no attempt is made to isolate the effects of coaching, stakeholders will assume that other factors have had a tremendous effect, and maybe the entire effect. Thus, credibility of coaching results deteriorates. 

When funding is provided to different functions in the organization—with different process owners—there is always a struggle to show, and sometimes even to understand, the connection between what they do and the results. If you do not tackle this issue, others will—leaving the coaching program with less than desired budgets, resources, and respect. 

Note: This article is excerpted from Measuring the Success of Coaching by Jack J. Phillips, Lisa Ann Edwards, and Patricia Pulliam Phillips.


© 2012 ASTD, Alexandria, VA. All rights reserved.

About the Author
Jack J. Phillips, PhD, is chairman of the ROI Institute and a world-renowned expert on measurement and evaluation. Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers worldwide. Phillips is also the author or editor of more than 100 articles and more than 75 books, including Measuring the Success of Leadership Development: A Step-by-Step Guide for Measuring Impact and Calculating ROI (ATD Press). His work has been featured in the Wall Street Journal, Bloomberg Businessweek, Fortune, and on CNN.
About the Author
Lisa Ann Edwards is the founder of Bloom Coaching Institute, an organization that advances coaching effectiveness through research, tools, training and consultation. Edwards’s coaching work has demonstrated as much as a 251 percent return on investment and has been shown to lift employee engagement nearly 20 percent. As head of Talent Management for Corbis, a Bill Gates' privately owned global media company, Lisa was responsible for designing and implementing effective talent development solutions such as leadership development, management development and coaching programs to ensure talent engagement, improve talent retention and serve to feed the talent pipeline. Lisa is a frequent contributing author to trade publications and has authored or contributed to the following books: Measuring the Success of Coaching: A Step-by-Step Guide to Measuring Impact and Calculating ROI (ASTD Press, 2012); Keep Looking Up (2010); Managing Talent Retention: An ROI Approach (2009) and has contributed case studies or chapters to five additional books. Lisa earned a Master of Science in Experimental Psychology from Southern Methodist University and is a certified coach from The Hudson Institute of Santa Barbara.
About the Author
Patti Phillips is president and CEO of the ROI Institute and is the ATD Certification Institute's 2015 CPLP Fellow. Since 1997, she has worked with organizations in more than 60 countries as they demonstrate the value of a variety of programs and projects. Patti serves on the board of the Center for Talent Reporting, as Distinguished Principal Research Fellow for The Conference Board, and as faculty on the UN System Staff College in Turin, Italy.

Patti has written and edited numerous books and articles on the topics of measurement, evaluation, and ROI. Recent publications include Measuring the Success of Leadership Development, Making Human Capital Analytics Work, Measuring the Success of Learning Through Technology, Measuring the Success of Organization Development, and Measuring Leadership Development: Quantify Your Program's Impact and ROI on Organizational Performance.
Be the first to comment
Sign In to Post a Comment
Sorry! Something went wrong on our end. Please try again later.