This discussion reviews elements of the study for "Time to Develop One Hour of Training," as well as the most common question asked based on the information shared in August 2009: "Why are the numbers so high?"
Since the article "Time to Develop One Hour of Training" was published, it has received a fair amount of attention, which is helpful for the field because acknowledging the need for better metrics around training and e-learning production benefits the industry. Notably, the most common question we are asked when we report or discuss our findings is: "Why are the numbers so high?"
But are the numbers really high? Do the numbers reported reflect a problem with the instructional systems design process or are other factors impacting the numbers? What role does client management or client understanding play in the development of an "hour" of training or e-learning?
As we've digested the data and spoken to developers and designers in the field, what we've begun to realize is that the development and production hours are high due to issues not directly related to the development of the instruction itself. For example, managing the client and their comprehension of the ultimate outcomes of the process and dealing with client delays and working with hectic schedules all add time to the project.
To determine the factors that lengthen the time to create one hour of instruction, let's start by looking at a few issues that impact the time for development; external and internal alike. At the end of each section are "Reduction Recommendations"; these are best practices for ensuring organized, accountable, and proactive projects that hopefully have fewer hours when completed.
Providing better client orientation to the process
The 2009 survey asked respondents to list factors that impacted development time of a project. A frequency count determined that "scope creep" (36 percent of qualitative responses) was the largest factor. The scope creep was primarily due to client changes that were "unexpected," "last minute," or "after sign-off."
The second largest factor was due to the SME or content reviewer (31 percnet of qualitative responses). The causes for the delay ranged from lack of availability or access to the reviewer, overdue reviews, and the sudden addition of more reviewers commenting after a portion of the project was approved. When late reviewers are allowed to make changes to approved material - project time increases. An overarching aspect to these review issues was "client expectations" (42 percent of qualitative responses), lack of knowing what they want and why they want it. The first indications of this type of problem usually occurs during an initial review of content; SMEs/reviewers will see what they do not like and comment constructively (or in some cases destructively). Occasionally, they'll even review tone, style, instructional strategies, and content chunking (items in which they have little expertise).
Instructional designers and developers are not miracle workers, but we are fundamentally educators. We analyze information and then regroup it to make it comprehensible so that a learner can better perform their job, so we should be able to use the same techniques to educate our clients and SME/reviewers. We should be able to provide an explanation of what we do, the overall process, SME expectations, how the client fits to that process, and their responsibilities. We need to explain how they can positively or negatively impact the project. We need to educate them in a way that helps them see the big picture of the formal instructional design process.
Reduction recommendation: Reassess how projects are onboarded with clients, old and new. Commit to a policy that all projects move forward only after a briefing with the client and the right client personnel. Provide either a presentation, a tutorial, or even brochure-ware on "this is what we do and this is how you fit into the process," along with a description of the stakeholder, SME/reviewer, and client sponsor roles and responsibilities. Have the client and respective stakeholders sign-off, which keeps a level of quality and shared responsibility for the success of the project. Twenty extra minutes during the kick-off meeting can potentially save hundreds of hours of aggravation and finger pointing down the road.
Building a stronger communication process
Given that the previous topic stems from a need to educate the client on our process, we cannot stop at just providing a basic orientation. After we start the process we must ask ourselves, "What are we doing to maintain comprehension and a sense of ownership and responsibility by the client for the success of the project throughout its duration?" If the client is overdue on materials or adds a new reviewer after approval, what steps are in place to communicate the impact of these actions and when are those potential risks communicated? Is it after the fact or too far ahead or not at all?
The idea behind communication is to pulse the client to obtain:
- how the client or external team is feeling about the progression of the project
- gaps in understanding current responsibilities or tasks
- identification of any potential problems or roadblocks
- other risks such as discovering the SME is unavailable due to another project emergency or a disagreement among SMEs concerning content.
When we communicate with the client we are also sharing as much as we are acquiring. We leverage communications to provide:
- status updates on the project.
- reminders for next steps and responsible parties to those tasks
- client, SME, and stakeholder action items and deliverable dates
- planning for future tasks and milestones.
Part of effective communication is the frequency of communication and taking the initiative to act on possible project risks. If communications are only occasional and actions are not taken towards communicated risks, a project can easily escalate; being exposed to higher risks, which in turn, increases project hours.
Misconceptions can also wreak havoc in communications; internal and external clients tend to think that they have very little involvement in a training development project. Designers and developers cannot just collect information during the initial meetings and come back later with the perfect piece of training. Although many clients would love that solution, collaboration needs to occur.
But clients are not always at fault; designers and developers have been known to keep moving a project forward assuming the client does not have any concerns because they have not asked any questions or shown any interest. Neither party knows what the other is thinking, comprehending, doing/not doing, etc. At best we may have an idea based on our last communication and that might not being saying much depending on when the last communication occurred and what, exactly, was discussed.
Reduction recommendation: Examine your current communication processes both internally and externally. Consider conducting a communication audit. How long does it take information to travel from the client or a team member to the right people? How long does it take after receipt for a response and does the response ensure resolution or other adequate information to fulfill the requests of the sender?
Ensure that the communication plan reflects that needs of the project. A new client may require extra email or phone calls to remind them of next steps or to see how they are handling their responsibilities within the project. A more seasoned client may not need as much day-to-day communication. They may be better served by providing weekly awareness of project actions to be taken by them as next steps. It is also important to consider different communication levels for an ongoing project versus and old project or a project that has new members transitioning into the team.
In addition, discussing an optimal communication plan with your client allows the client to have more shared responsibility for the project, and makes them feel more "part of the team" from the onset, providing a level of comfort to establishing effective communication channels from the start.
Recognizing the potential for change management
If we are lucky we work on a training project with a team of people from the client-side who all fulfill a specific role and remain dedicated to the project for the duration. Some may think we just wrote about training development Utopia. That's our point - when does Utopia happen and does it with any frequency? Most reading this will think "No, I cannot even count how many times SME's changed or a stakeholder was dropped from a project." Given that reality, recall how the loss of a SME or other project role on the client side was handled.
Change happens at any time for any reason. Ask questions like; is an appropriate process in place for onboarding new team members and transitioning others off? Is there a process for handling internal change management to ensure client projects do not come to a halt? Change management does not just account for the loss of a physical resource, it also accounts for technological changes, scope of work modifications, and timeline adjustments.
Reduction recommendation: Taking the change management leadership position provides a sense of security for the client in times of transition; they will realize that not all is doomed. This is assuming change management leadership gives you more control over the project and its ultimate outcome. Work with the client to discuss the best methods for ensuring that as the transition occurs the project continues to move forward.
With internal changes, audit the current process and if there isn't one, create a general plan that shows responsible job roles for initiating and implementing change and how those changes are to be communicated and disseminated. Having a plan in place for internal use will lend itself very effectively to helping clients manage their own situation.
Another internal step to take is to inventory resources. When taking on or continuing project work with a client it is always wise to investigate what additional resources can and will be available during the project duration if a ramp up is needed.
Adding a tandem technology mitigation process
When dealing with e-learning projects or implementing an online, behind-the-fire-wall social network it requires an understanding of the technology necessary to enable the learning process. Even if it is the client's responsibility to ensure technological compatibility between the LMS and a developed course or with the existing network, we would do good to serve our customers by guiding them through a process that mitigates these technological issues.
Recall that we could be working with clients who may not understand the technological implications of SCORM compatibility or of using a video-based message to teach about compliance. Often clients do not really understand technology but are often tasked with getting online instruction to work on their existing system with little regard to the difficulties that can sometimes ensue.
Reduction recommendation: Develop a technology mitigation process by starting an analysis that identifies and addresses technology-specific concerns for the project and involves the clients IT folks as soon as possible. Have this run as a parallel process that starts along with the rest of the project. After completing analysis, lay out plans to manage each issue that is relevant to the overall timeline and milestones.
Make sure the technology will work when it is needed as opposed to waiting to see if works when the training is built and ready to be deployed. Early assessment also ensures that the IT personnel within the organization will have acquired the knowledge necessary to manage and monitor the training technology. Doing small "proof of concept" modules can be a great help in testing the technological infrastructure and exposing problems early.
Any project manager worth her salt will tell you accountability of hours is critical to determining costs of a project, mitigating risk, and maintaining budgets for present and future projects. The more structure for reporting tasks on projects, the better estimators and data will be garnered for use currently and in the future. A ballpark estimate is more realistic then a statement of "it depends." The ability to replace a soft non-committal styled statement for a confident approximation is appealing to both the client and the designer and developer of the instruction.
We recognize that self-reporting will always skew project numbers. Self-reporting is flawed on many levels. However, the only alternative is to hire a person per respective team member to sit and log exactly what that team member does and for how long - not a practical solution.
Self-reporting can be inconsistent for many reasons. Some examples:
- accidental misreporting of hours
- uncertainness of how to categorize work that was done given the existing reporting parameters
- team members may lose track of time and forget to report work
- a team member could be new to a process and does not factor their hours for the learning curve; or they could account for those numbers and they could be high and atypical for a specific task
- a team member could be overly confident in completing work and underestimate how many hours they needed to complete the task
- time spent thinking about the project at home or on the drive to work may not by attributed to the project.
Reduction recommendation: With accountability we need to look at not only what we are doing presently but what we did previously. Auditing ourselves and past projects may help us keep better track of project hours now but will also provide clues as to why a project was not completed in the estimated time frame. Taking a lesson from a past experience could be one of the most beneficial actions a company can take for themselves if they want to tighten up the hours it takes to produce projects and manage clients.
In addition to the audits, examine the processes that are in place for reporting hours and look at what is being reported and by whom. If you tracked a new employee and their hours on projects for the next three years you would have a good set of data that would hopefully show the employees marked improvement and reduction in production hours for certain tasks. These numbers can help for project estimations and to also identify whether employees are mastering skills or not.
Working with clients is a never-ending process of education, communication, and proactive measures. But, we should also remember that our focus is not only on our client but on ourselves and how we operate, produce, and complete projects. We too have areas that we need to refine and reel in with respect to how long it takes us to create one hour of training. Using sets of common processes that can be applied to any project, and modified easily to tailor to the exact needs of a particular client, can control some of the "loose cannon" factors that impact the time it takes to develop one hour of instruction.
Maybe the numbers will always be high. One thing is clear it takes time to make good instruction. One cannot expect quality, effective training with unrealistic deadlines, little client participation or assistance, and incomplete processes to be developed rapidly. There is no "Easy" button for designing and developing training. But does there really need to be one?
Taking a look at how we apply accountability for project responsibilities and evaluating how effective they are would be are good first step towards making the entire process smoother for both your team and the client. Incorporating sound project processes and holding to them for even the smallest projects will not only improve the production time, but the relationship with the client, in addition to layering the project with proactive methodologies for mitigating even the smallest risks. All of this culminates into having project work history that we can use to mine for data that can and will eventually give us a more definitive answer.
Robyn A. Defelice is a consultant in the field of instructional technology providing senior level instructional design, project management, and business administration services to her clients. She is currently a doctoral candidate of the Communications Media and Instructional Technology program at Indiana University of Pennsylvania. Contact her at firstname.lastname@example.org or see what she has cooking at www.dishingdesign.com.
Karl M. Kapp is a consultant, scholar, and expert on the convergence of learning, technology, and business operations. He also is the author of Learning in 3D and Gadgets, Games and Gizmos for Learning. His background teaching e-learning classes, knowledge of adult learning theory and experience training CEOs and front line staff provides him with a unique perspective on organizational learning. His experience with technology companies and high-tech initiatives provides him with insights into the future of technology. He shares those insights and perspectives through writing, consulting, and coaching with clients in the field of e-learning. Follow Karl on his blog at www.karlkapp.blogspot.com, or contact him at email@example.com.