With a critical function about to undergo significant change, one federal agency developed a formula for accurately estimating the resources needed for training.
Resource planning is particularly important when significant training is anticipated in the near future and resources are limited. Effective planning, however, can be difficult, especially for training for jobs that do not yet exist or will undergo substantial changes.
To meet future demand for air travel, the Federal Aviation Administration (FAA) is implementing new technology that will result in substantial changes in the way air traffic controllers carry out their job responsibilities. It is anticipated that a significant amount of new training will be needed for controllers.
Because detailed information about the FAA's new technology was not yet available, and the technology would likely change over time, the agency partnered with the American Institutes for Research (AIR) to find a method for estimating training resources that was flexible and could accommodate these changes. It was determined that the use of algorithms, which consist of rules or instructions that define the operations required to reach a solution, would be the best strategy.
The first decision point was to determine which resources must be estimated. A logical starting point is to follow the steps in the ADDIE process for instructional systems design—analysis, design, development, implementation, and evaluation—and account for the resources that would be needed during each of these stages.
For our purposes, we collapsed analysis, design, and development into a single algorithm under the "development" umbrella. Then, we built additional algorithms to estimate the resources required for implementation and evaluation. Finally, because the FAA planned to roll out the technology in phases, with functionalities to be added over time, we built a fourth algorithm to capture the maintenance portion.
To ensure that each algorithm captures all of the resources required for that phase of training, it is important to identify the tasks associated with each training phase. For example, the implementation phase for this particular project includes 10 tasks, including scheduling training time for trainees and delivering training.
Numerous factors may affect the resources required for training. We identified the factors likely to affect resources in each phase of training and used them as the foundation for the algorithm. For example, implementation would require labor from trainees, instructors, and support personnel, but the amount of labor would be decided by the complexity of the training content, the delivery method, the number of hours of scheduled instruction, and the number of trainees.
Because such situational factors were likely to change, we opted to calculate labor hours and cost based on various multipliers, such as the estimated hours of instruction and the number of trainees. Examples of multipliers used in the algorithms are below, with some algorithms having more than one type of estimate.
Number of labor hours:
- for each training developer per hour of instruction
- for each trainee per hour of instruction
- for each class per hour of instruction
- for each support person per trainee
- for each training evaluator per hour of instruction
- as a percentage of the original labor hours to develop the training program
- as a percentage of the change in the course per the original labor hours to develop the training program.
- per trainee for the first 24 hours of instruction
- per training tool per number of training tools required.
An example of the various multipliers used in one algorithm is shown in the table on page 22.
Reasonable approaches for identifying the numeric values of multipliers include using existing numbers from your organization, from a similar organization, or from existing research; or you can work with training experts in your organization to build them from scratch. For example, when building the development algorithm, we started with labor hour estimates for developing training programs from existing research and then modified those multipliers slightly to better reflect the job of the controller and the organizational context.
However, no information was available to support the other algorithms. We relied on our team's experience to identify the remaining multipliers. For example, we estimated that support personnel would need approximately 0.17 hours per trainee to schedule their training time and to enter their training completions.
For the algorithms to be applied successfully, we needed to provide easy-to-follow instructions for our end users. To do so, we created a template to use when performing and recording the calculations identified in the algorithms. For example, multiply the duration of the training program by the number of classes by the multiplier in the algorithm to get the total number of hours for instructors to deliver the training. We opted to create the template in Excel. This approach to building instructions constituted a rigorous test of the algorithm, the instructions, and the template.
The final step in the process is to apply the algorithms. Using the instructions and the template made this a straightforward process that resulted in specific information about the resources that will be required to support new training for controllers. We recorded the results in our templates in preparation for communicating them to stakeholders.
Results and lessons learned
Considerable resources will be required to support the FAA's new training initiative, and the resource estimates will allow the agency to engage in appropriate strategic planning. As we designed algorithms to aid in resource estimation and strategic planning, the following lessons were learned along the way.
Algorithms are portable. Algorithms can be used to estimate resources for existing or future training initiatives, and they can be modified easily.
Timing is everything. Building algorithms in advance of any major organizational change means that you will be ready to develop resource estimates when those changes are announced.
More is better. The more information you have about the training program—such as the delivery method, number of hours of instruction, number of trainees, their existing knowledge and skills, and the complexity of the content—the more accurate your estimates will be.
Strike a balance. Building and applying the algorithms before everything is known about the proposed changes, as in our case, will result in rough estimates. However, if you wait until you have confidence in all of the parameters, you might be too late to provide strategic guidance. Striking a balance is the secret to creating an estimate that will be useful.
Choose your experts carefully. Certain factors have to be estimated, such as the training delivery method and the number of hours of instruction. You will need training experts who are experienced and can provide well-informed, accurate estimates.