In the movie Apollo 13, one of the scientists supporting the astronauts' return to Earth interrupts NASA Flight Director Gene Kranz, insisting "Power is everything! Without it, they don't talk to us, they don't correct their trajectory, they don't turn the heat shield around." He then lists the critical functions the astronauts need to survive re-entry.
While power may not be everything in the aftermath of a disaster, it is crucial. Without it, hospitals can't treat patients, shelters can't help victims, police and firefighters struggle to provide support, and communications fail. And on a more personal level, families wrestle to cope with their normal daily activities, delaying any return to normalcy.
For the Department of Energy's Emergency Support Function (ESF) 12, part of the Federal Emergency Management Agency (FEMA) National Response Framework, power is everything. Restoring power after a disaster is the goal of this team of about 90 dedicated responders who provide emergency support on a volunteer basis in addition to their regular jobs. The ESF-12 team has helped in several disasters, including the 2011 tornado in Joplin, Missouri; the 2010 earthquake in Haiti; the 2009 tsunami in American Samoa; the 2007 tornado in Greensburg, Kansas; and Hurricane Katrina in 2005.
Like most highly skilled and trained responders, ESF-12 members are required to take regular refresher training to ensure they are current with all regulations, to familiarize them with new processes and procedures, and, perhaps most important, to provide the opportunity to practice in preparation for the real thing. Prior to 2010, this annual training took the form of three days of traditional classroom training combined with a live in-person simulation. Although the training was effective, sending 90 people from 14 offices around the country to a central location was becoming increasingly costly—both because of travel costs and because of lost productivity and disruption for the employees.
To mitigate costs while maintaining the high quality of the training, DOE created an online learning solution. It partnered with Vivid Learning Systems, a national provider of online compliance safety training programs.
Creating a Two-Part Solution
To create online training that would meet the needs of the first responders, the first step was to understand those needs. Analyzing the structure of the original program, the Vivid Learning Systems team was able to glean that the first portion of the training—the traditional classroom training—was designed to impart knowledge, while the simulation component was designed to provide practice. Thus these two critical features became the base of a two-part online training program. Part one was a self-paced, asynchronous component that enabled responders to learn the information. Part two was a synchronous online component that would bring the responders together in a virtual space to get the critical practice they needed.
Imparting the Knowledge
Effective classroom training involves immersing learners in the topic, providing interaction, and engaging learners socially. Classroom training provides social motivation to participate through discussions with other learners and instructors. Asynchronous online learning, however, provides none of that social motivation to participate and requires that the learner be self-motivated enough to persist through the training. Therefore we had to consider these factors when designing the asynchronous online portion of the training program for ESF-12: How would we keep the learners engaged? How will we ensure they persist through the course?
After six months of research, development, and fine-tuning, Vivid Learning Systems produced a set of three online training modules designed to be as realistic and relevant as possible, with interactivity built in to heighten learner engagement. Another feature of the modules that both enhanced engagement and ensured mastery was the use of built-in case studies, scenario-based knowledge checks, and quizzes. For the simulation, participants would demonstrate mastery through completion of assessment features.
To complete the knowledge acquisition component of the training program, learners would log in to the custom-built training portal on their own schedule and work through the three modules, stopping and starting at will. The first two modules comprised these established learning objectives:
- recognize the team lead's role in establishing the ESF-12 workstation
- identify the team lead's role in facilitating energy issues with FEMA logistics, various partners, and ESF-12 counterparts
- identify the team lead's role in ensuring FEMA public assistance issues are resolved with ESF-12 counterparts
- identify the team lead's role in recommending and developing an exit strategy for demobilization
- identify how the Infrastructure Security and Energy Restoration (ISER) division is organized
- describe the response operations procedures
- identify the highlights and lessons learned from the American Samoa tsunami and the Haitian earthquake response events
- recall the planning and preparation activities necessary for response deployment.
The third module provided the training required for the learners to navigate the synchronous training platform so they could participate actively and effectively in the synchronous training exercise.
In converting the traditional classroom-based training program to an online blended learning experience, ESF-12's training team insisted that the responders get the opportunity to participate in a live simulated disaster response. This had been by far the most useful part of the training program, according to the team, and it provided the best preparation possible for responders.
Knowledge acquisition alone—whether classroom-based or online and regardless of how effective—doesn't equal the power of an activity in which learners are required to synthesize and incorporate knowledge and apply it. That's where the learning simulation comes into play. The previous incarnation of the training program enabled participants to integrate learning by allowing them to practice in a setting that mimicked a real disaster but was safe enough to make mistakes in. This was clearly too powerful a component of the training program to lose, and Vivid Learning Systems created a solution to orchestrate a disaster response simulation without physically bringing everyone to the same place: A hands-on response scenario that responders could participate in without leaving their desks, which would be aided by a synchronous training delivery platform.
The disaster response simulation began with responders accessing the exercise by logging into an online classroom through a link on the training portal. Once they "arrived" in the classroom, they were presented with background information on a fictional natural disaster affecting the island of Guam. The responders received the same kind of briefing they would get in a real disaster situation, including some verbally delivered notes and a simulated newscast that communicated the conditions of the disaster, in this case a tsunami.
Once they had the back story, it was time to get to work. Responders virtually "deployed" to several sub-classrooms designed to mimic response centers—specific areas from which responders orchestrate power restoration during a real disaster. Based on the information they had received, the team of responders at each response center created reports assessing the damage caused by the disaster and developed plans to restore energy in the affected areas. Throughout the exercise, highly trained ESF-12 responders simulated the roles of the different organizations involved in the disaster response process, such as FEMA, by periodically entering the various virtual response centers to deliver new information.
After completing the simulated disaster response training, responders were able to demonstrate mastery of these tasks:
- Disseminate information to the other activated response locations and communicate among response centers regarding issues during an event requiring multiple FEMA regions to be activated.
- Handle questions and actions that come to the ESF-12 workstation. Demonstrate the ability to filter, upgrade, and downgrade process requests as appropriate.
- Participate in an ISER conference call.
- Gather and assess information at the request of the response center or other FEMA counterparts. Assess the impacts to the energy infrastructure and act as a technical advisor regarding energy issues.
- Prepare a field input report for their location and submit to the other response centers and the emergency response organization director.
- Identify and implement an appropriate battle rhythm (meeting schedule, reporting requirements, or daily activities) for their assigned response location.
- Collaborate with the team lead and the subject matter experts within the FEMA organization to order and deliver supplies to the joint field office located on a remote island.
- Coordinate with the FEMA public assistance branch on the supplies and financial aid processes that support the restoration of the island energy infrastructure.
- Coordinate power and energy issues with the energy sector owners and operators to ensure that temporary and long-term power supply is restored.
- Facilitate industry requests by coordinating existing or new mutual aid agreements with other public power providers.
Measuring the Results
Such a comprehensive restructuring of the ESF-12 training program requires evidence that showed it had achieved the goal of cost reduction while maintaining the learning effectiveness of the original format. To show this, Vivid Learning Systems assessed the program on four levels: learner reaction, knowledge, behavior, and cost and organizational benefits. The results were excellent.
To evaluate learner reaction, learners completed a questionnaire and ranked a series of assessment items on a scale of one to five. ESF-12 has a history of delivering training that meets the highest evaluations from learners. The new training format was no exception; in fact, average scores increased; learners indicated the online training seemed more lifelike than the previous face-to-face program.
Measuring knowledge took place through the asynchronous course module assessment exercises and end-of-course quizzes. Learners had to complete the assessment exercises correctly and pass the end-of-course quiz to receive credit for the course and be eligible to participate in the simulation exercise. All responders successfully completed all these activities.
Learners were given the opportunity to demonstrate their mastery of the behaviors needed for successful deployment through their completion of the simulation exercise. As noted in the descriptions above, learners successfully demonstrated the required behaviors throughout the exercise.
Finally, the Vivid Learning Systems team was able to demonstrate that the most notable benefit of the ESF-12 blended learning approach was cost savings. ESF-12 saved more than $200,000 in its first year of the program solely by eliminating costly travel and lodging expenses for more than 90 people.
In addition to the travel savings, ESF-12 saved thousands more from the decrease in time responders had to spend away from their jobs. An exact cost savings is difficult to determine because of the different pay rates of the responders. However, using averages of three hours for completion of the synchronous disaster response, three hours for the completion of the learning modules, and a learner's annual salary of $60,000, the cost of labor for the total training decreased from an estimated $54,000 to $13,500. This was a conservative estimate of $40,500 in cost savings on labor alone.
Taking Learning to the Next Level
The cost savings and success of the first iteration of the online blended learning program co-developed by DOE and Vivid Learning Systems have led to a strong partnership and an exciting search for the next level of excellence. By incorporating new technologies and leveraging more interactivity, ESF-12's annual training program promises to keep these responders prepared to cope with any disaster at less cost.